https://amaarora.github.io/posts/2024-07-04%20SWA.html Sliding Window Attention, Longformer – The Long-Document TransformerAman AroraJuly 4, 2024 “…we will look take a deep dive into Sliding Window Attention (SWA) that was introduced as part of
What I Read: What’s Fair, What’s Hard
https://www.quantamagazine.org/the-question-of-whats-fair-illuminates-the-question-of-whats-hard-20240624 The Question of What’s Fair Illuminates the Question of What’s HardLakshmi ChandrasekaranJune 24, 2024 “Computational complexity theorists have discovered a surprising new way to understand what makes certain problems
What I Read: Detecting hallucinations, LLMs, semantic entropy
https://oatml.cs.ox.ac.uk/blog/2024/06/19/detecting_hallucinations_2024.html Detecting hallucinations in large language models using semantic entropySebastian Farquhar, Jannik Kossen, Lorenz Kuhn, Yarin Gal19 Jun 2024 “We show how one can use uncertainty to detect confabulations.”
What I Read: AI Engineers, Search
https://softwaredoug.com/blog/2024/06/25/what-ai-engineers-need-to-know-search What AI Engineers Should Know about SearchDoug TurnbullJune 25th, 2024 “Things AI Engineers Should Know about Search”