large language model – Page 11 – Andrew Fairless, Ph.D.

What I Read: Models Memorize or Generalize?

By Andrew Fairless on October 2, 2023September 4, 2023

https://pair.withgoogle.com/explorables/grokking/ Do Machine Learning Models Memorize or Generalize?By Adam Pearce, Asma Ghandeharioun, Nada Hussein, Nithum Thain, Martin Wattenberg and Lucas DixonAugust 2023 “Large language models can certainly seem like theyContinue readingWhat I Read: Models Memorize or Generalize?

What I Read: scaling laws, cross-entropy loss

By Andrew Fairless on September 28, 2023September 4, 2023

https://epochai.org/blog/the-direct-approach The Direct ApproachMatthew Barnett and Tamay Besiroglu “Empirical scaling laws can help predict the cross-entropy loss associated with training inputs, such as compute and data. However, in order toContinue readingWhat I Read: scaling laws, cross-entropy loss

What I Read: LLMs in Planning

By Andrew Fairless on September 25, 2023September 4, 2023

https://youtu.be/BmyB-4S9QuYAvenging Polanyi’s Revenge (Exploiting the Approximate Omniscience of LLMs in Planning..)Subbarao KambhampatiAug 1, 2023 “…we have to stop confusing the impressive form of the generated knowledge for correct content, andContinue readingWhat I Read: LLMs in Planning

What I Read: Economic Case for Generative AI

By Andrew Fairless on September 21, 2023September 4, 2023

https://a16z.com/2023/08/03/the-economic-case-for-generative-ai-and-foundation-models/ The Economic Case for Generative AI and Foundation Modelsby Martin Casado and Sarah Wang “…we believe there’s a very strong argument that generative AI is not only economically viable,Continue readingWhat I Read: Economic Case for Generative AI

What I Read: LLM-based Products

By Andrew Fairless on September 19, 2023September 4, 2023

https://eugeneyan.com/writing/llm-patterns/ Patterns for Building LLM-based Systems & Productseugeneyan “This post is about practical patterns for integrating large language models (LLMs) into systems and products.”

What I Read: Attention Off By One

By Andrew Fairless on September 13, 2023September 4, 2023

https://www.evanmiller.org/attention-is-off-by-one.html Attention Is Off By OneBy Evan MillerJuly 24, 2023 “…the current generation of AI models have an off-by-one error in a crucial place, and it’s making everyone’s Transformer modelsContinue readingWhat I Read: Attention Off By One

What I Read: What Do LLMs Know About Linguistics?

By Andrew Fairless on September 11, 2023August 1, 2023

https://thegradient.pub/what-do-llms-know-about-linguistics-it-depends-on-how-you-ask/ What Do LLMs Know About Linguistics? It Depends on How You AskTerra Blevins7/9/23 1:00 PM “… LLMs are highly sensitive to how the prompt is written…. We explore theContinue readingWhat I Read: What Do LLMs Know About Linguistics?

What I Read: LLMs

By Andrew Fairless on September 7, 2023August 1, 2023

https://willthompson.name/what-we-know-about-llms-primer What We Know About LLMs (Primer)Will Thompson (Twitter)July 23, 2023 “…it is worth reflecting on what we concretely know about LLMs at this point in time and how theseContinue readingWhat I Read: LLMs

What I Read: shape of AGI

By Andrew Fairless on August 29, 2023July 30, 2023

https://windowsontheory.org/2023/07/17/the-shape-of-agi-cartoons-and-back-of-envelope/?1 The shape of AGI: Cartoons and back of envelopeBoaz BarakJuly 17, 2023 “AIs are not “silicon humans.””

What I Read: What we don’t talk about

By Andrew Fairless on August 28, 2023July 30, 2023

https://veekaybee.github.io/2023/07/18/what-we-dont-talk-about-when-we-talk-about-building-ai-apps/ What we don’t talk about when we talk about building AI appsVicki Boykis7/17/23 8:00 PM “From the technical perspective, here’s what we don’t discuss when we talk about deepContinue readingWhat I Read: What we don’t talk about

Tag: large language model