large language model – Page 10 – Andrew Fairless, Ph.D.

What I Read: LLM Training, RLHF

By Andrew Fairless on October 30, 2023October 5, 2023

https://magazine.sebastianraschka.com/p/llm-training-rlhf-and-its-alternatives LLM Training: RLHF and Its AlternativesSebastian Raschka, PhDSep 10, 2023 “RLHF is an integral part of the modern LLM training pipeline due to its ability to incorporate human preferencesContinue readingWhat I Read: LLM Training, RLHF

What I Read: LLMs, single example

By Andrew Fairless on October 23, 2023September 9, 2023

https://www.fast.ai/posts/2023-09-04-learning-jumps/ Can LLMs learn from a single example?Jeremy Howard and Jonathan WhitakerSeptember 4, 2023 “We’ve noticed an unusual training pattern in fine-tuning LLMs. At first we thought it’s a bug,Continue readingWhat I Read: LLMs, single example

What I Read: GPT-4, 8 Models in One

By Andrew Fairless on October 12, 2023September 4, 2023

https://www.kdnuggets.com/2023/08/gpt4-8-models-one-secret.html GPT-4: 8 Models in One; The Secret is OutBy Dr. Mandar Karhade, MD. PhD., Sr. Director Data Strategy and Advanced Analytics at Avalere HealthAugust 24, 2023 “The Mixture ofContinue readingWhat I Read: GPT-4, 8 Models in One

What I Read: LLM research

By Andrew Fairless on October 10, 2023September 4, 2023

https://huyenchip.com//2023/08/16/llm-research-open-challenges.html Open challenges in LLM research8/15/23 8:00 PM “Never before in my life had I seen so many smart people working on the same goal: making LLMs better… I noticedContinue readingWhat I Read: LLM research

What I Read: Models Memorize or Generalize?

By Andrew Fairless on October 2, 2023September 4, 2023

https://pair.withgoogle.com/explorables/grokking/ Do Machine Learning Models Memorize or Generalize?By Adam Pearce, Asma Ghandeharioun, Nada Hussein, Nithum Thain, Martin Wattenberg and Lucas DixonAugust 2023 “Large language models can certainly seem like theyContinue readingWhat I Read: Models Memorize or Generalize?

What I Read: scaling laws, cross-entropy loss

By Andrew Fairless on September 28, 2023September 4, 2023

https://epochai.org/blog/the-direct-approach The Direct ApproachMatthew Barnett and Tamay Besiroglu “Empirical scaling laws can help predict the cross-entropy loss associated with training inputs, such as compute and data. However, in order toContinue readingWhat I Read: scaling laws, cross-entropy loss

What I Read: LLMs in Planning

By Andrew Fairless on September 25, 2023September 4, 2023

https://youtu.be/BmyB-4S9QuYAvenging Polanyi’s Revenge (Exploiting the Approximate Omniscience of LLMs in Planning..)Subbarao KambhampatiAug 1, 2023 “…we have to stop confusing the impressive form of the generated knowledge for correct content, andContinue readingWhat I Read: LLMs in Planning

What I Read: Economic Case for Generative AI

By Andrew Fairless on September 21, 2023September 4, 2023

https://a16z.com/2023/08/03/the-economic-case-for-generative-ai-and-foundation-models/ The Economic Case for Generative AI and Foundation Modelsby Martin Casado and Sarah Wang “…we believe there’s a very strong argument that generative AI is not only economically viable,Continue readingWhat I Read: Economic Case for Generative AI

What I Read: LLM-based Products

By Andrew Fairless on September 19, 2023September 4, 2023

https://eugeneyan.com/writing/llm-patterns/ Patterns for Building LLM-based Systems & Productseugeneyan “This post is about practical patterns for integrating large language models (LLMs) into systems and products.”

What I Read: Attention Off By One

By Andrew Fairless on September 13, 2023September 4, 2023

https://www.evanmiller.org/attention-is-off-by-one.html Attention Is Off By OneBy Evan MillerJuly 24, 2023 “…the current generation of AI models have an off-by-one error in a crucial place, and it’s making everyone’s Transformer modelsContinue readingWhat I Read: Attention Off By One

Tag: large language model