reinforcement learning – Andrew Fairless, Ph.D.

What I Read: Adaptive LLMs

By Andrew Fairless on April 30, 2025January 30, 2025

https://sakana.ai/transformer-squared Transformer²: Self-Adaptive LLMssakana.aiJanuary 15, 2025 “Imagine a machine learning system that could adjust its own weights dynamically to thrive in unfamiliar settings, essentially illustrating a system that evolves asContinue readingWhat I Read: Adaptive LLMs

What I Read: Reward Hacking

By Andrew Fairless on April 1, 2025December 21, 2024

https://lilianweng.github.io/posts/2024-11-28-reward-hacking Reward Hacking in Reinforcement LearningLilian WengNovember 28, 2024 “Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards,Continue readingWhat I Read: Reward Hacking

What I Read: passively learned, causality

By Andrew Fairless on December 4, 2024August 26, 2024

What can be passively learned about causality?Simons InstituteAndrew Lampinen (Google DeepMind)Jun 25, 2024 “What could language models learn about causality and experimentation from their passive training?”

What I Read: LLM Pre-training Post-training

By Andrew Fairless on November 18, 2024August 26, 2024

https://magazine.sebastianraschka.com/p/new-llm-pre-training-and-post-training New LLM Pre-training and Post-training ParadigmsA Look at How Modern LLMs Are TrainedSebastian Raschka, PhDAug 17, 2024 “Initially, the LLM training process focused solely on pre-training, but it hasContinue readingWhat I Read: LLM Pre-training Post-training

What I Read: Open-endedness, Agentic AI

By Andrew Fairless on November 13, 2024August 25, 2024

https://press.airstreet.com/p/open-endedness-is-all-well-need Open-endedness is all we’ll need: On “Agentic AI”Air Street Capital, Nathan Benaich, and Alex ChalmersAug 15, 2024 “Agentic systems offer the promise of agents – a software system thatContinue readingWhat I Read: Open-endedness, Agentic AI

What I Read: Contextual Bandit, LinUCB:

By Andrew Fairless on November 5, 2024August 15, 2024

https://truetheta.io/concepts/reinforcement-learning/lin-ucb A Reliable Contextual Bandit Algorithm: LinUCBDJ RichAugust 6, 2024 “A user visits a news website. Which articles should they be shown?”

What I Read: Hidden Infinity, Preference Learning

By Andrew Fairless on October 10, 2024July 14, 2024

https://www.cs.princeton.edu/~smalladi/blog/2024/07/09/dpo-infinity The Hidden Infinity in Preference LearningSadhika MalladiJuly 09 2024 “I demonstrate from first principles how offline preference learning algorithms (e.g., SimPO) can benefit from length normalization, especially when trainingContinue readingWhat I Read: Hidden Infinity, Preference Learning

What I Read: Extrinsic Hallucinations, LLMs

By Andrew Fairless on October 7, 2024July 14, 2024

https://lilianweng.github.io/posts/2024-07-07-hallucination Extrinsic Hallucinations in LLMsLilian WengJuly 7, 2024 “This post focuses on extrinsic hallucination. To avoid hallucination, LLMs need to be (1) factual and (2) acknowledge not knowing the answerContinue readingWhat I Read: Extrinsic Hallucinations, LLMs

What I Read: LLMs train LLMs

By Andrew Fairless on September 4, 2024June 15, 2024

https://sakana.ai/llm-squared Can LLMs invent better ways to train LLMs?Sakana AIJune 13, 2024 “…LLMs themselves have grown increasingly capable of generating hypotheses and writing code. This raises an intriguing question: canContinue readingWhat I Read: LLMs train LLMs

What I Read: Summarization, LLMs

By Andrew Fairless on September 3, 2024June 15, 2024

https://cameronrwolfe.substack.com/p/summarization-and-the-evolution-of Summarization and the Evolution of LLMsCameron R. Wolfe, Ph.D.Jun 03, 2024 “How research on abstractive summarization changed language models forever…”

Tag: reinforcement learning