Andrew Fairless – Page 4 – Andrew Fairless, Ph.D.

What I Read: Fine-tuning

By Andrew Fairless on December 11, 2024September 3, 2024

https://openpipe.ai/blog/fine-tuning-best-practices-series-introduction-and-chapter-1-training-data Fine-tuning Best Practices Series Introduction and Chapter 1: Training DataReid MayoAug 1, 2024 “We’ll explore how to choose the best data, common methods for collecting it, and common methodsContinue readingWhat I Read: Fine-tuning

What I Read: passively learned, causality

By Andrew Fairless on December 4, 2024August 26, 2024

What can be passively learned about causality?Simons InstituteAndrew Lampinen (Google DeepMind)Jun 25, 2024 “What could language models learn about causality and experimentation from their passive training?”

What I Read: Evaluating LLM-Evaluators

By Andrew Fairless on December 3, 2024August 26, 2024

https://eugeneyan.com/writing/llm-evaluators Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge)Eugene Yan “After reading this, you’ll gain an intuition on how to apply, evaluate, and operate LLM-evaluators. We’ll learn when to apply (i)Continue readingWhat I Read: Evaluating LLM-Evaluators

What I Read: sparsity, PyTorch, Hadamard product

By Andrew Fairless on December 2, 2024August 26, 2024

https://alexshtf.github.io/2024/07/07/HadamardParameterization.html Alex ShtoffFun with sparsity in PyTorch via Hadamard product parametrizationJul 7, 2024 “The beauty of sparsity inducing regularization is that we let our optimizer discover the sparsity patterns, insteadContinue readingWhat I Read: sparsity, PyTorch, Hadamard product

What I Read: tilted loss

By Andrew Fairless on November 25, 2024August 26, 2024

https://alexshtf.github.io/2024/06/14/Untilting.html Alex ShtoffUntilting the tilted lossJun 14, 2024 “Typically in machine learning we train a model by minimizing the average loss…. The parameter t can be thought as a kindContinue readingWhat I Read: tilted loss

What I Read: Classifying pdfs

By Andrew Fairless on November 21, 2024August 26, 2024

https://snats.xyz/pages/articles/classifying_a_bunch_of_pdfs.html Classifying all of the pdfs on the internetSantiago Pedroza2024-08-18 “How would you classify all the pdfs in the internet? Well, that is what I tried doing this time.”

What I Read: Tool Retrieval, RAG

By Andrew Fairless on November 20, 2024August 26, 2024

https://jxnl.co/writing/2024/08/21/trade-off-tool-selection Optimizing Tool Retrieval in RAG Systems: A Balanced ApproachJason Liu2024/08/21 “When it comes to Retrieval-Augmented Generation (RAG) systems, one of the key challenges is deciding how to select andContinue readingWhat I Read: Tool Retrieval, RAG

What I Read: LLM Pre-training Post-training

By Andrew Fairless on November 18, 2024August 26, 2024

https://magazine.sebastianraschka.com/p/new-llm-pre-training-and-post-training New LLM Pre-training and Post-training ParadigmsA Look at How Modern LLMs Are TrainedSebastian Raschka, PhDAug 17, 2024 “Initially, the LLM training process focused solely on pre-training, but it hasContinue readingWhat I Read: LLM Pre-training Post-training

Author: Andrew Fairless