loss function – Andrew Fairless, Ph.D.

What I Read: Auditing AI, How Much Access

By Andrew Fairless on November 14, 2023October 5, 2023

https://aipolicy.substack.com/p/ai-accountability-transparency-2 Auditing AI: How Much Access Is Needed to Audit an AI System?Chapter 2 in our AI Accountability & Transparency series(By Sarah H. Cen, Cosimo L. Fabrizio, James Siderius, AleksanderContinue readingWhat I Read: Auditing AI, How Much Access

What I Read: LLMs, single example

By Andrew Fairless on October 23, 2023September 9, 2023

https://www.fast.ai/posts/2023-09-04-learning-jumps/ Can LLMs learn from a single example?Jeremy Howard and Jonathan WhitakerSeptember 4, 2023 “We’ve noticed an unusual training pattern in fine-tuning LLMs. At first we thought it’s a bug,Continue readingWhat I Read: LLMs, single example

What I Read: Differentiable Trees

By Andrew Fairless on October 18, 2023September 4, 2023

https://ericmjl.github.io/blog/2023/8/7/journal-club-differentiable-search-of-evolutionary-trees/ Journal Club: Differentiable Search of Evolutionary TreesEric J. Ma2023-08-07 “…how the authors take a non-differentiable problem and turn it into a differentiable problem through interconversion between mathematical data structures.”

What I Read: Bonsai Networks, RNNs

By Andrew Fairless on October 3, 2023September 4, 2023

https://cprimozic.net/blog/growing-sparse-computational-graphs-with-rnns/ Growing Bonsai Networks with RNNsCasey Primozic “This writeup introduces what I’m calling Bonsai Networks – extremely sparse computational graphs produced by training and pruning RNNs. They provide an interpretableContinue readingWhat I Read: Bonsai Networks, RNNs

What I Read: scaling laws, cross-entropy loss

By Andrew Fairless on September 28, 2023September 4, 2023

https://epochai.org/blog/the-direct-approach The Direct ApproachMatthew Barnett and Tamay Besiroglu “Empirical scaling laws can help predict the cross-entropy loss associated with training inputs, such as compute and data. However, in order toContinue readingWhat I Read: scaling laws, cross-entropy loss

What I Read: Contrastive Representation Learning

By Andrew Fairless on July 1, 2021June 7, 2021

https://lilianweng.github.io/lil-log/2021/05/31/contrastive-representation-learning.html Contrastive Representation Learningby Lilian WengMay 31, 2021 “The goal of contrastive representation learning is to learn such an embedding space in which similar sample pairs stay close to eachContinue readingWhat I Read: Contrastive Representation Learning

What I Read: Traffic prediction with Graph Neural Networks

By Andrew Fairless on January 23, 2021December 30, 2020

https://deepmind.com/blog/article/traffic-prediction-with-advanced-graph-neural-networks Traffic prediction with advanced Graph Neural Networks “We discovered that Graph Neural Networks are particularly sensitive to changes in the training curriculum… the most successful approach to this problemContinue readingWhat I Read: Traffic prediction with Graph Neural Networks

What I Read: Autotuning Multi-Objective Optimization

By Andrew Fairless on January 19, 2021December 27, 2020

https://www.kdnuggets.com/2020/08/autotuning-multi-objective-optimization-linkedin-feed-ranking.html Autotuning for Multi-Objective Optimization on LinkedIn’s Feed RankingBy Marco Varela & Mavis Li, LinkedIn “First, we learned that in this particular optimization problem it pays to formulate your problemContinue readingWhat I Read: Autotuning Multi-Objective Optimization

Tag: loss function