loss – Andrew Fairless, Ph.D.

What I Read: tilted loss

By Andrew Fairless on November 25, 2024August 26, 2024

https://alexshtf.github.io/2024/06/14/Untilting.html Alex ShtoffUntilting the tilted lossJun 14, 2024 “Typically in machine learning we train a model by minimizing the average loss…. The parameter t can be thought as a kindContinue readingWhat I Read: tilted loss

What I Read: KL All You Need

By Andrew Fairless on August 21, 2024June 4, 2024

https://blog.alexalemi.com/kl-is-all-you-need.html KL is All You NeedAlexander A. Alemi2024-01-08 “…the core of essentially all modern machine learning methods is a single universal objective: Kullback-Leibler (KL) divergence minimization…. Understand KL, understand theContinue readingWhat I Read: KL All You Need

What I Read: Matryoshka Embedding

By Andrew Fairless on July 17, 2024May 6, 2024

https://huggingface.co/blog/matryoshka Introduction to Matryoshka Embedding ModelsTom AarsenJoshuaOmar SansevieroFebruary 23, 2024 “…Kusupati et al. (2022) were inspired to create embedding models whose embeddings could reasonably be shrunk without suffering too muchContinue readingWhat I Read: Matryoshka Embedding

What I Read: Adversarial Attacks on LLMs

By Andrew Fairless on February 6, 2024December 19, 2023

https://lilianweng.github.io/posts/2023-10-25-adv-attack-llm/ Adversarial Attacks on LLMsLilian WengOctober 25, 2023 “Adversarial attacks are inputs that trigger the model to output something undesired.”

What I Read: LLMs

By Andrew Fairless on September 7, 2023August 1, 2023

https://willthompson.name/what-we-know-about-llms-primer What We Know About LLMs (Primer)Will Thompson (Twitter)July 23, 2023 “…it is worth reflecting on what we concretely know about LLMs at this point in time and how theseContinue readingWhat I Read: LLMs

What I Read: Perspectives on diffusion

By Andrew Fairless on August 30, 2023July 30, 2023

https://sander.ai/2023/07/20/perspectives.html Perspectives on diffusionSander DielemanJuly 20, 2023 “Diffusion models appear to come in many shapes and forms…. these various perspectives each reveal new connections and are a breeding ground forContinue readingWhat I Read: Perspectives on diffusion

What I Read: Applying BERT to Speech

By Andrew Fairless on June 27, 2022June 27, 2022

https://thegradient.pub/an-illustrated-tour-of-applying-bert-to-speech-data/ An Illustrated Tour of Applying BERT to Speech DataJonathan Boigne10.May.2022 “Could we replace the text input in BERT with a speech sequence, mask a part of it, and similarlyContinue readingWhat I Read: Applying BERT to Speech

What I Read: Learning, not Enough Data Part 3

By Andrew Fairless on June 1, 2022May 23, 2022

https://lilianweng.github.io/posts/2022-04-15-data-gen/ Learning with not Enough Data Part 3: Data GenerationApril 15, 2022Lilian Weng “Here comes the Part 3 on learning with not enough data (Previous: Part 1 and Part 2).Continue readingWhat I Read: Learning, not Enough Data Part 3

What I Read: binary cross-entropy, log loss

By Andrew Fairless on October 25, 2021October 16, 2021

https://towardsdatascience.com/understanding-binary-cross-entropy-log-loss-a-visual-explanation-a3ac6025181a?gi=375ce73be21b Understanding binary cross-entropy / log loss: a visual explanationDaniel GodoyNov 21, 2018 “If you are training a binary classifier, chances are you are using binary cross-entropy / log lossContinue readingWhat I Read: binary cross-entropy, log loss

Tag: loss