entropy – Andrew Fairless, Ph.D.

What I Read: KL All You Need

By Andrew Fairless on August 21, 2024June 4, 2024

https://blog.alexalemi.com/kl-is-all-you-need.html KL is All You NeedAlexander A. Alemi2024-01-08 “…the core of essentially all modern machine learning methods is a single universal objective: Kullback-Leibler (KL) divergence minimization…. Understand KL, understand theContinue readingWhat I Read: KL All You Need

What I Read: Random Forests, Estimation of Information-Theoretic Quantities

By Andrew Fairless on February 27, 2024January 25, 2024

https://arxiv.org/abs/1907.00325v7 Random Forests for Adaptive Nearest Neighbor Estimation of Information-Theoretic QuantitiesRonan Perry, Ronak Mehta, Richard Guo, Eva Yezerets, Jesús Arroyo, Mike Powell, Hayden Helm, Cencheng Shen, Joshua T. Vogelstein “WeContinue readingWhat I Read: Random Forests, Estimation of Information-Theoretic Quantities

What I Read: scaling laws, cross-entropy loss

By Andrew Fairless on September 28, 2023September 4, 2023

https://epochai.org/blog/the-direct-approach The Direct ApproachMatthew Barnett and Tamay Besiroglu “Empirical scaling laws can help predict the cross-entropy loss associated with training inputs, such as compute and data. However, in order toContinue readingWhat I Read: scaling laws, cross-entropy loss

What I Read: Perspectives on diffusion

By Andrew Fairless on August 30, 2023July 30, 2023

https://sander.ai/2023/07/20/perspectives.html Perspectives on diffusionSander DielemanJuly 20, 2023 “Diffusion models appear to come in many shapes and forms…. these various perspectives each reveal new connections and are a breeding ground forContinue readingWhat I Read: Perspectives on diffusion

What I Read: Applying BERT to Speech

By Andrew Fairless on June 27, 2022June 27, 2022

https://thegradient.pub/an-illustrated-tour-of-applying-bert-to-speech-data/ An Illustrated Tour of Applying BERT to Speech DataJonathan Boigne10.May.2022 “Could we replace the text input in BERT with a speech sequence, mask a part of it, and similarlyContinue readingWhat I Read: Applying BERT to Speech

What I Read: binary cross-entropy, log loss

By Andrew Fairless on October 25, 2021October 16, 2021

https://towardsdatascience.com/understanding-binary-cross-entropy-log-loss-a-visual-explanation-a3ac6025181a?gi=375ce73be21b Understanding binary cross-entropy / log loss: a visual explanationDaniel GodoyNov 21, 2018 “If you are training a binary classifier, chances are you are using binary cross-entropy / log lossContinue readingWhat I Read: binary cross-entropy, log loss

Tag: entropy