self-supervised – Andrew Fairless, Ph.D.

What I Read: LLM Training, RLHF

By Andrew Fairless on October 30, 2023October 5, 2023

https://magazine.sebastianraschka.com/p/llm-training-rlhf-and-its-alternatives LLM Training: RLHF and Its AlternativesSebastian Raschka, PhDSep 10, 2023 “RLHF is an integral part of the modern LLM training pipeline due to its ability to incorporate human preferencesContinue readingWhat I Read: LLM Training, RLHF

What I Read: Topological Data Analysis

By Andrew Fairless on August 23, 2023July 9, 2023

https://datarefiner.com/feed/why-tda Why you should use Topological Data Analysis over t-SNE or UMAP?Edward KibardinApril 17, 2023 “Have you tried to run a complex supervised machine learning model, only to find unexpectedContinue readingWhat I Read: Topological Data Analysis

What I Learn: neural scaling, data pruning

By Andrew Fairless on April 11, 2023February 22, 2023

https://arxiv.org/abs/2206.14486 Beyond neural scaling laws: beating power law scaling via data pruningBen Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, Ari S. Morcos[Submitted on 29 Jun 2022 (v1), last revised 15Continue readingWhat I Learn: neural scaling, data pruning

What I Read: Self-Taught AI, Brain

By Andrew Fairless on October 4, 2022August 22, 2022

https://www.quantamagazine.org/self-taught-ai-shows-similarities-to-how-the-brain-works-20220811/ Self-Taught AI Shows Similarities to How the Brain WorksAnil AnanthaswamyContributing WriterAugust 11, 2022 “Self-supervised learning allows a neural network to figure out for itself what matters. The process mightContinue readingWhat I Read: Self-Taught AI, Brain

What I Read: Transformers in computer vision

By Andrew Fairless on September 6, 2022August 3, 2022

https://theaisummer.com/transformers-computer-vision/ Transformers in computer vision: ViT architectures, tips, tricks and improvementsNikolas Adaloglouon2021-10-21 “You are probably already aware of the Vision Transformer (ViT)…. We will address questions like how can youContinue readingWhat I Read: Transformers in computer vision

What I Read: Applying BERT to Speech

By Andrew Fairless on June 27, 2022June 27, 2022

https://thegradient.pub/an-illustrated-tour-of-applying-bert-to-speech-data/ An Illustrated Tour of Applying BERT to Speech DataJonathan Boigne10.May.2022 “Could we replace the text input in BERT with a speech sequence, mask a part of it, and similarlyContinue readingWhat I Read: Applying BERT to Speech

What I Read: Type-Aware Bi-Encoders for Open-Domain Entity Retrieval

By Andrew Fairless on May 31, 2022April 25, 2022

https://hazyresearch.stanford.edu/blog/2022-04-19-contrastive-3 TABi: Type-Aware Bi-Encoders for Open-Domain Entity RetrievalApr 19, 2022Megan Leszczynski, Dan Fu, Mayee Chen, and Chris Ré. “We introduce a method to improve retrieval of rare entities by incorporatingContinue readingWhat I Read: Type-Aware Bi-Encoders for Open-Domain Entity Retrieval

What I Read: Supervised Contrastive Learning

By Andrew Fairless on May 30, 2022April 25, 2022

https://hazyresearch.stanford.edu/blog/2022-04-19-contrastive-2 Improving Transfer and Robustness in Supervised Contrastive LearningApr 19, 2022 Mayee Chen, Dan Fu, Avanika Narayan, Michael Zhang, Zhao Song, Kayvon Fatahalian , and Chris Ré. “A major problemContinue readingWhat I Read: Supervised Contrastive Learning

What I Read: Semi-Supervised Learning

By Andrew Fairless on January 31, 2022December 12, 2021

https://lilianweng.github.io/lil-log/2021/12/05/semi-supervised-learning.html Learning with not Enough Data Part 1: Semi-Supervised LearningDec 5, 2021 by Lilian Weng “When facing a limited amount of labeled data for supervised learning tasks, four approaches areContinue readingWhat I Read: Semi-Supervised Learning

What I Read: Deep One-class Classification

By Andrew Fairless on October 4, 2021September 21, 2021

https://arxiv.org/abs/2011.02578 Learning and Evaluating Representations for Deep One-class ClassificationKihyuk Sohn, Chun-Liang Li, Jinsung Yoon, Minho Jin, Tomas Pfister “We present a two-stage framework for deep one-class classification. We first learnContinue readingWhat I Read: Deep One-class Classification

Tag: self-supervised