transformer – Page 5 – Andrew Fairless, Ph.D.

What I Read: Transformers, Brain

By Andrew Fairless on November 9, 2022September 27, 2022

https://www.quantamagazine.org/how-ai-transformers-mimic-parts-of-the-brain-20220912/ How Transformers Seem to Mimic Parts of the BrainStephen OrnesSeptember 12, 2022 “Neural networks originally designed for language processing turn out to be great models of how our brainsContinue readingWhat I Read: Transformers, Brain

What I Read: Emergent Features

By Andrew Fairless on October 6, 2022August 22, 2022

LLM.int8() and Emergent Features LLM.int8() and Emergent Features2022-08-17 by Tim Dettmers “…I use advanced quantization methods to achieve no performance degradation transformer inference at scale that makes large models moreContinue readingWhat I Read: Emergent Features

What I Read: BLOOM Training

By Andrew Fairless on September 12, 2022August 3, 2022

https://huggingface.co/blog/bloom-megatron-deepspeed The Technology Behind BLOOM TrainingStas BekmanPublished July 14, 2022. “…training ever larger language models has become the norm… the hidden knowledge about how to train such models rarely getsContinue readingWhat I Read: BLOOM Training

What I Read: Transformers in computer vision

By Andrew Fairless on September 6, 2022August 3, 2022

https://theaisummer.com/transformers-computer-vision/ Transformers in computer vision: ViT architectures, tips, tricks and improvementsNikolas Adaloglouon2021-10-21 “You are probably already aware of the Vision Transformer (ViT)…. We will address questions like how can youContinue readingWhat I Read: Transformers in computer vision

What I Read: Applying BERT to Speech

By Andrew Fairless on June 27, 2022June 27, 2022

https://thegradient.pub/an-illustrated-tour-of-applying-bert-to-speech-data/ An Illustrated Tour of Applying BERT to Speech DataJonathan Boigne10.May.2022 “Could we replace the text input in BERT with a speech sequence, mask a part of it, and similarlyContinue readingWhat I Read: Applying BERT to Speech

What I Read: Understanding, Simple AI

By Andrew Fairless on May 24, 2022April 25, 2022

https://www.quantamagazine.org/researchers-glimpse-how-ai-gets-so-good-at-language-processing-20220414/ Researchers Gain New Understanding From Simple AIMordechai RorvigStaff WriterApril 14, 2022 “Language processing programs are notoriously hard to interpret, but smaller versions can provide important insights into how theyContinue readingWhat I Read: Understanding, Simple AI

What I Read: Will Transformers Take Over Artificial Intelligence?

By Andrew Fairless on April 26, 2022March 12, 2022

https://www.quantamagazine.org/will-transformers-take-over-artificial-intelligence-20220310/ Will Transformers Take Over Artificial Intelligence?A simple algorithm that revolutionized how neural networks approach language is now taking on vision as well. It may not stop there.Stephen OrnesContributing WriterMarchContinue readingWhat I Read: Will Transformers Take Over Artificial Intelligence?

What I Read: Textless NLP

By Andrew Fairless on April 13, 2022March 12, 2022

https://ai.facebook.com/blog/textless-nlp-generating-expressive-speech-from-raw-audio Textless NLP: Generating expressive speech from raw audioSeptember 9, 2021 “GSLM leverages recent breakthroughs in representation learning, allowing it to work directly from only raw audio signals… for potentiallyContinue readingWhat I Read: Textless NLP

What I Read: To Understand Language is to Understand Generalization

By Andrew Fairless on February 16, 2022January 20, 2022

https://evjang.com/2021/12/17/lang-generalization.html To Understand Language is to Understand GeneralizationEric JangDec 17, 2021 “We all want ML models to generalize better, but defining “generalization” is hard. I suggest that the structure ofContinue readingWhat I Read: To Understand Language is to Understand Generalization

What I Read: Interpretable Time Series

By Andrew Fairless on February 14, 2022January 20, 2022

https://ai.googleblog.com/2021/12/interpretable-deep-learning-for-time.html Interpretable Deep Learning for Time Series ForecastingMonday, December 13, 2021Posted by Sercan O. Arik, Research Scientist and Tomas Pfister, Engineering Manager, Google Cloud “Multi-horizon forecasting, i.e. predicting variables-of-interest atContinue readingWhat I Read: Interpretable Time Series

Tag: transformer