training – Page 3 – Andrew Fairless, Ph.D.

What I Read: AIs producing own training data

By Andrew Fairless on July 10, 2023May 31, 2023

https://thegradient.pub/software2-a-new-generation-of-ais-that-become-increasingly-general-by-producing-their-own-training-data/ Software²: A new generation of AIs that become increasingly general by producing their own training dataMinqi Jiang22.Apr.2023 “We are at the cusp of transitioning from “learning from data” toContinue readingWhat I Read: AIs producing own training data

What I Read: Optimizing Machine Learning Training Pipelines

By Andrew Fairless on March 15, 2023February 4, 2023

https://medium.com/ntropy-network/bag-of-tricks-for-optimizing-machine-learning-training-pipelines-4f8d5cd3d432 Bag of tricks for optimizing machine learning training pipelinesArseny KravchenkoJan 5 “…we are constantly looking for ways to improve the efficiency of our machine learning pipelines, while keeping theContinue readingWhat I Read: Optimizing Machine Learning Training Pipelines

What I Read: Transformers Training

By Andrew Fairless on January 19, 2023December 16, 2022

https://www.borealisai.com/research-blogs/tutorial-17-transformers-iii-training/ Tutorial #17: Transformers III Training08/06/2021P. Xu, S. Prince “…we discuss challenges with transformer training dynamics and introduce some of the tricks that practitioners use to get transformers to converge.”

What I Read: BLOOM Training

By Andrew Fairless on September 12, 2022August 3, 2022

https://huggingface.co/blog/bloom-megatron-deepspeed The Technology Behind BLOOM TrainingStas BekmanPublished July 14, 2022. “…training ever larger language models has become the norm… the hidden knowledge about how to train such models rarely getsContinue readingWhat I Read: BLOOM Training

What I Read: Researchers Build AI That Builds AI

By Andrew Fairless on March 30, 2022February 3, 2022

https://www.quantamagazine.org/researchers-build-ai-that-builds-ai-20220125/ Researchers Build AI That Builds AIAnil AnanthaswamyJanuary 25, 2022“By using hypernetworks, researchers can now preemptively fine-tune artificial neural networks, saving some of the time and expense of training.”

What I Read: How Train Large Deep Learning Models

By Andrew Fairless on November 8, 2021October 24, 2021

https://www.assemblyai.com/blog/how-to-train-large-deep-learning-models-as-a-startup/ How to Train Large Deep Learning Models as a StartupDylan FoxOct 7, 2021Training large deep learning models is expensive and slow. Yet, startups are all about iterating fast. InContinue readingWhat I Read: How Train Large Deep Learning Models

What I Read: How to Train Really Large Models

By Andrew Fairless on October 26, 2021October 16, 2021

https://lilianweng.github.io/lil-log/2021/09/24/train-large-neural-networks.html How to Train Really Large Models on Many GPUs?Sep 24, 2021Lilian Weng “How to train large and deep neural networks is challenging, as it demands a large amount ofContinue readingWhat I Read: How to Train Really Large Models

What I Read: Continuous Training Strategy

By Andrew Fairless on May 12, 2021April 17, 2021

https://www.superwise.ai/resources/framework-for-a-successful-continuous-training-strategy Framework for a successful Continuous Training StrategyOr Itzary & Liran NahumMarch 18, 2021 “…ML models need to be retrained regularly…. Continuous training seeks to automatically and continuously retrain theContinue readingWhat I Read: Continuous Training Strategy

What I Read: Reproducing Deep Double Descent

By Andrew Fairless on February 7, 2021January 11, 2021

https://hippocampus-garden.com/double_descent/ Reproducing Deep Double DescentShion HondaJune 13, 2020 “…you’ve probably heard of bias-variance trade-off… But, do you know that there is a continuation of the U-shaped test risk curve? Surprisingly,Continue readingWhat I Read: Reproducing Deep Double Descent

What I Read: Deep Double Descent: Where Bigger Models and More Data Hurt

By Andrew Fairless on February 6, 2021January 11, 2021

https://arxiv.org/abs/1912.02292 Deep Double Descent: Where Bigger Models and More Data HurtPreetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, Ilya Sutskever “We show that a variety of modern deepContinue readingWhat I Read: Deep Double Descent: Where Bigger Models and More Data Hurt

Tag: training