neural network – Page 24 – Andrew Fairless, Ph.D.

What I Read: Mismatches between Optimization Analyses and Deep Learning

By Andrew Fairless on February 13, 2021January 14, 2021

http://www.offconvex.org/2020/10/21/intrinsicLR/ Mismatches between Traditional Optimization Analyses and Modern Deep LearningZhiyuan Li and Sanjeev AroraOct 21, 2020 “You may remember our previous blog post showing that it is possible to doContinue readingWhat I Read: Mismatches between Optimization Analyses and Deep Learning

What I Read: Attention with Performers

By Andrew Fairless on February 7, 2021January 11, 2021

https://ai.googleblog.com/2020/10/rethinking-attention-with-performers.html Rethinking Attention with PerformersFriday, October 23, 2020Posted by Krzysztof Choromanski and Lucy Colwell, Research Scientists, Google Research “To resolve these issues, we introduce the Performer, a Transformer architecture withContinue readingWhat I Read: Attention with Performers

What I Read: Reproducing Deep Double Descent

By Andrew Fairless on February 7, 2021January 11, 2021

https://hippocampus-garden.com/double_descent/ Reproducing Deep Double DescentShion HondaJune 13, 2020 “…you’ve probably heard of bias-variance trade-off… But, do you know that there is a continuation of the U-shaped test risk curve? Surprisingly,Continue readingWhat I Read: Reproducing Deep Double Descent

What I Read: Deep Double Descent: Where Bigger Models and More Data Hurt

By Andrew Fairless on February 6, 2021January 11, 2021

https://arxiv.org/abs/1912.02292 Deep Double Descent: Where Bigger Models and More Data HurtPreetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, Ilya Sutskever “We show that a variety of modern deepContinue readingWhat I Read: Deep Double Descent: Where Bigger Models and More Data Hurt

What I Read: AI for good, think form extraction

By Andrew Fairless on February 5, 2021January 10, 2021

http://jonathanstray.com/to-apply-ai-for-good-think-form-extraction To apply AI for good, think form extractionPosted on October 24, 2020 Author Jonathan Stray “Folks who want to use AI/ML for good generally think of things like buildingContinue readingWhat I Read: AI for good, think form extraction

What I Read: “Less than one”-shot learning

By Andrew Fairless on February 4, 2021January 10, 2021

https://www.technologyreview.com/2020/10/16/1010566/ai-machine-learning-with-tiny-data/ Artificial intelligence/Machine learningA radical new technique lets AI learn with practically no data“Less than one”-shot learning can teach a model to identify more objects than the number of examplesContinue readingWhat I Read: “Less than one”-shot learning

What I Read: Neural Networks Help Explain Brains

By Andrew Fairless on February 1, 2021January 7, 2021

https://www.quantamagazine.org/deep-neural-networks-help-to-explain-living-brains-20201028/ neuroscienceDeep Neural Networks Help to Explain Living Brains Deep neural networks, often criticized as “black boxes,” are helping neuroscientists understand the organization of living brains.“If their CNN mimicked aContinue readingWhat I Read: Neural Networks Help Explain Brains

What I Read: Can a neural network train other networks?

By Andrew Fairless on January 31, 2021January 6, 2021

https://towardsdatascience.com/can-a-neural-network-train-other-networks-cf371be516c6?gi=4765ee11be44 Can a neural network train other networks?An introduction to knowledge distillationTivadar DankaOct 5 “Now you have a huge model, which, although performs excellently, there is no way to deployContinue readingWhat I Read: Can a neural network train other networks?

What I Read: automatic differentiation with graphs

By Andrew Fairless on January 30, 2021January 5, 2021

https://ai.facebook.com/blog/a-new-open-source-framework-for-automatic-differentiation-with-graphs A new open source framework for automatic differentiation with graphsOctober 8th, 2020 “Just as PyTorch provides a framework for automatic differentiation with tensors, GTN provides such a framework forContinue readingWhat I Read: automatic differentiation with graphs

What I Read: Neural Architecture Search

By Andrew Fairless on January 26, 2021December 31, 2020

https://lilianweng.github.io/lil-log/2020/08/06/neural-architecture-search.html Neural Architecture SearchAug 6, 2020 by Lilian Weng “Automatically learning and evolving network topologies is not a new idea (Stanley & Miikkulainen, 2002). In recent years, the pioneering workContinue readingWhat I Read: Neural Architecture Search

Tag: neural network