https://www.asapp.com/blog/reducing-the-high-cost-of-training-nlp-models-with-sru/ Reducing the High Cost of Training NLP Models With SRU++By Tao Lei, PhDResearch Leader and Scientist at ASAPP “The Transformer architecture was proposed to accelerate model training in NLP….
What I Read: Neural Nets, How Brains Learn
https://www.quantamagazine.org/artificial-neural-nets-finally-yield-clues-to-how-brains-learn-20210218/ Artificial Neural Nets Finally Yield Clues to How Brains LearnAnil AnanthaswamyContributing WriterFebruary 18, 2021 “The learning algorithm that enables the runaway success of deep neural networks doesn’t work in
What I Read: Continual Learning, Amnesia, Neural Networks
https://medium.com/dataseries/ibm-uses-continual-learning-to-avoid-the-amnesia-problem-in-neural-networks-ae8241e1f3a3 IBM Uses Continual Learning to Avoid The Amnesia Problem in Neural NetworksUsing continual learning might avoid the famous catastrophic forgetting problem in neural networks.Jesus RodriguezJan 25 “Building neural networks
What I Read: Ensemble, knowledge distillation, and self-distillation
https://www.microsoft.com/en-us/research/blog/three-mysteries-in-deep-learning-ensemble-knowledge-distillation-and-self-distillation/ Three mysteries in deep learning: Ensemble, knowledge distillation, and self-distillationPublished January 19, 2021By Zeyuan Allen-Zhu , Senior Researcher Yuanzhi Li , Assistant Professor, Carnegie Mellon University “…besides this small
What I Read: Transformer Networks to Answer Questions About Images
https://medium.com/dataseries/microsoft-uses-transformer-networks-to-answer-questions-about-images-with-minimum-training-f978c018bb72 Microsoft Uses Transformer Networks to Answer Questions About Images With Minimum TrainingUnified VLP can understand concepts about scenic images by using pretrained models.Jesus RodriguezJan 12 “Can we build deep