https://ai.facebook.com/blog/computer-vision-combining-transformers-and-convolutional-neural-networks/ Better computer vision models by combining Transformers and convolutional neural networksJuly 8, 2021 “We’ve developed a new computer vision model… which combines… convolutional neural networks (CNNs) and Transformer-based models…
What I Read: Branch Specialization
https://distill.pub/2020/circuits/branch-specialization/ Branch SpecializationChelsea VossGabriel GohNick CammarataMichael PetrovLudwig SchubertChris OlahApril 5, 2021DOI 10.23915/distill.00024.008 “Branch specialization occurs when neural network layers are split up into branches. The neurons and circuits tend to
What I Read: Visualizing Weights
https://distill.pub/2020/circuits/visualizing-weights/ Visualizing WeightsChelsea VossNick CammarataGabriel GohMichael PetrovLudwig SchubertBen EganSwee Kiat LimChris OlahFeb. 4, 2021DOI 10.23915/distill.00024.007 “The problem of understanding a neural network is a little bit like reverse engineering a
What I Read: Ensemble, knowledge distillation, and self-distillation
https://www.microsoft.com/en-us/research/blog/three-mysteries-in-deep-learning-ensemble-knowledge-distillation-and-self-distillation/ Three mysteries in deep learning: Ensemble, knowledge distillation, and self-distillationPublished January 19, 2021By Zeyuan Allen-Zhu , Senior Researcher Yuanzhi Li , Assistant Professor, Carnegie Mellon University “…besides this small
What I Read: Transformer Networks to Answer Questions About Images
https://medium.com/dataseries/microsoft-uses-transformer-networks-to-answer-questions-about-images-with-minimum-training-f978c018bb72 Microsoft Uses Transformer Networks to Answer Questions About Images With Minimum TrainingUnified VLP can understand concepts about scenic images by using pretrained models.Jesus RodriguezJan 12 “Can we build deep