https://distill.pub/2020/circuits/visualizing-weights/
Visualizing Weights
Chelsea Voss
Nick Cammarata
Gabriel Goh
Michael Petrov
Ludwig Schubert
Ben Egan
Swee Kiat Lim
Chris Olah
Feb. 4, 2021
DOI 10.23915/distill.00024.007
“The problem of understanding a neural network is a little bit like reverse engineering a large compiled binary of a computer program…. At the end of the day, the weights are the fundamental thing you want to understand: how does this sequence of convolutions and matrix multiplications give rise to model behavior?”