https://benanne.github.io/2020/09/01/typicality.html
Musings on typicality
September 01, 2020
Sander Dieleman
“…typicality is a concept worth understanding. It sheds light on why beam search doesn’t work for autoregressive models of images, audio and video; why you can’t just threshold the likelihood to perform anomaly detection with generative models; and why high-dimensional Gaussians are “soap bubbles”.”