Deep Learning Optimization Theory — Introduction
Understanding the theory of optimization in deep learning is crucial to enable progress. This post introduces the experimental and theoretical approaches to studying it.
Omri Kaduri
Oct 26
“…optimization of complicated non-convex functions is hard to analyze. Since the optimization of deep neural networks… is non-convex, how can we attempt to answer those questions?”