https://arxiv.org/abs/2009.01797
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning
Martin Mundt, Yong Won Hong, Iuliia Pliushch, Visvanathan Ramesh
“…comparison of individual methods is nevertheless treated in isolation from real world application… It is assumed that during deployment a model is guaranteed to encounter data that stems from the same distribution as used for training. This poses a massive challenge as neural networks are well known to provide overconfident false predictions on unknown instances and break down in the face of corrupted data. In this work we argue that notable lessons from open set recognition… and the adjacent field of active learning… are frequently overlooked in the deep learning era. Based on these forgotten lessons, we propose a consolidated view to bridge continual learning, active learning and open set recognition in deep neural networks.”