https://amitness.com/2020/05/self-supervised-learning-nlp/
Self Supervised Representation Learning in NLP
Published May 23, 2020 in nlp
“While Computer Vision is making amazing progress on self-supervised learning only in the last few years, self-supervised learning has been a first-class citizen in NLP research for quite a while…. I will provide an overview of the various pretext tasks that researchers have designed to learn representations from text corpus without explicit data labeling.”