https://ruder.io/recent-advances-lm-fine-tuning/
Recent Advances in Language Model Fine-tuning
This article provides an overview of recent methods to fine-tune large pre-trained language models.
Sebastian Ruder
24 Feb 2021
“While pre-training is compute-intensive, fine-tuning can be done comparatively inexpensively. Fine-tuning is more important for the practical usage of such models as individual pre-trained models are downloaded—and fine-tuned—millions of times… I will highlight the most recent advances that have shaped or are likely to change the way we fine-tune language models…”