https://ai.googleblog.com/2020/10/rethinking-attention-with-performers.html
Rethinking Attention with Performers
Friday, October 23, 2020
Posted by Krzysztof Choromanski and Lucy Colwell, Research Scientists, Google Research
“To resolve these issues, we introduce the Performer, a Transformer architecture with attention mechanisms that scale linearly, thus enabling faster training while allowing the model to process longer lengths…”