What I Read: Attention with Performers

https://ai.googleblog.com/2020/10/rethinking-attention-with-performers.html

Rethinking Attention with Performers
Friday, October 23, 2020
Posted by Krzysztof Choromanski and Lucy Colwell, Research Scientists, Google Research


“To resolve these issues, we introduce the Performer, a Transformer architecture with attention mechanisms that scale linearly, thus enabling faster training while allowing the model to process longer lengths…”