The Transformer
- ai
Ashish Vaswani and colleagues introduced the Transformer in the 2017 paper Attention Is All You Need. The architecture used attention mechanisms to relate positions in a sequence, dispensing with recurrence and convolution in the proposed encoder-decoder design for translation. Its more parallelizable training and strong results made it influential beyond the original experiments. Later language-model families adapted Transformer ideas in different forms and at very different scales. The paper is an architectural milestone, not the first use of attention or the release date of every later generative model. [Original publication](https://arxiv.org/abs/1706.03762). **Timeline anchor:** 12 June 2017 CE. **References:** [Source 1](https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)), [Source 2](https://en.wikipedia.org/wiki/Attention_Is_All_You_Need).
Sources
Related
On other timelines
- Attention Is All You Need on The history of artificial intelligence by @athan
- Attention Is All You Need reports GPU-trained Transformer models on GPU and ASIC programming · parallel hardware, standards and applications by @athan