circagraph
Sign inSign up

The history of computer science

From counting boards and the first algorithms to deep learning: the theory, machines, languages, algorithms, systems, networks, software practice, security and AI that make up computer science, in calendar dates. Moments are tagged and colored by subfield; eras are shown as spans, and links connect ideas to what they led to. Summaries are original; each moment links to its Wikipedia articles as sources. Dates marked approximate say so.

Duration correction: Ongoing subjects previously drawn through a present/research cutoff are represented at their original starts, with no end asserted. Existing moments at those coordinates are reused; their attached cards retain the former duration’s notes, sources, tags and media. Bounded historical and coverage intervals remain durations.

88 moments · 2700-07 BCE – 2022-11-30 CE · by @athan · @athan

The Transformer

  • ai

Ashish Vaswani and colleagues introduced the Transformer in the 2017 paper Attention Is All You Need. The architecture used attention mechanisms to relate positions in a sequence, dispensing with recurrence and convolution in the proposed encoder-decoder design for translation. Its more parallelizable training and strong results made it influential beyond the original experiments. Later language-model families adapted Transformer ideas in different forms and at very different scales. The paper is an architectural milestone, not the first use of attention or the release date of every later generative model. [Original publication](https://arxiv.org/abs/1706.03762). **Timeline anchor:** 12 June 2017 CE. **References:** [Source 1](https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)), [Source 2](https://en.wikipedia.org/wiki/Attention_Is_All_You_Need).

Sources

Related

On other timelines

Open The Transformer in the timeline