Start your day with one quick fact.

Factspresso gives you short, interesting facts every day so you can learn something new in less than a minute.

The AI Architecture Behind Chatbots Was Originally Built for Translation

In 2017, researchers introduced a neural-network design called the Transformer in a paper titled Attention Is All You Need. Their main experiments focused on translating English into German and French—not building chatbots. The breakthrough was its use of self-attention, which lets a model weigh relationships between different words in a sequence without processing them strictly one at a time. This made the architecture more parallelizable and faster to train than many previous approaches.

Mini Facts

The Transformer paper was first submitted in June 2017.

It was written by eight researchers.

The original model removed the recurrent and convolutional layers commonly used in earlier sequence models.

The paper showed strong results on both English-to-German and English-to-French translation benchmarks.

Sources & References

The Path to LLM-based Machine Translation The Original Generative Task Language Model History — Before and After Transformer: The AI Revolution Transformer: A Novel Neural Network Architecture for Language Understanding