ImageNet Classification with Deep Convolutional Neural Networks
A deep convolutional network demonstrated how data, GPU computation and learned representations could transform visual recognition.
THE RESEARCH ATLAS
Attention Is All You Need
The Transformer replaces recurrent sequence processing with attention, creating a foundation for many later language and multimodal models.
Institution and location sources
Location refers to credited institutions or the listed organization, not author citizenship or an exhaustive collaboration list.
Why read this
Read the attention mechanism, positional encodings and encoder–decoder architecture before exploring later model families.
Citation
@misc{vaswani2017attention,
title = {Attention Is All You Need},
author = {Ashish Vaswani and Noam Shazeer and Niki Parmar and Jakob Uszkoreit and Llion Jones and Aidan N. Gomez and Lukasz Kaiser and Illia Polosukhin},
year = {2017},
eprint = {1706.03762},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
url = {https://arxiv.org/abs/1706.03762}
}Rights remain with the original authors. Refer to the original source for citations and licensing.
Related research
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Masked-language pretraining learns bidirectional representations that can be adapted to language-understanding tasks.
Language Models are Few-Shot Learners
GPT-3 studies how larger autoregressive language models perform new tasks through examples in their input context.