ImageNet Classification with Deep Convolutional Neural Networks
A deep convolutional network demonstrated how data, GPU computation and learned representations could transform visual recognition.
THE RESEARCH ATLAS
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
An IO-aware attention algorithm reduces memory transfers while computing exact attention.
Institution and location sources
Location refers to credited institutions or the listed organization, not author citizenship or an exhaustive collaboration list.
Why read this
Understand why hardware-aware algorithms matter alongside model architecture.
Citation
@misc{dao2022flashattention,
title = {FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness},
author = {Tri Dao and Daniel Y. Fu and Stefano Ermon and Atri Rudra and Christopher Ré},
year = {2022},
eprint = {2205.14135},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2205.14135}
}Rights remain with the original authors. Refer to the original source for citations and licensing.
Related research
Attention Is All You Need
The Transformer replaces recurrent sequence processing with attention, creating a foundation for many later language and multimodal models.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Masked-language pretraining learns bidirectional representations that can be adapted to language-understanding tasks.