ImageNet Classification with Deep Convolutional Neural Networks
A deep convolutional network demonstrated how data, GPU computation and learned representations could transform visual recognition.
THE RESEARCH ATLAS
Essential AI Papers: A Reading Path
From representation learning and Transformers to scaling, preferences and efficient training. Historical foundations are distinct from recent releases.
Attention Is All You Need
The Transformer replaces recurrent sequence processing with attention, creating a foundation for many later language and multimodal models.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Masked-language pretraining learns bidirectional representations that can be adapted to language-understanding tasks.
Language Models are Few-Shot Learners
GPT-3 studies how larger autoregressive language models perform new tasks through examples in their input context.
Denoising Diffusion Probabilistic Models
A generative model learns to reverse a gradual noise process, establishing an influential approach to image synthesis.
Learning Transferable Visual Models From Natural Language Supervision
Contrastive training aligns images with natural-language descriptions, enabling transfer to new visual categories.
LoRA: Low-Rank Adaptation of Large Language Models
Low-rank trainable updates adapt a frozen pretrained model without updating all of its parameters.
Training language models to follow instructions with human feedback
Human demonstrations and preference feedback are used to improve how a language model follows instructions.
Training Compute-Optimal Large Language Models
The study examines how to allocate a training-compute budget between model size and training data.
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
An IO-aware attention algorithm reduces memory transfers while computing exact attention.
Constitutional AI: Harmlessness from AI Feedback
A set of written principles guides critique, revision and AI-generated preference feedback during alignment.
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
A preference-based objective optimizes a language model directly, avoiding a separate reward-model training stage.
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
Robot control is formulated as conditional denoising of action sequences, linking diffusion models to visuomotor learning.