THE RESEARCH ATLAS

EMMA: End-to-End Multimodal Model for Autonomous Driving

Builds on Gemini to predict driving trajectories, perception objects and road structure from camera inputs and text.

Jyh-Jing Hwang; Runsheng Xu; Hubert Lin; Wei-Chih Hung; Jingwei Ji; Kristy Choi; Di Huang; Tong He; Paul Covington; Benjamin Sapp; Yin Zhou; James Guo; Dragomir Anguelov; Mingxing Tan
2024-10-30 · Waymo · Research paper
Source checked: 2026-10-25Institution base · Waymo 🇺🇸 United States
Institution and location sources

Location refers to credited institutions or the listed organization, not author citizenship or an exhaustive collaboration list.

arxiv.org ↗

waymo.com ↗

Why read this

Observe how general multimodal knowledge transfers to driving rather than treating language models and autonomy as unrelated tracks.

What it connects

Waymo explicitly identifies Gemini as the foundation; multiple driving tasks share a language representation.

Research benchmark evaluation; not deployment proof

Task evaluations do not establish deployment as the commercial Waymo Driver. The paper identifies video-context, 3D sensor-fusion and compute limitations.

Follow these connections ↗
Sources for relations and versions
Read original source ↗Original PDF ↗

Citation

@misc{hwang2024emma,
  title = {EMMA: End-to-End Multimodal Model for Autonomous Driving},
  author = {Jyh-Jing Hwang and Runsheng Xu and Hubert Lin and Wei-Chih Hung and Jingwei Ji and Kristy Choi and Di Huang and Tong He and Paul Covington and Benjamin Sapp and Yin Zhou and James Guo and Dragomir Anguelov and Mingxing Tan},
  year = {2024},
  eprint = {2410.23262},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  url = {https://arxiv.org/abs/2410.23262}
}

Rights remain with the original authors. Refer to the original source for citations and licensing.

Related research

2012 · Academic · Research paperCredited institutions · University of Toronto 🇨🇦 Canada

ImageNet Classification with Deep Convolutional Neural Networks

A deep convolutional network demonstrated how data, GPU computation and learned representations could transform visual recognition.

2020-06 · Academic · Research paperCredited institutions · UC Berkeley 🇺🇸 United States

Denoising Diffusion Probabilistic Models

A generative model learns to reverse a gradual noise process, establishing an influential approach to image synthesis.

2021-02 · OpenAI · Research paperCredited institutions · OpenAI 🇺🇸 United States

Learning Transferable Visual Models From Natural Language Supervision

Contrastive training aligns images with natural-language descriptions, enabling transfer to new visual categories.