Workshop on Data-Driven Autonomous Driving Simulation

This workshop will be held at the CVPR 2024, on June 18th, 2024, 8:30am to 5:30pm in Summit 342 at Seattle Convention Centre, WA, USA and streamed online via zoom (CVPR registration required).

Abstract

Real-world on-road testing of autonomous vehicles can be expensive or dangerous, making simulation a crucial tool to accelerate the development of safe autonomous driving (AD), a technology with enormous real-world impact. However, to minimise the sim-to-real gap, good agent behaviour models and sensor/perception imitation are paramount. A recent surge in published papers in this fast-growing field has led to a lot of progress, but several fundamental questions remain unanswered, for example regarding the fidelity and diversity of generative behaviour and perception models, generation of realistic controllable scenes at scale and the safety assessment of the simulation toolchain. In this workshop, our goal is to bring together practitioners and researchers from all areas of AD simulation and to discuss pressing challenges, recent breakthroughs and future directions.

Agenda

This full day workshop will take place on Tuesday June 18th, 08:45 to 17:30 PST (UTC-8).

Below times are in Seattle time. Current time in Seattle is .

Time

Event

Content

8:30

Welcome

8:45

Felix Heide

Torc Robotics & Princeton University

Generating The Invisible: Capturing and Generating Edge-cases in Autonomous Driving

9:15

Siva Manivasagam

Head of Sensor Simulation, Waabi

Generative AI for Developing and Deploying Self-driving Systems Safely

9:45

Poster Presentations 1

10:15

Coffee break and poster session

Posters are in Arch Building Exhibit Hall

11:00

Dragomir Anguelov

Vice President and Head of Research, Waymo

ML for Realistic and Efficient Driving Simulation

11:30

Aleksandr Petiushko

Head of ML Research, Nuro

Combining Imitation and Reinforcement Learning in Behavior

12:00

Lunch

14:00

Gustav Markkula

Professor, University of Leeds

Valid human agents in simulated AD testing: Behavioural phenomena and cognitive mechanisms

14:30

Poster Presentations 2

15:15

Coffee break and poster session

Posters are in Arch Building Exhibit Hall

16:00

Jamie Shotton

Chief Scientist, Wayve

Frontiers in Embodied AI for Autonomous Driving

16:30

Kashyap Chitta

University of Tübingen

Synthesizing Simulation Environments with Generative Models

17:00

Panel Discussion

With Dragomir Anguelov, Jamie Shotton, Gustav Markkula, Aleksandr Petiushko and Felix Heide. Moderated by Sanja Fidler.

17:30

End

Invited Speakers

Felix Heide

Torc Robotics & Princeton University

Siva Manivasagam

Head of Sensor Simulation, Waabi

Dragomir Anguelov

Vice President and Head of Research, Waymo

Aleksandr Petiushko

Head of ML Research, Nuro

Gustav Markkula

Professor, University of Leeds

Jamie Shotton

Chief Scientist, Wayve

Kashyap Chitta

University of Tübingen

Papers

Title

Authors

Links

DistillNeRF: Distilling Neural Radiance Fields into Sparse Voxels for Generalizable Scene Representations

Letian Wang, Seung Wook Kim, Jiawei Yang, Cunjun Yu, Boris Ivanovic, Steven Waslander, Yue Wang, Sanja Fidler, Marco Pavone, Peter Karkus

Poster
Video
Paper

A Two-Level Stochastic Model for the Lateral Movement of Vehicles Within Their Lane Under Homogeneous Traffic Conditions

Nicole Neis

Poster
Paper

Dense reinforcement learning for safety validation of autonomous vehicles

Shuo Feng, Haowei Sun, Xintao Yan, Haojie Zhu, Zhengxia Zou, Shengyin Shen, Henry X. Liu

Poster
Paper

Dynamic LiDAR Re-simulation using Compositional Neural Fields

Hanfeng Wu, Xingxing Zuo, Stefan Leutenegger, Or Litany, Konrad Schindler, Shengyu Huang

Poster
Video
Paper

Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents

Yuxi Wei, Zi Wang, Yifan Lu, Chenxin Xu, Changxing Liu, Hao Zhao, Siheng Chen, Yanfeng Wang

Poster
Paper

AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving

Mingfu Liang, Jong-Chyi Su, Samuel Schulter, Sparsh Garg, Shiyu Zhao, Ying Wu, Manmohan Chandraker

Paper

Multi-Level Neural Scene Graphs for Dynamic Urban Environments

Tobias Fischer, Lorenzo Porzi, Samuel Rota Bulò, Marc Pollefeys, Peter Kontschieder

Poster
Paper

KnowMoformer: Knowledge-Conditioned Motion Transformer for Controllable Traffic Scenario Simulation

Honglin He, Shu Li, Jingxuan Yang, Linxuan He, Yi ZHANG, Qiujing Lu, Shuo Feng

Poster
Paper

LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenes

shanlin sun, Bingbing Zhuang, Ziyu Jiang, Buyu Liu, Xiaohui Xie, Manmohan Chandraker

Poster
Paper

Multiverse Transformer: Advancing Closed-Loop Multi-Agent Simulation with Generative Model

Yu Wang, Tiebiao Zhao, Fan Yi, Guangzhi Cao

Poster
Paper

Neural Rendering for Safety-critical Autonomous Driving Simulation

William Ljungbergh, Adam Tonderski, Joakim Johnander, Holger Caesar, Kalle Åström, Michael Felsberg, Christoffer Petersson

Poster
Video
Paper

Sim-to-Real adversarial domain adaptation for 3D object detection

Maciej Wozniak, Mattias Hansson, Marko Thiel, Patric Jensfelt

Paper

NeuRAD: Neural Rendering for Autonomous Driving

Adam Tonderski, Carl Lindström, Georg Hess, William Ljungbergh, Lennart Svensson, Christoffer Petersson

Poster
Paper

Divide and Conquer: A Systematic Approach for Industrial Scale High-Definition OpenDRIVE Generation from Sparse Point Clouds

Leon Eisemann

Poster
Paper

SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Controllable Adversaries

Wei-Jer Chang, Francesco Pittaluga, Masayoshi Tomizuka, Wei Zhan, Manmohan Chandraker

Paper

HD Maps are Lane Detection Generalizers : A Novel Generative Framework for Single-Source Domain Generalization

Daeun Lee, Minhyeok Heo, Jiwon Kim

Poster
Paper

Scaling Is All You Need: Autonomous Driving with JAX-Accelerated Reinforcement Learning

Moritz Harmel, Anubhav Paras, Andreas Pasternak, Nicholas Roy, Gary Linscott

Paper

Taming Transformers for Realistic Lidar Point Cloud Generation

Hamed Haghighi, Amir Samadi, Mehrdad Dianati, Valentina Donzella, Kurt Debattista

Poster
Paper

RACL: Risk Aware Closed-Loop Agent Simulation with High Fidelity

Qiujing Lu, Ruoxuan Bai, Shu Li, Honglin He, Shuo Feng

Poster
Paper

Text-to-Drive: Diverse Driving Behavior Synthesis via Large Language Models

Phat Tan Nguyen, Tsun-Hsuan Wang, Zhang-Wei Hong, Sertac Karaman, Daniela Rus

Paper

Fairness in Autonomous Driving: Towards Understanding Confounding Factors in Object Detection under Challenging Weather

Bimsara Pathiraja, Caleb Liu, Ransalu Senanayake

Poster
Paper

DriveLM: Driving with Graph Visual Question Answering

Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Ping Luo, Andreas Geiger, Hongyang Li

Paper

DRIVEVLM: The Convergence of Autonomous Driving and Large Vision-Language Models

Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Chenxu Hu, Yang Wang, Kun Zhan, Peng Jia, Xianpeng Lang, Hang Zhao

Paper

DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes

Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, Ming-Hsuan Yang

Poster
Paper

HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splatting

Hongyu Zhou, Jiahao Shao, Lu Xu, Dongfeng Bai, Weichao Qiu, Bingbing Liu, Yue Wang, Andreas Geiger, Yiyi Liao

Poster
Paper

Organizers

Maximilian Igl

NVIDIA

Zan Gojcic

NVIDIA

Maximilian Naumann

Bosch Center for Artifical Intelligence and KIT

Jonah Philion

NVIDIA

Thomas Gilles

Waabi

Yue Wang

NVIDIA

Peter Karkus

NVIDIA

Or Litany

NVIDIA

Azadeh Dinparastdjadid

Waymo

Xinshuo Weng

NVIDIA

Yiren Lu

Waymo Research

Katie Luo

NVIDIA

Anqi Joyce Yang

Waabi

Jiawei Yang

NVIDIA

Sanja Fidler

NVIDIA and University of Toronto

Shimon Whiteson

Waymo UK and University of Oxford

Marco Pavone

NVIDIA and Stanford University

Contact: agents4ad@googlegroups.com