Foundations and Trends® in Computer Graphics and Vision > Vol 12 > Issue 1–3

Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art

By Joel Janai, Max-Planck-Institute for Intelligent Systems Tübingen, Germany University of Tübingen, Germany, joel.janai@tuebingen.mpg.de | Fatma Güney, College of Engineering, Koç University, Turkey, fguney@ku.edu.tr | Aseem Behl, Max-Planck-Institute for Intelligent Systems Tübingen, Germany University of Tübingen, Germany, aseem.behl@tuebingen.mpg.de | Andreas Geiger, Max-Planck-Institute for Intelligent Systems Tübingen, Germany University of Tübingen, Germany, andreas.geiger@tuebingen.mpg.de

 
Suggested Citation
Joel Janai, Fatma Güney, Aseem Behl and Andreas Geiger (2020), "Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art", Foundations and Trends® in Computer Graphics and Vision: Vol. 12: No. 1–3, pp 1-308. http://dx.doi.org/10.1561/0600000079

Publication Date: 06 Jul 2020
© 2020 Joel Janai, Fatma Güney, Aseem Behl and Andreas Geiger
 
Subjects
Deep learning
 

Free Preview:

Download extract

Share

Download article
In this article:
1. Introduction 
2. History of Autonomous Driving 
3. Sensors 
4. Datasets & Benchmarks 
5. Object Detection 
6. Object Tracking 
7. Semantic Segmentation 
8. Semantic Instance Segmentation 
9. Stereo 
10. Multi-View 3D Reconstruction 
11. Optical Flow 
12. 3D Scene Flow 
13. Mapping, Localization & Ego-Motion Estimation 
14. Scene Understanding 
15. End-to-End Learning for Autonomous Driving 
16. Conclusion 
Acknowledgements 
References 

Abstract

Recent years have witnessed enormous progress in AI-related fields such as computer vision, machine learning, and autonomous vehicles. As with any rapidly growing field, it becomes increasingly difficult to stay up-to-date or enter the field as a beginner. While several survey papers on particular sub-problems have appeared, no comprehensive survey on problems, datasets, and methods in computer vision for autonomous vehicles has been published. This monograph attempts to narrow this gap by providing a survey on the state-of-the-art datasets and techniques. Our survey includes both the historically most relevant literature as well as the current state of the art on several specific topics, including recognition, reconstruction, motion estimation, tracking, scene understanding, and end-to-end learning for autonomous driving. Towards this goal, we analyze the performance of the state of the art on several challenging benchmarking datasets, including KITTI, MOT, and Cityscapes. Besides, we discuss open problems and current research challenges. To ease accessibility and accommodate missing references, we also provide a website that allows navigating topics as well as methods and provides additional information.

DOI:10.1561/0600000079
ISBN: 978-1-68083-688-2
326 pp. $99.00
Buy book (pb)
 
ISBN: 978-1-68083-689-9
326 pp. $280.00
Buy E-book (.pdf)
Table of contents:
1. Introduction
2. History of Autonomous Driving
3. Sensors
4. Datasets & Benchmarks
5. Object Detection
6. Object Tracking
7. Semantic Segmentation
8. Semantic Instance Segmentation
9. Stereo
10. Multi-View 3D Reconstruction
11. Optical Flow
12. 3D Scene Flow
13. Mapping, Localization & Ego-Motion Estimation
14. Scene Understanding
15. End-to-End Learning for Autonomous Driving
16. Conclusion
Acknowledgements
References

Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art

Recent years have witnessed enormous progress in AI-related fields such as computer vision, machine learning, and autonomous vehicles. As with any rapidly growing field, it becomes increasingly difficult to stay up-to-date or enter the field as a beginner. While several survey papers on particular sub-problems have appeared, no comprehensive survey on problems, datasets, and methods in computer vision for autonomous vehicles has been published. This monograph attempts to narrow this gap by providing a survey on the state-of-the-art datasets and techniques. Our survey includes both the historically most relevant literature as well as the current state of the art on several specific topics, including recognition, reconstruction, motion estimation, tracking, scene understanding, and end-to-end learning for autonomous driving. Towards this goal, we analyze the performance of the state of the art on several challenging benchmarking datasets, including KITTI, MOT, and Cityscapes. Besides, we discuss open problems and current research challenges. To ease accessibility and accommodate missing references, we also provide a website that allows navigating topics as well as methods and provides additional information.

 
CGV-079