Sacha Arnoud — Director of Engineering and head of perception at Waymo, formerly on Google's Street View team leading the Street Smart deep-learning project.
The gist
Sacha Arnoud, Waymo's director of engineering and perception lead, gives an MIT lecture tracing self-driving from Google's 2009 Chauffeur project to Waymo's driverless milestone of removing safety drivers in late 2017. He explains the rise of deep learning inside Google, from Street View house-number recognition to real-time embedded perception on the cars. He details perception techniques: sensor fusion across cameras, radar, and lidar; segmentation; single-shot detection; embeddings; and recurrent networks for behavior prediction. He emphasizes that algorithms are only part of the work, with labeling, compute, TensorFlow infrastructure, and a three-pronged testing program of real driving, simulation, and a structured test facility. He closes with future directions around expanding operating domains and deeper semantic understanding.
Big reveals
- Waymo removed the safety driver entirely for the first time in November 2017, a major confidence and maturity milestone.
- Waymo has been continuously operating fully driverless cars in the Phoenix/Chandler, Arizona area since that event.
- Waymo completed ten challenging 1,000-mile loops across California 100% autonomously back in 2010.
- The first deep-learning application to succeed in production across Alphabet was Street View house-number recognition in 2012.
- Waymo crossed three million autonomous miles in May 2017 and four million by November 2017.
- Waymo drove 2.5 billion simulated miles in data centers in a single year, three orders of magnitude beyond real driving.
- Waymo runs a simulated fleet of 25,000 cars 24/7 in Google data centers.
Things worth remembering
- 94% of US crashes today involve human error, much of it avoidable distraction.
- Arnoud's mantra: when you are 90% done you still have 90% to go, requiring 10x improvements across team, sensors, and quality.
- The word2vec embedding space famously yields king minus man plus woman approximately equals queen.
- Waymo's four million driven miles represent roughly 300 years of human driving equivalent, about 60 trips around the globe.
- Waymo built a 90-acre structured test facility on a former Air Force base in California with traffic lights and railroad crossings.
- Exhaust smoke can trigger false lidar laser points, which the system must learn to safely ignore.
- A car reflected in a bus can fool naive detection into seeing two cars, requiring semantic understanding to discard.
- A mannequin-like figure lying in a pickup truck bed shows why the car must understand a 'pedestrian' traveling with a vehicle is not on the road.