Drago Anguelov — Principal scientist at Waymo leading the research team; PhD from Stanford under Daphne Koller, formerly at Google (perception, Street View) and Zoox, working on machine learning for autonomous vehicle perception, prediction, and planning.
The gist
In this MIT lecture on deep learning for self-driving cars, Waymo principal scientist Drago Anguelov presents 'taming the long tail of autonomous driving challenges.' He walks through the core AI tasks of perception, prediction, and planning, illustrating rare real-world scenarios Waymo's cars have handled, like cyclists carrying stop signs, falling poles, and red-light runners. He frames machine learning as a 'factory' powered by infrastructure, high-quality labeled data, and models, and details Waymo's use of AutoML, simulation (7+ billion simulated miles), and learned smart agents for realistic testing. He argues that a hybrid system combining learned models with expert domain knowledge is the right approach for handling rare, safety-critical edge cases.
Big reveals
- Waymo celebrated its 10-year anniversary the month of the talk, having started as a Google moonshot under Sebastian Thrun to drive ten 100-mile segments.
- In 2015 Waymo completed the world's first fully autonomous ride on public roads in Austin, with a blind passenger inside.
- Waymo launched its first commercial driverless service in the Phoenix metro area where people can hail rides via a phone app.
- Waymo simulates the equivalent of 25,000 virtual cars driving ten million miles a day, totaling over seven billion simulated miles as a key part of its release process.
- Waymo's pipeline produces auto-labels by using knowledge of both the past and future of objects to better annotate data, then trains models to replicate that without seeing the future.
- Waymo trained a deep neural network on 60 hours of driving footage to imitate driving, even running a real car with it at the Castle air force base test grounds.
- Waymo applied Google's AutoML to its lidar segmentation and lane detection, finding network architectures that beat human engineers on quality and latency.
Things worth remembering
- Waymo had achieved over 10 million autonomously driven miles on public roads at the time of the talk.
- The 'long tail' of rare driving situations matters disproportionately; getting common cases working is one effort, handling the rest is another entirely.
- The speaker's Google team invented the Inception neural net architecture and the SSD object detector, and won ImageNet 2014.
- Waymo uses a 90-acre former air force base (Castle) to deliberately stage rare and dangerous test situations safely.
- The 'DAgger' problem, identified by Stephane Ross (now at Waymo) and Andrew Bagnell, describes how small imitation-learning errors compound over many steps.
- Waymo designs redundant complementary sensors (camera, lidar, radar) with 360-degree field of view so failures in one modality are covered by others.
- The pure imitation-learning agent failed on edge cases, such as driving off the road on a turn due to limited range and failing to complete a U-turn.
- Waymo uses inverse reinforcement learning to tune trajectory-optimization agents that produce conservative or aggressive driving behaviors for simulation testing.
- An audience member raised Kahneman's type 1 / type 2 reasoning, noting reasoning is under-explored in deep learning compared to instinctive perception.
- Neural nets can estimate their own uncertainty via direct regression, network sampling, or dropout, and via environmental consistency constraints.