Karl Iagnemma & Oscar Beijbom — Karl Iagnemma is president of Aptiv Autonomous Mobility (founder of nuTonomy, acquired by Aptiv in 2017); Oscar Beijbom is Aptiv's machine learning lead. Both spoke at MIT's deep-learning-for-self-driving-cars course.
The gist
In this MIT lecture, Karl Iagnemma traces the evolution of autonomous driving from the 2007 DARPA Urban Challenge through nuTonomy's Singapore robotaxi deployment to Aptiv's current operations giving paid Lyft rides in Las Vegas. He focuses on the central problem of safety and validation: why it is hard to trust neural networks in safety-critical systems, covering trust in data, implementation, and algorithms, plus rare events, regulation, and revalidation. He argues neural networks should be 'caged' inside broader safety architectures rather than used as end-to-end black boxes. Oscar Beijbom then presents the technical work: PointPillars, a fast point-cloud encoder for 3D lidar object detection, and nuScenes, a freely released annotated autonomous-driving dataset. A Q&A covers validation, 5G, cross-country deployment, and data augmentation.
Big reveals
- Aptiv runs about 75 cars from a 130,000 sq ft Vegas garage, with 30 cars connected to the Lyft network so any rider can opt into an autonomous pickup.
- Aptiv has given over 30,000 rides to more than 50,000 passengers, driven over a million miles in Vegas, with a 4.95 average star rating.
- The team rejects the idea of going from pixels to actuator commands with a single learned black box, primarily because of safety and validation concerns.
- Aptiv's approach is 'caging the learning': wrapping a powerful neural network (e.g. a trajectory proposer) inside a rigorously verifiable safety system.
- PointPillars achieved a 10-100x speedup over VoxelNet by replacing slow 3D convolutions with a simple point-net over vertical pillars.
- Aptiv publicly released the nuScenes dataset (1,000 twenty-second scenes, over 1 million 3D bounding boxes) free for academic research.
- To date the industry has driven only about 12-14 million autonomous miles total, versus the ~275 million miles RAND estimates are needed to statistically claim safety.
Things worth remembering
- Aptiv has roughly 156,000 employees and about $13 billion in revenue across ~50 countries; Karl's autonomous-driving group is about 700 people.
- In 2007 DARPA-era cars ran blade servers in the trunk, generating so much heat they needed extra air conditioners and alternators.
- Early nuTonomy test cars (Mitsubishi i-MiEV) pushed the front seat so far forward that six-foot-three Karl couldn't ride along, prompting a switch to a Renault Zoe.
- RAND estimates you must drive 275 million miles without a crash to claim a lower fatality rate than humans with 95% confidence.
- Adversarial noise imperceptible to humans can make a classifier read a turtle as a rifle, or a duct-taped stop sign as a yield sign.
- The 2012 Krizhevsky/Hinton deep-learning paper had been cited about 35,000 times as of the talk.
- VoxelNet ran at about 5 Hz, bottlenecked by slow 3D convolution; PointPillars' encoder runs in about 1.3 milliseconds.
- Aptiv designs its system not to rely on 5G, treating it as nice-to-have infrastructure outside their control.
- Aptiv uses 'rule books' to reprioritize driving rules so the same system can adapt from right-hand to left-hand driving between cities.
- To train rare-event detection, the team copy-pastes lidar point returns of objects from one sweep into another, and can simulate things like a piano on the road from a 3D model.