Francois Chollet — Creator of the Keras deep learning library, AI researcher and software engineer at Google, and an outspoken voice on the future of artificial intelligence.
The gist
Francois Chollet argues against the popular narrative of an intelligence explosion, contending that intelligence is not a property of a brain in isolation but emerges from the interaction of brain, body, and environment, and that recursively self-improving systems hit exponential friction rather than exploding. He traces the origin and design philosophy of Keras and its integration into TensorFlow. He explains why deep learning is fundamentally point-by-point geometric interpolation that cannot generalize like symbolic, rule-based programs, and advocates combining the two through program synthesis. He raises strong concerns about content recommendation algorithms maximizing engagement and enabling mass manipulation of behavior. Finally, he offers his own definition of intelligence as the efficiency of turning experience into generalizable programs, and warns about the overhyping of AI causing a partial backlash.
Big reveals
- Chollet publicly questioned the entire intelligence-explosion narrative, drawing heated pushback because for many people AI has become a belief system rather than science.
- He argues science itself is a recursively self-improving superhuman system, yet its output is only linear while its resource consumption grows exponentially.
- Keras was started in February 2015, originally built mainly to provide a reusable open-source LSTM/recurrent-network implementation.
- TensorFlow lead Rajat showed up and invited Chollet to integrate the Keras API into TensorFlow, and he never returned to his old research.
- Deep neural networks can only interpolate near training points and cannot do the abstract, rule-based generalization that even a few-line sorting algorithm achieves.
- Combining AI with our digital lives enables mass manipulation and psychological control of behavior at population scale, a very real present danger.
- Chollet defines intelligence as the efficiency with which you turn experience into generalizable programs.
- He predicts no full AI winter but expects a real backlash, especially around overhyped autonomous vehicles and AGI promises.
Things worth remembering
- Apparent exponential scientific progress is really exponential resource consumption (papers, patents) tracking headcount, not actual significance of discoveries.
- Michael Nielsen measured scientific progress by having experts rate the significance of discoveries over 150 years and found a flat curve across all disciplines.
- In late 2014/early 2015 Caffe (mostly C++) was the dominant deep learning library, far more popular than Theano, driven by computer vision.
- Defining models in Python code rather than YAML config files was a deliberate, then-niche design decision for Keras.
- The 'weight agnostic neural networks' paper shows an architecture, even without trained weights, already encodes knowledge about a task.
- Rich Sutton's 'The Bitter Lesson' argues general methods leveraging computation beat hand-coded prior knowledge.
- Genetic encoding is extremely low-bandwidth, so our innate priors are a tiny, slowly-written subset of possible world knowledge, mostly shared with great apes.
- We have innate knowledge of what makes a face but cannot have DNA-encoded knowledge of male vs female faces because that distinction is too evolutionarily recent.
- The part of human DNA dealing with the brain is only on the order of megabytes.
- Chollet warns autonomous-vehicle overhype mirrors the overpromising that historically triggers AI winters.