In 1992, a neural network sat down to play backgammon against itself. It had no opening book, no database of grandmaster games, no human demonstrations of strategy. It had a reward signal — win or lose — and time. Lots of time. Gerald Tesauro at IBM Research let it run, and what emerged from that self-play loop was something that made professional players stop and reconsider everything they thought they knew about the game. TD-Gammon didn’t just reach human-competitive level. At its strongest versions, it identified positional strategies that human experts had been playing incorrectly for decades.
The “TD” stood for temporal difference learning, a reinforcement learning technique that propagates value estimates backward through time — crediting or blaming earlier decisions for outcomes that only became clear later. The mathematical foundations came from Richard Sutton’s work in the 1980s, itself inspired by how animal brains seem to encode prediction error signals in dopamine circuits. Tesauro took that framework, attached it to a backgammon simulator, and let the system bootstrap its own understanding of position entirely from scratch.
What makes this milestone extraordinary is not the backgammon skill, impressive as it was. It’s the proof of concept it delivered: a system could learn a complex, high-dimensional strategy problem through pure self-play without any human knowledge baked in, and arrive at genuinely novel conclusions. TD-Gammon was producing superhuman insights about the value of running strategies and back-game timing that tournament players began to actually adopt. The machine wasn’t imitating human expertise. It was generating its own.
At the time, the mainstream of AI research was still dominated by symbolic systems — expert knowledge hand-coded by human practitioners, rule-based engines that required someone who already understood a domain to laboriously transcribe that understanding into logical structures. TD-Gammon was a live demonstration that there was another path. Given a clear objective and enough self-generated experience, a learning system could build its own model of a domain’s structure.
The full implications took years to detonate. TD-Gammon was, in retrospect, a direct ancestor of AlphaGo’s policy and value networks, of AlphaZero’s ruthless self-play loops, of the reinforcement learning from human feedback pipelines that now shape large language models. The core insight — that you could use a neural network as a function approximator inside a reinforcement learning loop and get something that generalizes rather than memorizes — is threaded through nearly every major system driving AI forward today.
There’s something almost eerie about looking back at the technical setup. Tesauro’s network had roughly 40,000 weights. The training ran on hardware that would be laughably underpowered by any modern measure. And yet the fundamental machinery was there: self-generated training data, bootstrapped value estimation, a network learning to compress strategic intuition into parameters. Scale that idea up across three decades of compute improvements, algorithmic refinement, and theoretical development, and you start to understand the lineage of systems like OpenAI’s o-series models or DeepMind’s AlphaProof.
What Tesauro built also carried a quieter message about knowledge itself. Human backgammon expertise, accumulated over centuries of play, turned out to contain errors — persistent conventional wisdoms that were simply wrong, and that TD-Gammon corrected not by reasoning about them explicitly but by discovering, through sheer volume of self-generated experience, that a different approach won more often. That should have been a hint about what AI would eventually do to fields far more consequential than backgammon.
We’re still cashing the check that summer wrote. Reinforcement learning now trains robots to manipulate objects with fingertip delicacy, guides protein structure prediction past the limits of physical intuition, and powers the reasoning loops that let AI systems critique and revise their own outputs. The loop that started in a backgammon simulator never really closed — it just kept expanding into harder problems, and it shows no sign of stopping.