Is TD-Gammon still competitive in 2026?

Modern engines (gnubg, eXtreme Gammon, Nardex) all comfortably exceed TD-Gammon's strength because of larger networks, better training regimes, and orders of magnitude more compute. But TD-Gammon's architecture was conceptually right — modern systems are direct descendants, not departures.

What did AlphaGo borrow from TD-Gammon?

The core idea of training a value function via self-play with TD-style updates. AlphaGo added a separate policy network and Monte-Carlo Tree Search; AlphaZero unified the policy and value into a single network and removed all human game data. But the foundational TD-from-self-play recipe is from TD-Gammon.

Why did dice-game RL move slower than Go RL after TD-Gammon?

A few reasons. Dice variance makes the credit-assignment problem noisier. Backgammon's perceived prestige in the AI research community was lower than Go's. And the practical engines (gnubg, XG) were good enough for the community, so academic interest moved on. Long narde got essentially zero academic attention.

What's left to improve in backgammon engines today?

Three areas. (1) Latency at higher depth tiers — modern hardware should support ply-3+ as default if engineering is right. (2) Long-narde-specific architectures — most engines are bolt-on adaptations of backgammon networks. (3) Hosted, queryable APIs — the ecosystem assumes you embed the engine, not call it.

TD-Gammon to Modern AI: 30 Years of Reinforcement Learning

2026-07-17

In 1992, Gerald Tesauro published a paper at IBM Research describing a backgammon program that taught itself to play through self-play, using temporal-difference learning over a neural network. Within a year of training on (then-impressive) hardware, TD-Gammon was rated near the top of human play. It was the first reinforcement-learning system to reach world-class level in a non-trivial domain.

This article traces what happened in the 30+ years since. The arc goes from TD-Gammon through the open-source engines that institutionalised its approach, through the AlphaGo era when the techniques jumped to deterministic games, into modern systems that combine search and neural evaluation. We focus on what changed, what didn’t, and what’s still open in backgammon and narde specifically.

TD-Gammon (1992)

Tesauro’s setup was deceptively simple:

Network: 3-layer feed-forward, ~80 hidden units, sigmoid activations.
Inputs: 198-element raw position encoding (checker counts per point + side-to-move + cube).
Outputs: four probabilities — win, gammon win, lose, gammon lose.
Training: self-play games. After each move, TD(λ) update with λ=0.7. Reward signal: actual game outcome at terminal.
No human game data. No opening books. No expert databases. Just self-play.

Within months of training, TD-Gammon 1.0 reached intermediate play. Versions 2.0 and 3.0 (with deeper networks and ply-2 search) reached the level of the best human players. Tesauro’s published reports describe TD-Gammon discovering moves that human experts hadn’t considered standard — particularly in opening play and back-game positions — leading to revisions in opening theory.

What made this remarkable in 1992:

The dominant view was that game-playing AI required handcrafted evaluation functions or expert knowledge encoding.
Reinforcement learning was a niche topic in academic ML, dwarfed by supervised learning and symbolic AI.
Neural networks were widely considered unable to scale to interesting problems (the prevailing «AI winter» mood).

TD-Gammon was a counter-example to all three claims. It influenced the field profoundly even though most attention at the time was elsewhere.

The open-source institutionalisation: gnubg

Following TD-Gammon, the techniques diffused into community-built engines. The most important was GNU Backgammon (gnubg), started in the late 1990s and still maintained today. gnubg used the TD-Gammon recipe — self-play TD training over a feed-forward network — but at much larger scale and with several refinements:

Multiple networks for different game phases (contact / race), routed at evaluation time.
Deeper search (ply-2 by default, ply-3 and rollouts for serious analysis).
Match-equity tables for cube decisions in match play.
A full UI, command-line tools, and PGN/SGF interoperability.

gnubg became the reference engine for the academic and competitive backgammon community. To this day, when researchers want to «check what the engine thinks,» they typically check gnubg.

In parallel, eXtreme Gammon (XG) emerged as a closed-source Windows-only engine. XG focused heavily on match-equity table accuracy and produced what’s widely considered the best cube-decision engine on the market. It has no API and no Linux/macOS versions, which limits its use in software development but doesn’t dent its dominance among serious tournament players.

These two engines covered the «good enough» space for over two decades. Academic interest in backgammon RL declined sharply after the late 1990s — partly because the practical problem felt solved, partly because attention shifted to harder games.

The Go interregnum and AlphaGo (2016)

For 15 years after TD-Gammon, deep RL on board games was relatively dormant. The next watershed was AlphaGo, which used a refined version of the TD-Gammon recipe to defeat world-champion Lee Sedol in 2016.

AlphaGo’s stack:

Policy network: trained initially from human expert games (KGS database), then refined via self-play with policy-gradient methods.
Value network: trained from self-play games using TD-style updates.
Monte-Carlo Tree Search: used at decision time, with the policy network biasing tree expansion and the value network estimating leaf values.

The intellectual lineage from TD-Gammon is obvious: a neural network trained from self-play, evaluated against game outcomes. AlphaGo added the policy network (TD-Gammon used pure value-based selection) and added MCTS (TD-Gammon used direct ply-search). But the central idea — improve a network by playing it against itself — is from 1992.

Two years later, AlphaZero (2017) generalised AlphaGo by removing the human-game initialisation entirely and using the same architecture for chess, shogi, and Go. AlphaZero played all three at superhuman level after a few hours of training on TPUs.

Why MCTS didn’t take over backgammon

AlphaZero’s results were dramatic enough that you might expect MCTS to have flooded into backgammon engines. It mostly hasn’t. Two reasons:

Dice inflate the search tree. In Go, every node in the MCTS tree is a position you’ve reached by deterministic move sequences. In backgammon, every node is followed by a chance node (dice roll) with 21 children. To get the same effective depth, you have to expand 21× more nodes per ply, or sample dice and accept variance.

Network+search already works well. gnubg with ply-2 evaluation is strong. The marginal gain from MCTS over network+ply-2 is much smaller in backgammon than the gain from MCTS over alpha-beta in chess (which AlphaZero showed). The cost-benefit of porting MCTS to dice games was harder to justify.

That said, hybrid approaches exist. wildbg (Rust, 2022+) and several research engines have experimented with sampled-MCTS for backgammon. Results are interesting but haven’t produced a dominant engine.

Modern stack: what changed since gnubg

If you wrote a backgammon engine today targeting 2026 hardware and software ecosystems, you’d make different choices than gnubg made in the late 1990s — even though the algorithmic core is the same. The differences are mostly engineering:

GPU inference. ONNX runtime + CUDA gets you sub-millisecond per-position evaluation on a contemporary GPU. gnubg runs on CPU. For multi-tenant API serving, GPU is essential.

Rust over C. Memory safety without garbage collection, strong type system for the engine’s invariants, modern build system. Engineering quality of life is dramatically better.

HTTP API. gnubg and XG are local-process tools. A 2026 engine should expose evaluations over HTTP for use in web apps, mobile apps, coaching platforms.

Multi-phase networks. gnubg uses 2 phases (contact + race). For long narde we use 4 (blocking, race, X-escaped, O-escaped) plus a unified network for early positions. The cost is small (~2MB extra weights), the gain in evaluation accuracy is meaningful.

Real expert games in the training set. TD-Gammon and gnubg trained almost exclusively from self-play. Modern training pipelines benefit from injecting real played games into the training set, especially decision points from games between strong human players. The diversity of «interesting positions» in real games is hard to reproduce with self-play alone.

Continuous training in production. The model improves while the service runs. Each new tournament champion replaces the previous one, and old positions get re-evaluated by the new champion to refine their training labels. Run indefinitely on a single GPU.

These are evolutionary, not revolutionary changes. The fundamental algorithm — TD-style learning over a neural value function trained primarily from self-play — is unchanged from 1992.

The state of long narde

Long narde got essentially zero academic attention through this entire arc. The reason is geographic and cultural: the game is dominantly played in Russia and the post-Soviet space, while the AI/RL research community concentrated on games familiar to Western researchers (chess, backgammon, Go).

This is starting to change. The Russian Ministry of Sports formalised the rules (currently order №347 of 30.04.2025), giving a stable rule reference for engineering. Several engines (Nardex among them) are now natively supporting long narde with phase-segmented networks and the proper mars/koks output structure.

But the ecosystem is decades behind backgammon’s. There’s no widely-recognised reference engine like gnubg-for-narde. There’s no equivalent of XG’s match-equity tables. Validation sets are hand-curated, not community-standardised. The work is interesting precisely because it’s young.

What’s still open

Three areas where the state of the art has clear room to improve:

Default ply-3 inference. Modern GPUs support deeper search than is currently used in real-time UIs. With careful batch processing and caching, ply-3 should be achievable as default. Latency is the constraint, not raw compute.
Variant-specific architectures. Most narde engines are adaptations of backgammon networks (modify input encoding, retrain). A from-scratch architecture designed for the long-narde state space — exploiting the head, the four phases, the prize structure — could give better accuracy per parameter.
Hosted analysis at scale. The ecosystem still assumes you bundle the engine into your app. A queryable API that handles analysis for thousands of concurrent games is a different engineering problem than a desktop tool, and we’re early in solving it.

The next two articles in this series cover the practical side: how a training pipeline actually looks (Training a backgammon NN) and what our specific architecture is (Inside Nardex Engine).