Training a Backgammon Neural Network: A Practical Guide

The previous articles in this series covered why backgammon is RL-friendly and how the field arrived at modern self-play training. This one is about the practical engineering: what a training pipeline actually looks like in production, end to end. We’ll focus on choices from our pipeline (concrete decisions) rather than abstract recipes.

The loop

Production training is a continuous loop, not a one-shot job. The cycle:

Position generation → Rollout evaluation → Training → Tournament → Promote → Reclassify → Repeat

Each step takes hours to days depending on the size of the dataset and the depth of evaluation. The whole loop runs continuously on a single GPU for as long as you want the engine to keep improving — there is no «training is done» state. The model gets better as long as the loop runs.

Let’s walk through each stage.

Stage 1: position generation

You can’t train without examples. The challenge is that a randomly-sampled set of positions is heavily skewed toward unlikely game states (early opening, very late race) and undersamples the strategically interesting middle game.

Our pipeline mixes three sources:

Random positions, distribution-matched. Sample checker placements from a distribution that approximates the empirical frequencies of real game positions. This gives broad coverage of legal positions including unusual ones, which prevents the network from over-specialising to common shapes.

Self-play positions. The current network plays itself, recording every position it visits. These are biased toward the network’s strategic preferences (it tends to reach positions it considers good), but that’s also the data distribution it will face in inference, so the bias is partially desirable.

Real expert positions. Decision points extracted from real played games — usually online play archives or tournament databases. The most valuable source per position because real games concentrate on the positions humans actually face. Hardest to obtain at scale.

In our long-narde database we currently have ~11 million positions across all sources, with over 300,000 expert positions specifically. The expert positions punch well above their weight in training: adding them to the loss visibly improves equity-error on the held-out portion of the same set.

Stage 2: rollout evaluation

For each new position, you need a training label — the «correct» equity estimate. You don’t have it (no oracle). The next-best thing is a high-quality rollout from the current strongest model.

A rollout from a position means: simulate K games from that position to the end, with both sides using the engine to choose moves, and average the outcomes. K=1296 (=6⁴) is the typical value because it covers many dice trajectories. Some positions get K=10000 if you need very tight equity estimates.

Rollout cost per position varies with game length and depth tier. A 1296-game rollout at ply-2 takes seconds to minutes per position depending on hardware. Multiplied by millions of positions, the rollout step is the dominant compute cost in the pipeline.

A few practical refinements:

  • Variance reduction. Generate paired antithetic dice sequences. Each game is paired with a game using the «opposite» dice rolls; the two outcomes tend to offset noise.
  • Quasi-Monte-Carlo sampling. Instead of fully random dice, use a low-discrepancy sequence over the 21 dice combinations to ensure even coverage.
  • Truncated rollouts. For positions deep in the race, you can stop the rollout early and use the network’s evaluation as a leaf value. Saves time, slightly increases bias.

Each rollout output becomes the supervised target for that position in the next training cycle.

Stage 3: supervised training

Now you have (position, label) pairs. Train a neural network to predict the label given the position. This is the easy part — standard supervised regression.

Architectural choices:

Inputs. Encode the position as a fixed-length vector. For backgammon, ~186 inputs is standard (gnubg uses 198, our network uses 186). For long narde, 194 inputs (different starting structure, no bar). The encoding matters: redundant features (point counts AND piece counts AND total checker counts) often help generalisation.

Outputs. For backgammon: six outputs covering normal/gammon/backgammon × win/lose. For long narde: six outputs covering win, win_m, win_k (nested ordinal) plus the symmetric losing-side trio. Equity is computed downstream from the output distribution.

Network sizes. Small by modern standards. Multi-layer perceptron with ~100–300 hidden units per layer, 2–4 layers. The state space is small enough that bigger networks overfit. You’re not training GPT — you’re training a function approximator with bounded inputs and clear targets.

Phase routing. For long narde, we use one network per phase plus a unified network for early positions. Phase is detected in O(1) from checker counts; the right network is invoked at evaluation time.

Loss. Mean squared error against the rollout-derived target. No fancy losses needed; the training data quality dominates.

Training time per pass: hours, depending on dataset size and batch sizes. Multiple epochs over the same data are normal.

Stage 4: tournament gating

You’ve trained a candidate network. Don’t deploy it yet. Compare it head-to-head against the current production network in a tournament.

The mechanics: play N matches between the candidate and the champion, with each match using a separately seeded random opening. Track aggregate points scored. If the candidate wins by enough to be statistically significant (depends on N — for N=10000 matches, a few hundred points of margin is usually enough), promote it. Otherwise, reject.

Why bother with tournament gating? Because rollout-derived targets are not perfect. A model trained on noisy targets might score well on the same-distribution holdout but play worse in actual games. Tournament play is the ground truth for «is this engine actually better?»

Several tournament refinements:

  • Self-play vs anchored. Pure self-play tournaments are vulnerable to local optima. Mixing in matches against an «anchor» (a fixed strong opponent) prevents the engine from drifting toward strategies that beat itself but fail against humans.
  • Match length. Shorter matches (1-pt, 3-pt) emphasise raw position evaluation; longer matches stress cube decisions. Pipelines should test both.
  • Cube enabled vs disabled. Sometimes you want to test positional strength independently of cube decisions.

Stage 5: model promotion

The tournament winner becomes the new production model. The previous champion is retained in the model registry as a reference but no longer used in inference.

In our pipeline, model promotion is an explicit operation: db_registry::promote_winner(game_id, phase_id, winner_id). The registry stores model metadata (UUID, training date, equity-loss vs prior champion, gating thresholds) in Postgres so we can audit which models were active at any point in time.

Rejected models are kept too — sometimes a model that lost one tournament wins a later one when the data distribution shifts. We use reject-models --ids <uuid,...> to mark them explicitly.

This sounds bureaucratic for a research project, but in production where the model is serving real traffic, the audit trail matters. Bad model gets shipped → user reports a strange best-move recommendation → you need to know which model was active and how it scored against its predecessor.

Stage 6: reclassification

Now you have a new champion. The old positions in your database have labels generated by the previous champion. Are those labels still valid?

Mostly yes, mostly no. The new champion will give slightly different equity estimates for the same positions. For the next training cycle, you want labels from the strongest model you have. So you reclassify: re-rollout the existing positions with the new champion, replacing their labels.

This is expensive — you’re effectively redoing stage 2 over historical data. Practical optimisation: don’t reclassify everything. Prioritise positions where:

  • The new champion’s first-pass evaluation differs significantly from the stored label (likely a position where retraining matters).
  • The position is in the most recently sampled distribution (ages out the old data).
  • The position has high variance in its current label (low rollout count, high estimated standard error).

A continuously-running reclassifier processes the database in priority order, refreshing labels at whatever rate compute allows. Over weeks it works through the whole dataset, after which the cycle starts again with the next champion.

Common gotchas

A few practical pitfalls we’ve hit:

GPU memory contention. Running rollout generation and reclassification simultaneously on the same GPU can pile up CUDA memory and cause CUBLAS_STATUS_ALLOC_FAILED panics. The supervisor restarts the failed process but in-flight queue items are lost. Solution: serialise GPU-heavy operations or use separate GPUs.

Position source provenance. Self-play, random, and expert positions should be tagged so you can ablate the contribution of each source. Without provenance, when training quality drops, you can’t tell whether you accidentally biased the data mix.

Tournament-status vs promotion-status. A model marked status='active' doesn’t necessarily get used at inference. Inference uses a separate profile_models join from the is_active=true profile. These can drift if you forget to update the profile after promoting. The lesson: have a single source of truth for «what’s running in production» and verify periodically.

Phase boundaries. When position passes from one phase to another mid-game, the relevant network changes. Make sure the rollout uses the right network for each step, not just the network of the position you started from. Otherwise you train on rollouts that don’t reflect actual play.

How long until it’s good

We don’t publish specific PR numbers because the comparison setup isn’t reproducible enough yet (engine-vs-engine matches depend on opening books, depth tiers, match-equity tables, all of which differ slightly between systems). What we can say:

  • Initial training to club-level play: weeks of continuous loop on a single GPU.
  • To engine-competitive level: months. The marginal compute per equity-improvement gets steeper as the network approaches its capacity ceiling.
  • Continuous improvement after that: smaller and smaller per-cycle gains. Most of our recent improvements come from data-side changes (more expert positions, better rollout variance reduction) rather than architectural changes.

A full latency-and-accuracy report will accompany our public API release. Right now the engine is in private beta; access requests are at /developers.

Where to go next

The final article in this series, Inside Nardex Engine: architecture deep-dive, covers the specific stack choices we made — Rust + ONNX + CUDA, the multi-phase network setup for long narde, the HTTP API surface, and how it all fits together end to end.

If you want the engine-user perspective rather than the engine-developer one: Reading Nardex Analysis Output is the API reference.