Colossus against eXtreme Gammon at 2-ply

#The result

Colossus, HedgeHog's strongest engine, played 20,000 cubeful money games against eXtreme Gammon 2.10, the reference program most strong players analyse with. Both engines searched at 2-ply, and Colossus won by 0.019 points per game.

The luck-adjusted interval stays above zero, so the margin is not the dice. At the plain network level, with no lookahead on either side, Colossus beats XG's networks by 0.047 points per game. The lead narrows when both engines search, but it holds.

#What was played

• Money games with the cube, Jacoby and beavers off, cube limit 64.
• Both engines at 2-ply in eXtreme Gammon's counting: every candidate play checked against all 21 of the opponent's possible replies. Cube decisions were made at the same depth on both sides. (HedgeHog's analysis normally looks one turn further ahead on a cube decision. Here Colossus was held to XG's depth.)
• XG played as itself. Every XG move and every XG cube action was eXtreme Gammon 2.10's own choice at its 2-ply setting, with the opening book off and analysis on a single thread, so that it gives the same answer every time.
• Colossus played as HedgeHog plays, through the same decision code that analyses your matches, ranking moves by their value with the cube in play.
• Colossus had White in every game. Who moves first is decided by the opening roll, so the colour gives no edge.
• The number of games was fixed at 20,000 before the first game was played, and nothing was stopped early or run again.

#Taking the luck out

A single money game swings by about three points on average, almost all of it dice, so even 20,000 raw results leave an interval about 0.09 points wide. That is the raw row above, and on its own it cannot separate the two engines.

The luck-adjusted figure removes the dice. For every roll after the opening one, a fixed reference network works out what the roll was worth: the value of the best play with the dice that came up, minus the average over all 36 rolls. That is the roll's luck, positive when it helped the player who threw it. Each game's result has the luck of all its rolls subtracted, scaled by the cube in play at the time.

Luck depends only on the position and the dice, never on what either player then chose, and it averages to exactly zero over all possible rolls. So subtracting it cannot push the result towards either engine. It only removes noise: here it made 20,000 games as precise as about 480,000 unadjusted ones. The reference network was GNU Backgammon's, neither of the two contestants, so a reference that favoured one of them could not even be suspected. The same idea makes a rollout more precise: How backgammon engines evaluate positions explains it there.

#Dice nobody chose

A result is only as good as the dice it was played with. Anyone who controls the random numbers could run the match many times and publish the best one. So these dice came from drand, a public randomness beacon run by the League of Entropy, a group of independent organisations. The match used its quicknet beacon, which publishes a new signed random number every three seconds.

Before the match started, the round to be used was written down: round 33183836, due at 22:21:12 CEST on 18 October 2026. When it was published, its first sixteen hexadecimal digits (38056030563a863f) became the seed of every roll in the match. Anyone can look the round up, and the dice of all 20,000 games follow from it alone. Nobody, the people running the match included, could have chosen them or tried several.

#How the games went

Colossus won slightly fewer games than XG, but more of its wins were gammons, and it took fewer of the doubles it was offered.

#Where the points came from

Every game ends with a number of points for the winner: one for a single game, two for a gammon, three for a backgammon, all multiplied by the cube. Counting the games by that number shows where the margin landed. The last two columns split each row's net points into what the dice handed Colossus (negative when they favoured XG) and what is left once that luck is taken out.

Over the whole match the dice favoured Colossus by only 56 points, so almost all of its 439 came from the play. Within the rows they moved a great deal. Colossus's extra four-point wins were mostly luck: 325 of their 468 points. The two-point games look even until the luck is taken out, and then they are where Colossus gained most: XG had 221 points of luck in them.

The same games sorted by the cube in play at the end. (A game that ended with a double passed is counted at the cube value it was played for, before the double.)

Games that never got past the first double are nearly all doubles that were passed, and there XG comes out ahead, with or without the luck: Colossus passed 33.5% of the doubles XG offered it, where XG passed 29.6% of Colossus's. Once a double was taken and the game went on at a cube of 2, Colossus gained 318 points after luck, most of its margin. Above a cube of 4 there are too few games to read anything into, and the dice dominate them: 53 of the 64 points of the one game that reached a 64-cube were luck.

Read the rows with one caution. Only the total is a fair measure of skill. A row is chosen by how the game ended, and both the dice and the play decide that: a game Colossus won big is partly in the four-point row because it was lucky, and a double Colossus passed puts its game in the first row of the cube table. So the rows show where the points landed, not which kind of position either engine plays better.

#Check it yourself

Everything needed to check the result is in one download: colossus-vs-xg-2ply.zip.

• All 20,000 games as .mat files, which GNU Backgammon, eXtreme Gammon and most other programs can open. Counting who won what reproduces the raw result exactly.
• The beacon round and the seed, with verify_dice.py, a short Python script that regenerates every roll of every game from the seed and checks it against the files.
• The source of the luck calculation, which replays every game, checks that every move is legal and that every result matches the board, and recomputes both rows of the table above.
• A summary of each side's error rate in the match, as eXtreme Gammon 2.10 scores it at 2-ply.

The reference network for the luck figures is GNU Backgammon's, so GNU Backgammon's own analysis of these files reports a luck figure for every roll as well, an independent check. It computes luck with its own settings, so its figures come close to these rather than matching them digit for digit.

#What this does not show

This is one depth and one kind of play: money games at 2-ply against eXtreme Gammon 2.10. It says nothing yet about matches to a fixed length, where the score changes every decision, or about deeper search, where both engines see further and the margin may move either way. Those are the next comparisons. The error rate of either engine is a different question again: an engine can lose individual decisions to another and still win the games.