Lab / Poker demo
Poker demo
Play Texas hold’em against an agent I trained by self-play. The hold’em engine I wrote in C++ and the agent’s network run in this browser tab, compiled to WebAssembly: you take one seat and three copies of the agent take the other three.
How it works
The opponents are one small neural network, 36,871 parameters, trained by self-play and playing all three agent seats. The table uses the format it was trained and measured in: four seats, blinds of 1 and 2 chips, and every stack reset to 200 chips (100 big blinds) before each hand, so nothing carries over from one hand to the next. The button moves one seat each hand.
What was measured: on 40,000 duplicate deals against three copies of each of three simple bots, with the rule below on, the agent won +365 [338, 392] big blinds per 100 hands against bots that act at random, +357 [340, 375] against bots that always call or check, and +27 [23, 32] against tight bots that play a narrow range of starting hands; 95% intervals over deals (bench/RESULTS.md, section 6, “After selection”, at commit 845f46f).
The agents play with a rule added after the network was selected, so it is not part of the pre-registered evaluation: they may not fold when they could check for free. Without it the selected network often did. With it, the agent did better against the calling and tight bots; against the random bots the effect was not clear.
The limits. The agent was measured against those three bots only, not against people. It was measured one copy at a time against three copies of a bot, never three copies of the agent at one table, which is what this page seats. It was evaluated with the button fixed on one seat and the agent rotated through the seats, not with a moving button as here. It often re-raises by the minimum, so a hand between agents can run to dozens of decisions. And the engine’s rules are simplified where the repository says so: a raise is only to the current bet plus one big blind, half the pot or the pot, or all-in, and an all-in for less than a full raise reopens the betting.
The agents act one decision at a time: the first 6 in a row about 0.9 s apart, any after that 0.35 s apart, because the long hands are chains of minimum re-raises. Pause, Next action and Skip control the pace. With reduced motion set on your device, nothing moves on a timer: the agents act when you press Next action or Skip.
Everything runs in this browser tab. The deck is shuffled from a seed drawn with crypto.getRandomValues, and nothing is sent anywhere. Because the whole table runs here, the agents’ cards exist in the page’s memory: the page hides them until a showdown, but anyone inspecting the browser’s memory could read them. Nothing is at stake; the chips are only a score. The module is PokerRLPlayground’s web/ build from release v0.1.0, commit 57b741f, compiled with Emscripten 6.0.13; its agent chose the same action as the original PyTorch network in all 2,674 recorded test decisions.