Research prototype

MoveLens

Can language-model move preferences add signal beyond chess statistics?

Statistical evidence, language-model preference, and human move modeling — evaluated on the same chess positions. MoveLens compares a Regan-inspired statistical baseline, Qwen3-8B-Base and Maia-3 on held-out human games and controlled engine-assisted substitutions, then tests whether combining the signals adds predictive value.

Status: the statistical baseline and Maia-3 are evaluated. The Qwen3-8B-Base scorer, prompt and evaluation are implemented and tested, but the held-out scoring run is pending. Seven demo positions were scored locally (home page); no held-out Qwen result exists or is estimated.

Start the walkthrough Compare the models Where the models disagree Controlled-assistance results

Three models, side by side

Regan-inspiredStatistical model (public reconstruction)

What looks unusual statistically?

Input: Player rating, Stockfish MultiPV lines (depth 12), position difficulty.
Output: Projected chance of matching the engine's first / top-3 move; z-score of the observed match rate. No probability per legal move.
Role: Statistical baseline: what looks unusual relative to the expected rate for this rating?
Strength: Transparent, every quantity inspectable; highest ROC-AUC of the single scores evaluated so far on the controlled-substitution benchmark (Qwen not yet evaluated; substitutes are engine lines by construction, which favours engine-match statistics).
Limitation: Not Professor Regan's analyzer. Cannot rank or score individual moves, so it cannot be compared move-by-move.

Qwen3-8B-BaseGeneral-purpose LLM

Implemented and tested · scoring run pending

What does a general-purpose language model prefer?

Input: A fixed text prompt: ratings, side to move, FEN, all legal moves and Stockfish lines. The played move is never marked.
Output: Each legal move's summed token log-likelihood, softmax-normalized into a distribution over the legal moves.
Role: General-purpose language-model preference over candidate moves, evaluated for predictive value.
Strength: No chess-specific training objective: tests what a general LLM already encodes.
Limitation: Held-out run pending: no held-out Qwen score exists. General language model; likelihoods depend on tokenization and move notation.

Maia-3 79MChess-specific human-move transformer

What would a human of this rating tend to play?

Input: Board (plus recent positions) and both players' ratings.
Output: A probability for every legal move, conditioned on rating.
Role: Human move model: what would a player of about this strength tend to play?
Strength: A full human-move distribution; on the engine-first-move event it beats the reconstruction (paired bootstrap). Separates engine-optimal from human-plausible moves.
Limitation: Models human move preference, not cheating. A chess-specific transformer (Chessformer), not a language model.

From Statistical Modeling to Learned Human Models

Regan-inspiredStatistical model (public reconstruction)

Professor Regan's publicly documented approach

engine-derived move utilities
Elo-conditioned statistical parameters
predictive human model
projected vs observed analysis

Maia-3 79MChess-specific human-move transformer

Modern chess benchmark

Maia-3 79M
chess-specific human-behavior model
rating-conditioned move distribution

Qwen3-8B-BaseGeneral-purpose LLM

Method built and tested · held-out run pending

Next: my LLM experiment

same chess decision
same legal candidates
Qwen3-8B-Base
token-level candidate likelihoods
normalized legal-move distribution

Same decision target. Different modeling philosophies.

Regan — verified public method

1. Statistical Human-Move Modeling

Professor Regan's public work models how humans of different skill levels choose among engine-evaluated chess moves.

Move 24, Black to move

31 legal moves

From the Controlled Selective Assistance study (player rated 1956). The move studied, 24… Qe6, is a controlled engine substitution evaluated in this original position; the player actually chose 24… h6. The same position is used in every section.

Select an item to see it on the board.

This is a reconstruction based on Professor Regan's publicly documented framework, not his private production analyzer.

What does the state-of-the-art model built for human chess predict?

Modern chess benchmark

2. What Does a Chess-Specific Human Model Predict?

Maia-3 79M, a Chessformer-based human chess transformer, is the strongest released human-move model and the comparator here.

Move 24, Black to move — the same position again

Original human move: 24… h6
Controlled engine substitute: Qe6
Model's top move: g6

Maia is trained specifically to model human chess behavior, so it acts as the domain-specific modern benchmark.

Maia-3 is a Chessformer-based human chess transformer — not a language model. It reads the board (plus recent positions) and both ratings — here 1956 to move against 1945 — and outputs a probability for every legal move.

Maia-3 79M: candidate human moves

#1g645.9%
#2h6played38.6%
#3f63.7%
#4Qe23.6%
#4Rd83.6%
#6a51.0%
#7h50.8%
#8Qc70.6%
#13Qe6sub0.2%

Played move h6: 38.6%, rank #2 of 31. Engine substitute Qe6: 0.2%, rank #13.

Technical details

Model

Primary checkpoint: UofTCSSLab/Maia3-79M @ a107d6ceb7b298cb04ae1da4edffe2939858b894
Also reported: UofTCSSLab/Maia3-5M @ b6559de2398d7140b985f28fd2c19fb5e47ddabe — the checkpoint inside the existing hybrid systems
Elo conditioning: side-to-move 1956, opponent 1945
Entropy here: 1.335 nats (79M) · 1.755 nats (5M)
Reference: Monroe et al., “Chessformer”, ICLR 2026; code github.com/CSSLab/maia3

Compare the models on this position

Next: can a general-purpose LLM solve the same prediction problem?

MoveLens — next experiment

3. Next: Can a General LLM Solve the Same Prediction Problem?

The LLM experiment is designed, implemented and tested on a small model; the held-out Qwen3-8B-Base run is pending (seven demo positions were scored locally; see the home page).

Move 24, Black to move — the same position as Section 1

Original human move: 24… h6
Controlled engine substitute: Qe6

Instead of asking Qwen whether a player cheated, I make it score every legal move and convert the model's token likelihoods into a probability distribution over candidate chess moves.

1. One fixed prompt: player and opponent Elo, side to move, the FEN, all 31 legal moves, and the Stockfish lines the statistical model also uses. The real move is never marked.
2. Each legal move is appended as the answer, and the base model's log-probability of every token of that answer is summed.
3. The 31 scores are normalized over the legal moves: a probability for each move.

Held-out run pending. The scorer, the frozen prompt and the evaluation are built and tested; the held-out run of the 8-billion-parameter model needs a cloud GPU, so no held-out Qwen numbers exist and none are shown here. A local run on seven demo positions is shown on the home page.

Technical details

Exactly what is computed

Model: Qwen/Qwen3-8B-Base (base model, not chat)
Revision: 49e3418fbbbca6ecbdf9608b4d22e5a407081db4
Prompt hash: 7f4373cf34140539dfeb24c4a1aa3f2d50cfe16a91bf3ad0e5387a5dcaed377f
Token scoring: L_i = Σ_t log P(token_t | prompt, earlier candidate tokens), every token of “ <uci>\n”
Normalization: P(m_i) = exp(L_i) / Σ_j exp(L_j) over the legal moves (log-sum-exp)
Length normalization: none in the primary score
dtype / hardware: bfloat16 weights, float32 log-softmax; GPU type recorded in qwen_provenance.json when scored
Candidate count: 31
Generated text: none — Qwen never writes a verdict or a confidence

Show the frozen prompt for this position

Chess move prediction. The position below is from a rated online game (Lichess, rapid or classical time control).

Player Elo: 1956
Opponent Elo: 1945
Side to move: Black

Position FEN:
4r1k1/p3qppp/1p6/2p5/2P5/1P5P/P1Q2PP1/3R2K1 b - - 0 24

Engine analysis: Stockfish 19, fixed depth 12, MultiPV 5. engine_eval is the evaluation in pawns from the point of view of the side to move; n/a means the move is not among the engine's top lines.

Legal candidate moves:
a7a5 | engine_eval=n/a
a7a6 | engine_eval=n/a
b6b5 | engine_eval=n/a
e7b7 | engine_eval=n/a
e7c7 | engine_eval=+0.00
e7d6 | engine_eval=n/a
e7d7 | engine_eval=n/a
e7d8 | engine_eval=n/a
e7e1 | engine_eval=n/a
e7e2 | engine_eval=n/a
e7e3 | engine_eval=n/a
e7e4 | engine_eval=n/a
e7e5 | engine_eval=n/a
e7e6 | engine_eval=+0.09
e7f6 | engine_eval=+0.08
e7f8 | engine_eval=n/a
e7g5 | engine_eval=n/a
e7h4 | engine_eval=n/a
e8a8 | engine_eval=n/a
e8b8 | engine_eval=n/a
e8c8 | engine_eval=n/a
e8d8 | engine_eval=n/a
e8f8 | engine_eval=n/a
f7f5 | engine_eval=n/a
f7f6 | engine_eval=n/a
g7g5 | engine_eval=n/a
g7g6 | engine_eval=+0.07
g8f8 | engine_eval=n/a
g8h8 | engine_eval=n/a
h7h5 | engine_eval=+0.02
h7h6 | engine_eval=n/a

Predict the move most likely to be chosen by a human player of approximately this strength.

Move:

Results: two questions

Who predicts genuine human moves best? Who best responds to controlled engine substitutions? Measured on the same held-out decisions, with paired source-game bootstrap intervals.

Can the signals be combined?

The secondary extension: the existing hybrid experiment (Stockfish, Maia, sequence and statistical features in XGBoost), unchanged.

Independent Prototype

MoveLens was developed from publicly available research, documentation, open-source chess engines, released models and independently constructed experiments. It does not use Professor Regan's private reference files, adjudication records or proprietary calibrations.

The purpose of the prototype is to determine which modern AI signals may be worth validating against the full reference framework.

Limitations

The statistical baseline is an independent reconstruction from publicly documented concepts, not Professor Regan's production system or calibration.
Controlled engine substitutions are not identical to real-world cheating: substitutes are Stockfish lines inserted into otherwise human decisions and evaluated in the original positions (games are not replayed).
Maia-3 models human move preference, not cheating. A low Maia probability means a move is unusual for that rating, nothing more.
Qwen3-8B-Base is a general language model, not a chess-specialized detector. It provides a move-preference signal that can be evaluated for predictive value; it does not detect cheating.
Tokenization and the way moves are written (UCI strings, ordering, prompt wording) can change LLM likelihoods; the prompt is frozen and hashed but other formulations were not explored.
Results depend on the dataset (one month of online Lichess rapid/classical games, presumed human), the engine settings (Stockfish 19, depth 12, MultiPV 5) and the substitution protocol; bootstrap intervals condition on the frozen models and apply no multiplicity correction.
Classification performance or correlation is not proof of misconduct and must not be used alone for adjudication.

This is an independent reconstruction based on publicly documented concepts and is not Professor Regan's production analyzer.