{"schema":"movelens.demo.v1","generator":"experiments/22_export_movelens_demo.py","generated_utc":"2026-10-07T18:02:01Z","label_rule":{"quantity":"probability the statistical model assigned to what happened (engine #1 played or not)","unusual_below":0.25,"somewhat_unexpected_below":0.5},"models":{"statistical":{"name":"Regan-inspired statistical model","code":"services/analyzer/models/projection.py","method":"logistic-regression difficulty model + Elo band x difficulty decile projection table (training split), kappa-scaled z-scores (validation split). Not KNN."},"stockfish":{"version":"Stockfish 19","depth":12,"multipv":5,"threads":1},"maia":{"checkpoint":"UofTCSSLab/Maia3-79M","revision":"a107d6ceb7b298cb04ae1da4edffe2939858b894"},"qwen":{"model_id":"Qwen/Qwen3-8B-Base","revision":"49e3418fbbbca6ecbdf9608b4d22e5a407081db4","prompt_hash":"7f4373cf34140539dfeb24c4a1aa3f2d50cfe16a91bf3ad0e5387a5dcaed377f","prompt":{"version":"chessassistlens.qwen-prompt.v1","condition":"regan_aligned","template":"Chess move prediction. The position below is from a rated online game (Lichess, rapid or classical time control).\n\nPlayer Elo: {elo}\nOpponent Elo: {opp_elo}\nSide to move: {side}\n\nPosition FEN:\n{fen}\n\nEngine analysis: {engine}. engine_eval is the evaluation in pawns from the point of view of the side to move; n/a means the move is not among the engine's top lines.\n\nLegal candidate moves:\n{candidates}\n\nPredict the move most likely to be chosen by a human player of approximately this strength.\n\nMove:","line_format":"{uci} | engine_eval={ev}","candidate_format":" {uci}\n","engine":"Stockfish 19, fixed depth 12, MultiPV 5","eval_rule":"clip(cp, -1000, 1000) / 100, formatted %+.2f; moves outside MultiPV -> n/a","move_order":"lexicographic UCI","few_shot_examples":0,"sha256":"7f4373cf34140539dfeb24c4a1aa3f2d50cfe16a91bf3ad0e5387a5dcaed377f"},"status":"precomputed-local","scope":"7 demo positions only","runtime":{"model_id":"Qwen/Qwen3-8B-Base","revision":"49e3418fbbbca6ecbdf9608b4d22e5a407081db4","dtype":"bfloat16","quantization":null,"hardware":"laptop CPU with CPU-RAM + disk offload (accelerate device_map='auto'; modules per device {'cpu': 16, 'disk': 24}; no layers on the 4 GB RTX 3050 Ti)","modules_per_device":{"cpu":16,"disk":24},"candidate_batch":4,"torch":"2.14.1+cu126","transformers":"5.18.0","prompt_hash":"7f4373cf34140539dfeb24c4a1aa3f2d50cfe16a91bf3ad0e5387a5dcaed377f","scoring":"sum of log-probabilities of every token of ' <uci>\\n'; softmax over legal moves; nothing generated","scope":"7 MoveLens demo positions; not the held-out evaluation","consistency_check":{"position_id":"li_ItVPTIJx:20","moves":["h2g3","a1b1"],"max_abs_diff_nats":0.08283466333523393},"generated_utc":"2026-10-07T18:01:43Z"}}},"examples":[{"role":"agree","title":"Models mostly agree","rule":"Statistical reading 'expected', Maia-3 rank of the played move <= 3 and, if Qwen was scored, Qwen rank <= 3; the featured position if it qualifies, otherwise the qualifying pool position with the highest Maia probability.","position_id":"li_aVMHaGzT:47","fen":"4r1k1/p3qppp/1p6/2p5/2P5/1P5P/P1Q2PP1/3R2K1 b - - 0 24","move_no":24,"color":"black","elo":1956,"opp_elo":1945,"played":"h7h6","played_san":"h6","n_legal":31,"stockfish":{"lines":[{"move":"e7e6","san":"Qe6","cp":9},{"move":"e7f6","san":"Qf6","cp":8},{"move":"g7g6","san":"g6","cp":7},{"move":"h7h5","san":"h5","cp":2},{"move":"e7c7","san":"Qc7","cp":0}],"played_rank":null,"played_loss_cp":null},"statistical":{"features":{"elo":1956,"eval_before_cp":9.0,"gap12_cp":1.0,"near_optimal_moves":5,"legal_moves":31,"eval_swing_cp":10.0,"phase":"middlegame"},"difficulty":0.81843,"p_t1":0.17023,"p_t3":0.42352,"expected_loss_cp":37.936678,"engine_match":false,"p_observed_outcome":0.82977,"label":"expected","game_z_t1_original":0.202688},"maia":{"played_p":0.38623,"played_rank":2,"top3":[{"move":"g7g6","san":"g6","p":0.458659,"rank":1},{"move":"h7h6","san":"h6","p":0.38623,"rank":2},{"move":"f7f6","san":"f6","p":0.037065,"rank":3}],"entropy":1.33502},"qwen":{"played_p":0.147027,"played_rank":3,"top3":[{"move":"e7e6","san":"Qe6","p":0.272324,"rank":1},{"move":"g7g6","san":"g6","p":0.242582,"rank":2},{"move":"h7h6","san":"h6","p":0.147027,"rank":3}],"played_logprob":-4.617164,"played_n_tokens":5,"entropy":2.093729,"js_vs_maia":0.234133},"qwen_prompt":"Chess move prediction. The position below is from a rated online game (Lichess, rapid or classical time control).\n\nPlayer Elo: 1956\nOpponent Elo: 1945\nSide to move: Black\n\nPosition FEN:\n4r1k1/p3qppp/1p6/2p5/2P5/1P5P/P1Q2PP1/3R2K1 b - - 0 24\n\nEngine analysis: Stockfish 19, fixed depth 12, MultiPV 5. engine_eval is the evaluation in pawns from the point of view of the side to move; n/a means the move is not among the engine's top lines.\n\nLegal candidate moves:\na7a5 | engine_eval=n/a\na7a6 | engine_eval=n/a\nb6b5 | engine_eval=n/a\ne7b7 | engine_eval=n/a\ne7c7 | engine_eval=+0.00\ne7d6 | engine_eval=n/a\ne7d7 | engine_eval=n/a\ne7d8 | engine_eval=n/a\ne7e1 | engine_eval=n/a\ne7e2 | engine_eval=n/a\ne7e3 | engine_eval=n/a\ne7e4 | engine_eval=n/a\ne7e5 | engine_eval=n/a\ne7e6 | engine_eval=+0.09\ne7f6 | engine_eval=+0.08\ne7f8 | engine_eval=n/a\ne7g5 | engine_eval=n/a\ne7h4 | engine_eval=n/a\ne8a8 | engine_eval=n/a\ne8b8 | engine_eval=n/a\ne8c8 | engine_eval=n/a\ne8d8 | engine_eval=n/a\ne8f8 | engine_eval=n/a\nf7f5 | engine_eval=n/a\nf7f6 | engine_eval=n/a\ng7g5 | engine_eval=n/a\ng7g6 | engine_eval=+0.07\ng8f8 | engine_eval=n/a\ng8h8 | engine_eval=n/a\nh7h5 | engine_eval=+0.02\nh7h6 | engine_eval=n/a\n\nPredict the move most likely to be chosen by a human player of approximately this strength.\n\nMove:"},{"role":"stat-vs-maia","title":"Statistical model and Maia disagree","rule":"Over all held-out test decisions: the played move is Stockfish's #1 although the statistical model projected only <= 25% for that, and Maia-3 gives the played move 50-95%; highest Maia probability.","position_id":"li_Xoxn2IZI:58","fen":"8/4k3/p1r1p1R1/6p1/3p4/8/Pr3PPP/4R1K1 w - - 0 30","move_no":30,"color":"white","elo":2258,"opp_elo":2263,"played":"g6g5","played_san":"Rxg5","n_legal":26,"stockfish":{"lines":[{"move":"g6g5","san":"Rxg5","cp":-44},{"move":"f2f3","san":"f3","cp":-44},{"move":"g1f1","san":"Kf1","cp":-44},{"move":"h2h4","san":"h4","cp":-46},{"move":"h2h3","san":"h3","cp":-56}],"played_rank":1,"played_loss_cp":0},"statistical":{"features":{"elo":2258,"eval_before_cp":-44.0,"gap12_cp":0.0,"near_optimal_moves":5,"legal_moves":26,"eval_swing_cp":2.0,"phase":"endgame"},"difficulty":0.771847,"p_t1":0.221729,"p_t3":0.549889,"expected_loss_cp":23.742794,"engine_match":true,"p_observed_outcome":0.221729,"label":"unusual","game_z_t1_original":2.485354},"maia":{"played_p":0.948846,"played_rank":1,"top3":[{"move":"g6g5","san":"Rxg5","p":0.948846,"rank":1},{"move":"g2g3","san":"g3","p":0.014184,"rank":2},{"move":"h2h3","san":"h3","p":0.012323,"rank":3}],"entropy":0.300198},"qwen":{"played_p":0.07454,"played_rank":5,"top3":[{"move":"f2f3","san":"f3","p":0.236057,"rank":1},{"move":"h2h3","san":"h3","p":0.145783,"rank":2},{"move":"f2f4","san":"f4","p":0.104675,"rank":3}],"played_logprob":-5.222478,"played_n_tokens":5,"entropy":2.628931,"js_vs_maia":0.485464},"qwen_prompt":"Chess move prediction. The position below is from a rated online game (Lichess, rapid or classical time control).\n\nPlayer Elo: 2258\nOpponent Elo: 2263\nSide to move: White\n\nPosition FEN:\n8/4k3/p1r1p1R1/6p1/3p4/8/Pr3PPP/4R1K1 w - - 0 30\n\nEngine analysis: Stockfish 19, fixed depth 12, MultiPV 5. engine_eval is the evaluation in pawns from the point of view of the side to move; n/a means the move is not among the engine's top lines.\n\nLegal candidate moves:\na2a3 | engine_eval=n/a\na2a4 | engine_eval=n/a\ne1a1 | engine_eval=n/a\ne1b1 | engine_eval=n/a\ne1c1 | engine_eval=n/a\ne1d1 | engine_eval=n/a\ne1e2 | engine_eval=n/a\ne1e3 | engine_eval=n/a\ne1e4 | engine_eval=n/a\ne1e5 | engine_eval=n/a\ne1e6 | engine_eval=n/a\ne1f1 | engine_eval=n/a\nf2f3 | engine_eval=-0.44\nf2f4 | engine_eval=n/a\ng1f1 | engine_eval=-0.44\ng1h1 | engine_eval=n/a\ng2g3 | engine_eval=n/a\ng2g4 | engine_eval=n/a\ng6e6 | engine_eval=n/a\ng6f6 | engine_eval=n/a\ng6g5 | engine_eval=-0.44\ng6g7 | engine_eval=n/a\ng6g8 | engine_eval=n/a\ng6h6 | engine_eval=n/a\nh2h3 | engine_eval=-0.56\nh2h4 | engine_eval=-0.46\n\nPredict the move most likely to be chosen by a human player of approximately this strength.\n\nMove:"},{"role":"qwen-vs-maia","title":"Qwen and Maia disagree","rule":"Among the 5 remaining locally scored demo positions: largest Jensen-Shannon divergence between Qwen's and Maia-3's move distributions.","position_id":"li_MY3MVlQx:40","fen":"8/rp1bpk1p/3p1pp1/p2P4/4PP2/8/PP2B1PP/2R3K1 w - - 0 21","move_no":21,"color":"white","elo":2288,"opp_elo":2361,"played":"c1c7","played_san":"Rc7","n_legal":34,"stockfish":{"lines":[{"move":"g1f2","san":"Kf2","cp":49},{"move":"h2h4","san":"h4","cp":40},{"move":"g2g4","san":"g4","cp":27},{"move":"g1f1","san":"Kf1","cp":24},{"move":"h2h3","san":"h3","cp":24}],"played_rank":null,"played_loss_cp":null},"statistical":{"features":{"elo":2288,"eval_before_cp":49.0,"gap12_cp":9.0,"near_optimal_moves":5,"legal_moves":34,"eval_swing_cp":1.0,"phase":"endgame"},"difficulty":0.7158,"p_t1":0.257345,"p_t3":0.595745,"expected_loss_cp":24.403242,"engine_match":false,"p_observed_outcome":0.742655,"label":"expected","game_z_t1_original":2.04188},"maia":{"played_p":0.949124,"played_rank":1,"top3":[{"move":"c1c7","san":"Rc7","p":0.949124,"rank":1},{"move":"g1f2","san":"Kf2","p":0.041702,"rank":2},{"move":"b2b3","san":"b3","p":0.002708,"rank":3}],"entropy":0.245547},"qwen":{"played_p":0.000426,"played_rank":33,"top3":[{"move":"g2g3","san":"g3","p":0.22122,"rank":1},{"move":"g1f2","san":"Kf2","p":0.149518,"rank":2},{"move":"h2h3","san":"h3","p":0.09688,"rank":3}],"played_logprob":-10.335077,"played_n_tokens":5,"entropy":2.610204,"js_vs_maia":0.622376},"qwen_prompt":"Chess move prediction. The position below is from a rated online game (Lichess, rapid or classical time control).\n\nPlayer Elo: 2288\nOpponent Elo: 2361\nSide to move: White\n\nPosition FEN:\n8/rp1bpk1p/3p1pp1/p2P4/4PP2/8/PP2B1PP/2R3K1 w - - 0 21\n\nEngine analysis: Stockfish 19, fixed depth 12, MultiPV 5. engine_eval is the evaluation in pawns from the point of view of the side to move; n/a means the move is not among the engine's top lines.\n\nLegal candidate moves:\na2a3 | engine_eval=n/a\na2a4 | engine_eval=n/a\nb2b3 | engine_eval=n/a\nb2b4 | engine_eval=n/a\nc1a1 | engine_eval=n/a\nc1b1 | engine_eval=n/a\nc1c2 | engine_eval=n/a\nc1c3 | engine_eval=n/a\nc1c4 | engine_eval=n/a\nc1c5 | engine_eval=n/a\nc1c6 | engine_eval=n/a\nc1c7 | engine_eval=n/a\nc1c8 | engine_eval=n/a\nc1d1 | engine_eval=n/a\nc1e1 | engine_eval=n/a\nc1f1 | engine_eval=n/a\ne2a6 | engine_eval=n/a\ne2b5 | engine_eval=n/a\ne2c4 | engine_eval=n/a\ne2d1 | engine_eval=n/a\ne2d3 | engine_eval=n/a\ne2f1 | engine_eval=n/a\ne2f3 | engine_eval=n/a\ne2g4 | engine_eval=n/a\ne2h5 | engine_eval=n/a\ne4e5 | engine_eval=n/a\nf4f5 | engine_eval=n/a\ng1f1 | engine_eval=+0.24\ng1f2 | engine_eval=+0.49\ng1h1 | engine_eval=n/a\ng2g3 | engine_eval=n/a\ng2g4 | engine_eval=+0.27\nh2h3 | engine_eval=+0.24\nh2h4 | engine_eval=+0.40\n\nPredict the move most likely to be chosen by a human player of approximately this strength.\n\nMove:"}],"pool":["li_ItVPTIJx:20","li_gJTPcrY0:40","li_MY3MVlQx:40","li_nhSbU99G:29","li_05BKPNaA:24","li_aVMHaGzT:47","li_Xoxn2IZI:58"]}