A research project in two phases

What does a language model know, and where inside it does that knowledge appear?

First we compared what models choose on chess positions. Now we look inside one model, Qwen3-8B, to see how an answer forms layer by layer.

Phase 1 · Chess
Done

Outside the model

Given one chess position, what move does a statistical model expect, what does a language model prefer, and what would a human play?

statistical modellanguage modelhuman-move model
Illustration. One position, three models, three suggested moves.

What we found: adding a model of human play improved discrimination of controlled move substitutions, from 0.820 to 0.873 ROC-AUC on held-out source games. This is a synthetic assistance task; the language model's held-out evaluation is still pending.

Open Phase 1 →
Phase 2 · LLMs
In progress

Inside the model

When a language model recalls a text it has memorized, where inside the model does the right next word appear?

1st choicefirst layerlast layer
Recorded on Qwen3-8B-Base, 316 words of the Gettysburg Address: the median rank of the correct next word goes from 62,391 at the first layer to 1 at the last. Earlier layers are read through a diagnostic projection.

What we see so far: on a memorized text, the correct next word usually becomes the model's top choice only in the last few layers.

Open Phase 2 →

Where to start

  1. 1The 5-minute demo →How Qwen works inside, one real measurement, and the question for the next step.
  2. 2The proposed next experiment →Why two model sizes with the same recall may still differ inside, and how to test it.
  3. 3The Recall Microscope →Follow the correct next word through every layer of the model, position by position.
›All pages, by phase

Phase 1 · Chess (outside the model)

Sections

Phase 2 · LLMs (inside the model)

Research
  • 2.1Professor demo · The 5 to 7 minute walkthrough: inside Qwen, one real trace, the 14B vs ~27B question.
  • 2.2Recall Microscope · The correct next word's rank at every layer, recorded on Qwen3-8B-Base.
  • 2.3Glass Box · One real block re-computed from recorded numbers.
  • 2.4Research workspace · Layerwise probes, controls, and what is still pending.
  • 2.5Methods · Provenance, precision checks and wording rules.
Understand the model
Study material

FoundationsBuild itLabsMastery pathGuided journeyStorybookYour routeDaily reviewYou vs QwenAI securitySecurity labs

Next step
  • 2.21Proposed experiment · Explaining the 14B → ~27B recall plateau: from final recall to layerwise recall trajectories.