A tiny model with one output layer guesses which of four words comes next. Training compares the guess with the correct word, computes the loss, the gradient, and nudges the weights. This website never trains Qwen; only these toy weights change.
One example, one update at a time
Toy calculation· 3 inputs, 4 candidate wordsCorrect next word:
Prediction (shares)
- cat21.7%
- dog ◎25.5%
- car35.1%
- sun17.8%
Loss = −log p(target) = 1.3679
Update rule
W ← W − η · hᵀ(p − target)
Every weight moves a little in the direction that lowers this example's loss.
Predict: after one update p(dog) will…
From this toy to Qwen
Qwen3-8B-Base was pretrained with the same idea (predict the next token, cross-entropy loss, gradients through every layer by the chain rule, an optimizer step) applied to all 8,190,735,360 parameters over a very large corpus. Instruction-tuned and preference-optimized versions add further training stages; they are not claimed for the base checkpoint shown here.