Level 0 · Foundations

Everything an LLM is made of, from zero.

Nine short modules. Each one is a single idea you can play with, the reason an LLM needs it, and two questions to check yourself. No prior maths beyond multiplying and adding. About 15 minutes each.

The whole route, from zero to red team

Track your progress →

The modules

  1. 01Vectors and the dot productA vector is a list of numbers you can picture as an arrow. The dot product measures how much two arrows point the same way.Play with it →
  2. 02Matrices are functionsA matrix is a machine that turns one vector into another. Multiplying by it stretches, rotates and mixes the coordinates.Play with it →
  3. 03Probability and softmaxSoftmax turns any list of scores into probabilities: all positive, adding up to 1, with bigger scores getting more.Play with it →
  4. 04Loss: measuring how wrongCross-entropy loss is −log(probability the model gave to the right answer). Confident and right: near 0. Confident and wrong: huge.Play with it →
  5. 05Gradients and gradient descentThe gradient says which way is uphill for the loss. Training takes small steps the opposite way, downhill.Play with it →
  6. 06Backpropagation: the chain rule at scaleBackprop computes every gradient in one backward sweep, by multiplying local slopes along the path from the loss back to each weight.Play with it →
  7. 07Neural networks learn by fittingA neural network is matrices with simple nonlinear functions between them. Gradient descent bends it to fit the data.Play with it →
  8. 08Predicting the next tokenA language model gives a probability to every possible next token. Generating text is sampling one, appending it, and repeating.Play with it →
  9. 09Why attentionGood predictions need the right earlier words, which can be far back. Attention lets every position look back at all earlier ones and pick what matters.Play with it →