Scaling: distributed training, checkpoints, evaluation

A · 10-second intuition A model too big for one GPU is split across many: copies of the model on different data, pieces of each layer, or groups of layers.