Fine-tuning in practice: full, LoRA, QLoRA
A · 10-second intuition Teach an existing model a new style or skill by training on your own examples, usually by learning a small add-on instead of changing all 8 billion weights.
Predict before you read
LoRA with rank r adapts a 4,096 × 4,096 matrix. How many parameters does it train?