Evaluating LLMs: loss, benchmarks, contamination, LLM judges

A · 10-second intuition You cannot improve or trust what you do not measure, and every way of measuring an LLM has a known way of fooling you.