Features, superposition and sparse autoencoders

A · 10-second intuition A model stores far more concepts than it has dimensions by packing them in as overlapping directions; sparse autoencoders try to pull those directions back apart.