Mechanistic interpretability

Neural Networks Are Already Clean

Goodfire argues that superposition was the wrong explanation for interpretability's canonical messy neurons: redoing OpenAI's neuron project, about 70% of neurons look like roughly one thing, against under 2% originally. The 2014 InceptionV1 lacked residual connections and LayerNorm, so some of the mess was architecture and tooling. That makes the Waymo escalation layer the opposite case: a human layer you can see and staff, where the network inside may be more readable than assumed.