It's not obvious to me that this is the case. Chris Olah and others talk about "superposition" as a mechanism to explain "polysemantic" neurons that arise in im

It's not obvious to me that this is the case. Chris Olah and others talk about "superposition" as a mechanism to explain "polysemantic" neurons that arise in image classifiers. To me, that suggests (using very vague, hand-wavy terms) that the optimization process is attempting to pack in as many concepts as possible into the finite parameter space. Certainly the scaling of GPT-3 suggests that these larger models are not necessarily any sparser than smaller ones. [1]: https://distill.pub/2020/circuits/zoom-in/ https://distill.pub/2020/circuits/zoom-in/