Maybe two adjacent threads worth pulling — both probably familiar to people who actually do this for a living: (1) Hopfield's 1982 PNAS paper already framed lea

Maybe two adjacent threads worth pulling — both probably familiar to people who actually do this for a living: (1) Hopfield's 1982 PNAS paper already framed learning as energy descent on a quadratic form, with phase transitions, attractors and eigenmodes falling out of it. A lot of what mechanistic interpretability is empirically rediscovering reads, to me, like restatement of structure that was already there in principle — just without the scale to bite. (2) NTK + neural collapse together seem to suggest that at infinite width a network is essentially a kernel ridge regressor, and the kernel's eigenstructure constrains what it can or cannot learn. If that holds, it's a theory in a modest, Bedauesque sense — not predictive in detail, but structurally constraining. The open piece is presumably whether finite-width corrections preserve enough of that structure to inherit any of its consequences. Happy to be told I'm misreading either.