I think it's incorrect that the insight has gone nowhere. See, for eg, contrastive loss / clip, or vqgan image generation. Arguably also diffusion models. More

I think it's incorrect that the insight has gone nowhere. See, for eg, contrastive loss / clip, or vqgan image generation. Arguably also diffusion models. More generally, in my experience as an AI researcher, understandings of the geometry of data leads directly to changes in model architecture. Though people disparage that as "trial and error" it is far more directed than people on the outside give credit for.