Beautiful article. I have been using neural networks since the mid-1980s and I sometimes think of a high dimension space representing a neural network, and I ha

Beautiful article. I have been using neural networks since the mid-1980s and I sometimes think of a high dimension space representing a neural network, and I have wondered if there were special places in that space. Clearly there must be an astronomically large number of “dead sub-spaces”. From the article “Well-trained transformers, regardless of task, occupy the same relatively small subspace of parameter space”