6 ms·
I don't think that was their point, I think their point was that neural networks 'create' their optimization space by using lengths, distances, and angles. You
by kentuckyrobby 1y ago
I don't think that was their point, I think their point was that neural networks 'create' their optimization space by using lengths, distances, and angles. You can't reframe it from a topological standpoint, otherwise optimization spaces of some similar neural networks on similar problems would topologically comparable, which is not true.
- theahura 1y agoWell, sorta. There is some evidence to suggest that neural networks learn 'universal' features (cf Anthropic's circuits thread). But I'll openly admit to being out of my depth here, and maybe I just don't understand OPs point