4 ms·
Mechinterp in general is just completely undervalued right now (and agreed Anthropic's team is doing the most rigorous work, now accompanied by Goodfire). They'
by thoughtpeddler 5mo ago
Mechinterp in general is just completely undervalued right now (and agreed Anthropic's team is doing the most rigorous work, now accompanied by Goodfire). They're doing the closest work to neuroscience's in vivo 'thought-tracing', which is just the most wild science fiction sort of thing to be working on, and yet I feel the average person has no idea this sort of work is happening. When combined with the idea of the 'universal subspace hypothesis' (explored under the paper of the same name), you really start to bridge the gap from engineering to something more philosophical and spiritual. But I digress...
- gom_jabbar 5mo agoHaven't heard about the universal subspace hypothesis yet, so I appreciate the digression.
- thoughtpeddler 5mo agoYa, super interesting research area the authors explored of basically trying to answer the question: "Is there a canonical/intrinsic way that concepts/representations/information are 'stored' in the universe/reality?". They tested that by performing "spectral analysis of over 1100 models - including 500 Mistral-7B LoRAs, 500 Vision Transformers, and 50 LLaMA-8B models ... by applying spectral decomposition techniques to the weight matrices of various architectures", and concluding that "deep neural networks trained across diverse tasks exhibit remarkably similar low-dimensional parametric subspaces", showing that "neural networks systematically converge to shared spectral subspaces regardless of initialization, task, or domain". Not just philosophically interesting but also has practical implications for being smarter about how to reuse models, model merging, developing more sustainable training and inference algos, etc. Paper source: https://arxiv.org/abs/2512.05117 https://arxiv.org/abs/2512.05117
- Kye 5mo agoMaybe related: https://news.ycombinator.com/item?id=47322887 https://news.ycombinator.com/item?id=47322887
- vessenes 5mo agoYeah that layer looping is super interesting. Also, it’s memory friendly with the right inference harness.
- gom_jabbar 5mo ago[dead]
- vessenes 5mo agoVery Chomsky friendly. Interesting paper, thank you!
- bkjlblh 5mo agoit's not undervalues, many people are working on it following anthropic's lead. It just doesn't seem to be any useful, so it's even overvalued