3 ms·

When I said about a group of neurons I was talking about LLMs, but some of the same ideas probably apply. Yes, it’s probably not as simple as that, and that’s w

When I said about a group of neurons I was talking about LLMs, but some of the same ideas probably apply. Yes, it’s probably not as simple as that, and that’s why we can’t understand them. I think they just used GPT-4 to help automate it on a large scale, which could be important to help understand the whole system especially for larger models

wizzwizz4 3y ago
> I think they just used GPT-4 to help automate it on a large scale, No, they used it as a crude description language for Solomonoff–Kolmogorov–Chaitin complexity analysis. They could have used a proper description language, and got more penetrable results – and it would've raised questions about the choice of description language, and perhaps have led to further research on the nature of conceptual embeddings. Instead, they used GPT-4 to make the description language "English" (but not really – since GPT-4 doesn't interpret it the same way as humans do), and it's unclear how much that has affected the results. Here's the paper, if you want to read it again: https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html https://openaipublic.blob.core.windows.net/neuron-explainer/... Some excellent ideas, but implemented ridiculously. It's a puff piece for GPT-4, the Universal Hammer. They claim that "language models" can do this explaining, but the paper only really shows that the authors can do explaining (which they're pretty good at, mind: it's still an entertaining read).