2 ms·
> I think they just used GPT-4 to help automate it on a large scale, No, they used it as a crude description language for Solomonoff–Kolmogorov–Chaitin complex
by wizzwizz4 3y ago
> I think they just used GPT-4 to help automate it on a large scale,
No, they used it as a crude description language for Solomonoff–Kolmogorov–Chaitin complexity analysis. They could have used a proper description language, and got more penetrable results – and it would've raised questions about the choice of description language, and perhaps have led to further research on the nature of conceptual embeddings. Instead, they used GPT-4 to make the description language "English" (but not really – since GPT-4 doesn't interpret it the same way as humans do), and it's unclear how much that has affected the results.
Here's the paper, if you want to read it again: https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html https://openaipublic.blob.core.windows.net/neuron-explainer/... Some excellent ideas, but implemented ridiculously. It's a puff piece for GPT-4, the Universal Hammer. They claim that "language models" can do this explaining, but the paper only really shows that the authors can do explaining (which they're pretty good at, mind: it's still an entertaining read).