4 ms·

I'll grant that LLMs do more than just sit there, but what Gopnik is pointing out is that, at least as of last August (in a very rapidly progressing field), the

I'll grant that LLMs do more than just sit there, but what Gopnik is pointing out is that, at least as of last August (in a very rapidly progressing field), they also do not, of themselves approach AGI without the addition of numerous other ingredients, particularly training data, alignment, and prompt engineering. One could argue that in the original allegory, the stones don't merely sit either, but contribute (with some clever persuasive rhetoric) to facilitating the cooperation required to brew a compelling stew. All analogies melt if you push them loudly enough.

gwern 2y ago
> They also do not, of themselves approach AGI without the addition of numerous other ingredients, particularly training data, alignment, and prompt engineering. That's also not true. A base LLM like GPT-3, without any (specialized) training data, alignment, or prompt engineering, embodied quite a bit of agency (able to answer questions, simulate coding, or take actions in a while-loop), and in principle, can be AGI if scaled up. Consider Gato for an example: no alignment, no prompt engineering, just a GPT on data. This is unlike a stone soup which is completely un-nutritious without the villagers donating all the edible ingredients to make an actual soup, where the stone itself does nothing (a LLM definitely does something); the two scenarios are different qualitatively, not just quantitatively. Gopnik's analogy and goal for her analogy are both irretrievably flawed. > All analogies melt if you push them loudly enough. This analogy didn't melt after being pushed too far. It caught on fire spontaneously in the garage while no one was looking before half the parts arrived or had been unpacked.