4 ms·
This is a terrible analogy. Stone soup is disanalogous to generative AI models in almost every way that could matter. This analogy offers no insight, and at bes
by gwern 2y ago
This is a terrible analogy. Stone soup is disanalogous to generative AI models in almost every way that could matter. This analogy offers no insight, and at best comes off as a pretext for redistributive policies: "you mean our generative AI models" --Bugs Bunny
A stone soup does nothing by itself. It just sits there. LLMs do not just sit there: Claude and o1/o3 and r1 are increasingly active agents. Gopnik just plain ignores this, and indeed, says already obviously false things like "For some time, I’ve argued that a common conception of AI is misguided. This is the idea that AI systems like large language and vision models are individual intelligent agents, analogous to human agents. Instead, I’ve argued that these models are “cultural technologies” like writing, print, pictures, libraries, internet search engines, and Wikipedia." But last I checked, I couldn't ask 'a picture' to go research anime for me and summarize the results, nor could I put it in a self-improving loop to learn how to solve advanced math problems I can't even understand.
Ingredients in soups are used up and destroyed, and can only contribute to one soup; copies of text do none of that.
The villagers had to go out of their way to proactively add ingredients to the soup - indeed, that is the entire point of the original moral! OP seems to think that people donated all the data that they had "stashed away on the Internet" (what a phrase). Whereas with generative models, they don't, and in fact, that's a big reason many people are so mad.
Each ingredient added to the stone soup makes up a meaningful percentage of the output; any single contribution to generative models at the billions-scale is usually invisible and that contribution can be omitted without any measurable change on just about any metric.
Ingredients in soup may be tastier, but they are generally not meaningfully more nutritious. A potato cooked on its own is as nutritious and full of calories as it would be in the soup.
Soups can only be eaten, and eaten once. LLMs do a lot of things, which we are still discovering, and are reusable indefinitely, and are already spreading out into fields no one dreamed of (like in psychology & economics, increasingly more research is doing 'in silico' with LLMs as humans).
The villagers also donated their ingredients for free. The stone itself does nothing and the 'soup maker' likewise does barely nothing and contributes no ingredients but the stone. LLM trainers spend literally tens to hundreds of billions of dollars, including billions spent collectively on creating data through Scale etc. (Rumor has it that OA and Anthropic alone are spending hundreds of millions of dollars on expert programmers and other PhD specialties and that this is part of why their models are so much better.) Notably, the little mention of 'Kenya' implies they do it for free as they "jump at the chance" - obviously, the actual Kenyan villagers are very interested in being paid.
Making the soup doesn't make making future soups cheaper nor does it make the future soups tastier; making LLMs drives experience curves which are some of the fastest ever documented, which is why the cost of high-quality outputs has dropped by multiple orders of magnitude in just years, outpacing Moore's law.
So... no. There is pretty much no way in which LLM is like a 'stone soup', except in the vague sense that both involve a lot of humans at some point, I guess.
- dredmorbius 2y agoI'll grant that LLMs do more than just sit there, but what Gopnik is pointing out is that, at least as of last August (in a very rapidly progressing field), they also do not, of themselves approach AGI without the addition of numerous other ingredients, particularly training data, alignment, and prompt engineering. One could argue that in the original allegory, the stones don't merely sit either, but contribute (with some clever persuasive rhetoric) to facilitating the cooperation required to brew a compelling stew. All analogies melt if you push them loudly enough.
- gwern 2y ago> They also do not, of themselves approach AGI without the addition of numerous other ingredients, particularly training data, alignment, and prompt engineering. That's also not true. A base LLM like GPT-3, without any (specialized) training data, alignment, or prompt engineering, embodied quite a bit of agency (able to answer questions, simulate coding, or take actions in a while-loop), and in principle, can be AGI if scaled up. Consider Gato for an example: no alignment, no prompt engineering, just a GPT on data. This is unlike a stone soup which is completely un-nutritious without the villagers donating all the edible ingredients to make an actual soup, where the stone itself does nothing (a LLM definitely does something); the two scenarios are different qualitatively, not just quantitatively. Gopnik's analogy and goal for her analogy are both irretrievably flawed. > All analogies melt if you push them loudly enough. This analogy didn't melt after being pushed too far. It caught on fire spontaneously in the garage while no one was looking before half the parts arrived or had been unpacked.