6 ms·
Stone Soup AI (2024)
- hansonkd 2y agoIn the soup story the villagers freely gave up their carrots and onions and the travelers didn't give any guarantees that they wouldn't be consumed. In the AI analogy, it is a bit closer in my mind if the travelers would say "Don't worry your onions and carrots and garnishes won't be consumed by us! Put them in the pot and we will strain them out, they are still yours to keep!" We, the villagers, are dumping our data into the AI soup with a promise that it won't be used when we are using the API or check a little "private mode" box.
- dfltr 2y agoAnd to top it all off, they're charging us for the soup, and it's getting more expensive every time we give them another ingredient.
- RodgerTheGreat 2y agoIt would be more accurate to imagine a version of the tale where the stone soup chef rifles through people's houses to collect ingredients without permission (if they were against it surely they would've opted out of his services and obtained guard dogs?), and then opened a stand to sell the soup in the town square at premium prices while tainting the wares of his fellow vendors with his leftover slop.
- disqard 2y agoYes! This nuance captures more of today's reality -- esp. the "tainting", which others have also noted (e.g. Emily Bender's "Information Oil Spill")
- jpadkins 2y agoThe analogy breaks down because physical property and intellectual property are different. When we input creative works into training sets, we do not withhold those works from someone else! Digital copies are different than scarce resources. * Also, all the AI ToS I've read have stated they will use my inputs to improve their services. I haven't seen an AI service state they won't use my inputs. * Against Intellectual Property is a good book that explores this idea https://cdn.mises.org/15_2_1.pdf https://cdn.mises.org/15_2_1.pdf
- TZubiri 2y agoHuh. I wouldn't have expected free software and libertarianism to converge on this one. https://www.gnu.org/philosophy/not-ipr.en.html https://www.gnu.org/philosophy/not-ipr.en.html It's either a horseshoe or a bipartisan line
- deleted 2y ago[deleted]
- throw10920 2y agoThe analogy is perfectly apt. When an AI is trained on work that you've produced, it steals your effort - your work and effort and sweat has been taken by the model and its users. ...unless you think that your employer should be able to withhold wages from you because there's no "physical property" that you've provided to them.
- chanux 2y agoIn the folk tale, the villagers give stuff they own, willingly. The soup chefs do not go sneakily pick stuff up. Oh but the villagers were kind of fooled into giving. OK, but it benefits everyone. No mention of soup costing money later.
- lovich 2y agoLiterally all “promises” mean nothing unless backed up by force. The government was a nice backplane to ensure that, but now that its decisions are unreliable, all interactions with other parties are under these natural law rules. I don’t think this being AI really changes the deal given that starting situation
- dredmorbius 2y agoA frequent trope, but not universally true. Many social conventions are less implemented by force than by withdrawal of cooperation. That's an aggression, but of a very mild formf, but regardless one which is remarkably effective without requiring an offensive stance or the risks concomitant to same.
- 2099miles 2y agoPsh, the companies are freely giving up the data. It is unmentioned where the villagers got the carrot initially, maybe they also stole it from the library or promised their users the carrot would not be eaten. Lol
- rezmason 2y agoI think that if we tried, we could come up with a pretty large cookbook of stone soups.
- kridsdale1 2y agoEvery soup is a Stone soup if you consider the metal pot as a stone.
- dredmorbius 2y agoOr bake in ceramic!
- lsy 2y agoAdopting this perspective would improve the quality of efforts around this technology. Instead of thinking of it as somehow creating an "intelligence", seeing it as a complex lens on the training data that is controlled by the prompt helps you understand that the output isn't generated by the model, but by people. And various existing pieces of human effort are brought into focus and collimated by nudging the lens in different directions with a "prompt". The user then gives those pieces meaning and determines whether the result is useful or not. This makes certain things more clear: notions of "truth" are not in play beyond statistical happenstance, certain efforts to make outputs uniform are more trouble than they're worth, and valuable use cases are strongly correlated with the ability and convenience of the user to confirm the usefulness of the result.
- kridsdale1 2y agoI see the models as oracular seeing-stones like a wizard might use. Ponder the orb! Probe its secrets! Holographically, all our text is encoded in there. If you know how to query.
- eMPee584 2y agooh so wonderous times ahead may they converge towards peace and prosperity for the whole galaxy
- 01HNNWZ0MV43FF 2y agoVerily winds converge with force
- xpe 2y agoUsing various metaphors carefully and fluidly is key. No one is sufficient. Not this one, nor any other. I say: go back to basics. One good foundational point is dispelling confusion and conflation around "intelligence". So many people have woefully narrow and unexamined notions of "intelligence". It wouldn't be unfair to say many people have broken definitions. Broken because they just aren't good enough to make meaningful progress in a modern world where many kinds of agents display many different kinds of intelligence. Such broken definitions are often too specific; too arbitrary; too rooted in binary thinking. Many of our current language patterns are liabilities. Not to mention corporate and organizational cultures where hazy definitions slide around and few people will admit that they don't really know what others mean by the term. Sometimes it feels like a big charade where no one wants to hurt anyone's feelings nor appear uninformed. And so it goes, some kind of elaborate mystical ritual where the confused participants lead each other further into madness. With this in mind, I find tremendous value in Stuart Russell's definition of intelligence: the ability of an agent to solve some task. An agent is anything that makes a decision: a human, an animal, a system of any kind. This definition intentionally leaves out any notion of (a) humans; (b) consciousness; (c) some arbitrary quality line. This usage cuts through so much bullsh*t. I highly recommend finding a way to shift conversations towards it wherever possible. This isn't easy in my experience. We have so much baggage and crufty thinking, even we're able to put aside our baser instincts. One might say that Russell's definition just "kicks the can down the road". I don't think so. It encourages people to define their metrics a bit more clearly -- hopefully out loud or on paper -- for a particular context. It is one step closer to clarifying things. One step in the right direction -- to stop pretending like we all know each other means -- and instead actually pose an answerable question. Now, what about "general" intelligence you say? Well, one step at a time. Wait until a group of people have demonstrated some ability to find some kind of consensus on particular tasks. It is hard work to socialize these ideas. Defining general intelligence in meaningful ways is really hard and contentious. It often becomes a lightning rod for all number of other disagreements. As one example, look at the shitstorm around various sociological attempts to measure the general aspects of intelligence in humans. Without attempting to summarize it in any detail, there has been a huge dumpster fire involving: poor statistical understanding, shoddy research, tone-deaf communication, willful misinterpretation, accusations of racism, and so on. There are pockets of truth in there, but even trying finding the core nuggets of useful truth something makes everything radioactive, depending on the context. A typical person in modern culture is usually unable to calmly make sense of these issues, and who can blame them? Statistical understanding doesn't grow on trees. The same goes for understanding machine learning theory.
- pzh 2y agoThis comparison overlooks the fact that, in the original folktale, the stone soup remains a soup —- it never turns into a ribeye steak. Similarly, in the AI version, an LLM will always remain an LLM.
- doitLP 2y agoBut everyone benefits by eating the soup and has a good time partying together. The last page of the soldiers running away before the town realizes they were tricked is interesting though.
- deleted 2y ago[deleted]
- kridsdale1 2y agoThe thesis appears to be that the CEOs are hoodwinking the populace in to giving up their cultural wealth to build proprietary systems. But TFA also mentioned Wikipedia. Crowd-RLHF trained models are the same. The people know they are volunteering their own labor and information to improve the model because the model gives them value and they want to share the value with humankind. Everybody enjoys the soup.
- xerox13ster 2y agoWe gave Wiki the info. AI took the info. These things are not the same.
- CaptainFever 2y agoYou cannot take information, as it can only be duplicated.
- gkbrk 2y agoTook the info? So the info is now gone and only the AI has it? Who deleted the original, the AI did that too?
- debo_ 2y agoI thought this was going to be about the NPC/monster AI in Dungeon Crawl Stone Soup.
- aamar 2y agoI would ask anyone making these kinds of deflationary arguments to explain if the same argument can be applied to the best of human creative work. Humans also use the raw materials of others, whether that’s words, musical scales, genres, idioms, or anecdotes. Where is the line between recapitulation and innovation? Is it a line that we think current LLMs are definitely not crossing, and definitely will not cross in the near future? If so, make that argument.
- sdwr 2y ago> Good artists copy, great artists steal
- beepbooptheory 2y agoFrom TFA: > To be fair, although the story is intended to be debunking, the folktale also has a positive moral that applies to AI. The collective resources of many humans can make something that no individual could, and that really is magical. Its not deflationary, its just about reframing, reattributing what is so impressive about LLMs. We get so caught up in the tech itself, that it exists at all, understandably considering the way the discourse goes, we don't stop to appreciate how its even possible at all; that is, all of us (broadly). So many people just cant get past Sci-Fi mentality, they make the current AI into a kind of weird but promising baby, but we can also, much more easily and nicely, consider it a beautiful reflection of human writing at large. And whats even with all this constant pressure for it be more than that? All the arguments, philosophical gotchas, weird Skinnerism... Its like you're given a perfectly good hamburger and all you can say is "this is pretty much a steak if you squint".
- aamar 2y agoEven with that paragraph, I still interpret the essay as deflationary. Even though the stone has some role to play (as a social trigger), it’s materially different than the carrots, onions, etc. (which provide actual nutrition and flavor). We can draw clear distinctions. The question is whether this difference-in-kind is real in the case of AIs. I’d respond the same way to your hamburger vs. steak analogy. Sure, sometimes the LLM gives us a fine burger and not a steak, and it’s best for us to have the right attitude in that case. But if LLM’s can produce “steaks” (that is, whatever talented humans do) in the imminent future, that has _enormous practical impact_.
- Hasu 2y agoA couple of thoughts: 1) The story of stone soup is the story of how some grifters got a free meal. I don't think it's moral instruction, or an example to be learned from, unless you are a grifter. 2) In the stone soup example and in cases like Wikipedia, the soup is freely shared with everyone, regardless of their contributions. Is AI like that, or in the AI stone soup story, are the travelers charging everyone for a bowl of soup? Doesn't that change the story quite a bit?
- sdwr 2y agoIf you take off your cynicism-tinted glasses, it's the story of how community is more than the sum of its parts, and how it sometimes needs a "beautiful lie" as a catalyst (like justice, or freedom!)
- Hasu 2y agoIf you think that community needs a group of strangers to con them into coming together and being more than the sum of its parts, you are more cynical than I am.
- jimmaswell 2y agoSociety at large depends on the collective belief in society. It would stop existing tomorrow if everyone stopped pretending it existed. Laws, court rulings, road signs, it's all imaginary, but the collective illusion allows us to accomplish a lot more than than the alternative.
- card_zero 2y agoNumbers, language, boundaries between physical objects, all imaginary. Space, time, meaning, France, you name it. Alternatively: all real.
- Terr_ 2y agoWhile I can see how it can be retold that way, the core plot-mechanic is still (A) fraud by pot-stirrers and (B) greed by participants. At each step, the participant (especially the first) is deliberately misled to believe that they can secure valuable soup for less-valuable ingredients. It is not an appeal to their better nature--in many tellings the travelers have already tried that--but an appeal to their baser nature. For it to be a positive story, one must accept that the ends have somehow justified the means.
- wbakst 2y agoi like this so much "stone soup" could be seen as a trick (to get the villagers to provide that which they were previously unwilling), but i like that it's multiple different villagers who provide individual ingredients -- it's the coming together of everyone and their individual contributions that ultimately makes the soup so good
- scrumper 2y agoI first read this story in the back of the manual for a DOS program called Fractint in the very early '90s. It was a super-fast fractal generator made by a collective called the Stone Soup Group. It's still around but the SSG disappeared years ago. The story stuck with me, I told it to my kids only a few weeks ago.
- dredmorbius 2y agoMy take on "Stone Soup" is that it was written as an allegory for cooperation, as well as, perhaps, a guide to how to induce it in the face of reluctance. Of course, intent and outcome can differ, and Gopnik takes the piece to a new place. But then, that's also in the spirit of the original as I read it (individual ingredients creating a greater whole). And of course, as with all metaphor and allegory, there are limits to the comparison. But utility as well, and the point that AI LLMs require significant additions on top of the LLM-trained stones bears pointing out.
- bigfishrunning 2y agoAI is only stone soup if a) you get charged for the soup after adding your carrots and b) they heat the water by burning your house down
- htrp 2y agoThis was one of the talks at Neurips 24 in Dec, highly recommend
- tehjoker 2y agoI think the tension about AI is that yea, it is a reflection of all of our contributions, but the benefits are privatized and potentially used to deprive people of sustenance via automating their jobs. The problem is capitalism, not the technology itself. Science and technology can be used for social good. They are the product of all of our efforts and knowledge combined with labor, yet companies make big bucks selling them back to us while also depriving people of things that they need, unless they are fortunate enough to pay, and then they provide them in the most blood sucking way possible.
- topherjaynes 2y agoGopnik is a great writer, and this is a very good take. She has a great sense of how to bring psychology to tech. Also fun fact: She's married to Alvy Ray Smith for all the computer graphics/pixar fans out there. I'd love to here them debate tech takes!
- n4michael 2y ago> To be fair, although the story is intended to be debunking, the folktale also has a positive moral that applies to AI. So maybe in this story, LLMs are not so much the stones (trickery) but rather the pot (the unlocking technology).
- rcpt 2y ago> We have a magic algorithm that will make artificial general intelligence from just gradient descent, next-token prediction, and transformers Which exec is this?
- dosinga 2y agoIt's a nice story and I get the bit about the culture, but it does overlook the fact that you don't need the stones while you very much do need the LLMs.
- gwern 2y agoThis is a terrible analogy. Stone soup is disanalogous to generative AI models in almost every way that could matter. This analogy offers no insight, and at best comes off as a pretext for redistributive policies: "you mean our generative AI models" --Bugs Bunny A stone soup does nothing by itself. It just sits there. LLMs do not just sit there: Claude and o1/o3 and r1 are increasingly active agents. Gopnik just plain ignores this, and indeed, says already obviously false things like "For some time, I’ve argued that a common conception of AI is misguided. This is the idea that AI systems like large language and vision models are individual intelligent agents, analogous to human agents. Instead, I’ve argued that these models are “cultural technologies” like writing, print, pictures, libraries, internet search engines, and Wikipedia." But last I checked, I couldn't ask 'a picture' to go research anime for me and summarize the results, nor could I put it in a self-improving loop to learn how to solve advanced math problems I can't even understand. Ingredients in soups are used up and destroyed, and can only contribute to one soup; copies of text do none of that. The villagers had to go out of their way to proactively add ingredients to the soup - indeed, that is the entire point of the original moral! OP seems to think that people donated all the data that they had "stashed away on the Internet" (what a phrase). Whereas with generative models, they don't, and in fact, that's a big reason many people are so mad. Each ingredient added to the stone soup makes up a meaningful percentage of the output; any single contribution to generative models at the billions-scale is usually invisible and that contribution can be omitted without any measurable change on just about any metric. Ingredients in soup may be tastier, but they are generally not meaningfully more nutritious. A potato cooked on its own is as nutritious and full of calories as it would be in the soup. Soups can only be eaten, and eaten once. LLMs do a lot of things, which we are still discovering, and are reusable indefinitely, and are already spreading out into fields no one dreamed of (like in psychology & economics, increasingly more research is doing 'in silico' with LLMs as humans). The villagers also donated their ingredients for free. The stone itself does nothing and the 'soup maker' likewise does barely nothing and contributes no ingredients but the stone. LLM trainers spend literally tens to hundreds of billions of dollars, including billions spent collectively on creating data through Scale etc. (Rumor has it that OA and Anthropic alone are spending hundreds of millions of dollars on expert programmers and other PhD specialties and that this is part of why their models are so much better.) Notably, the little mention of 'Kenya' implies they do it for free as they "jump at the chance" - obviously, the actual Kenyan villagers are very interested in being paid. Making the soup doesn't make making future soups cheaper nor does it make the future soups tastier; making LLMs drives experience curves which are some of the fastest ever documented, which is why the cost of high-quality outputs has dropped by multiple orders of magnitude in just years, outpacing Moore's law. So... no. There is pretty much no way in which LLM is like a 'stone soup', except in the vague sense that both involve a lot of humans at some point, I guess.
- CaptainFever 2y agoI compared stone soup to AI before, but for very different reasons. That is, you cannot convince the villagers to contribute their data/food by appealing to them; rather, you have to trick them into giving up their data/food. But the result is bigger than the sum of its parts, including the villagers (if they are willing to drink the soup). IRL, the trick is ToSes and pro-AI laws. Meanwhile, some villagers may be willing to contribute in the first place: those will be free culture advocates, public domain advocates, pirates, etc.
- antonkar 2y agoAnother metaphor: We can build the Artificial Static Place Intelligence – instead of creating AI/AGI agents that are like librarians who only give you quotes from books and don’t let you enter the library itself to read the whole books. Why not expose the whole library – the entire multimodal language model – to real people, for example, in a computer game? To make this place easier to visit and explore, we could make a digital copy of our planet Earth and somehow expose the contents of the multimodal language model to everyone in a familiar, user-friendly UI of our planet. We should not keep it hidden behind the strict librarian (AI/AGI agent) that imposes rules on us to only read little quotes from books that it spits out while it itself has the whole output of humanity stolen. We can explore The Library without any strict guardian in the comfort of our simulated planet Earth on our devices, in VR, and eventually through some wireless brain-computer interface (it would always remain a game that no one is forced to play, unlike the agentic AI-world that is being imposed on us more and more right now and potentially forever)
- 2099miles 2y agoGreat post