3 ms·

I like the stone soup narrative on AI. It was mentioned in a recent Complexity podcast, I think by Alison Gopnik of SFI. It's analogous to the Pragmatic Program

I like the stone soup narrative on AI. It was mentioned in a recent Complexity podcast, I think by Alison Gopnik of SFI. It's analogous to the Pragmatic Programmar story about stone soup, paraphrasing: Basically you start with a stone in a pot of water — a neural net technology that does nothing meaningful but looks interesting. You say: "the soup is almost done, but would taste better given a bunch of training data." So you add a bunch of well curated docs. "Yeah, that helps but how about adding a bunch more". So you insert some blogs, copy righted materials, scraped pictures, reddit, and stack exchange. And then you ask users to interact with the models to fine tune it, contribute priming to make the output look as convincing as possible. Then everyone marvels at your awesome LLM — a simple algorithm. How wonderful, this soup tastes given that the only ingredients are stones and water.

CaptainFever 2y ago
The stone soup story was about sharing, though. Everyone contributes to the pot, and we get something nice. The original stone was there to convince the villagers to share their food with the travellers. This goes against the emotional implication of your adaptation. The story would actually imply that copyright holders are selfish and should be contributing what they can to the AI soup, so we can get something more than the sum of our parts. From Wikipedia: > Some travelers come to a village, carrying nothing more than an empty cooking pot. Upon their arrival, the villagers are unwilling to share any of their food stores with the very hungry travelers. Then the travelers go to a stream and fill the pot with water, drop a large stone in it, and place it over a fire. One of the villagers becomes curious and asks what they are doing. The travelers answer that they are making "stone soup", which tastes wonderful and which they would be delighted to share with the villager, although it still needs a little bit of garnish, which they are missing, to improve the flavor. > The villager, who anticipates enjoying a share of the soup, does not mind parting with a few carrots, so these are added to the soup. Another villager walks by, inquiring about the pot, and the travelers again mention their stone soup which has not yet reached its full potential. More and more villagers walk by, each adding another ingredient, like potatoes, onions, cabbages, peas, celery, tomatoes, sweetcorn, meat (like chicken, pork and beef), milk, butter, salt and pepper. Finally, the stone (being inedible) is removed from the pot, and a delicious and nourishing pot of soup is enjoyed by travelers and villagers alike. Although the travelers have thus tricked the villagers into sharing their food with them, they have successfully transformed it into a tasty meal which they share with the donors. (Open source models exist.)
unraveller 2y ago
First gen models trained on books directly. Latest Phi distilled textbook-like knowledge down from disparate sources to create novel training data. They are all fairly open about this change and some are even allowing upset publishers to confirm that their work wasn't used directly. So stones and ionized water go in the soup.

CaptainFever 2y ago

The stone soup story was about sharing, though. Everyone contributes to the pot, and we get something nice. The original stone was there to convince the villagers to share their food with the travellers. This goes against the emotional implication of your adaptation. The story would actually imply that copyright holders are selfish and should be contributing what they can to the AI soup, so we can get something more than the sum of our parts. From Wikipedia: > Some travelers come to a village, carrying nothing more than an empty cooking pot. Upon their arrival, the villagers are unwilling to share any of their food stores with the very hungry travelers. Then the travelers go to a stream and fill the pot with water, drop a large stone in it, and place it over a fire. One of the villagers becomes curious and asks what they are doing. The travelers answer that they are making "stone soup", which tastes wonderful and which they would be delighted to share with the villager, although it still needs a little bit of garnish, which they are missing, to improve the flavor. > The villager, who anticipates enjoying a share of the soup, does not mind parting with a few carrots, so these are added to the soup. Another villager walks by, inquiring about the pot, and the travelers again mention their stone soup which has not yet reached its full potential. More and more villagers walk by, each adding another ingredient, like potatoes, onions, cabbages, peas, celery, tomatoes, sweetcorn, meat (like chicken, pork and beef), milk, butter, salt and pepper. Finally, the stone (being inedible) is removed from the pot, and a delicious and nourishing pot of soup is enjoyed by travelers and villagers alike. Although the travelers have thus tricked the villagers into sharing their food with them, they have successfully transformed it into a tasty meal which they share with the donors. (Open source models exist.)

unraveller 2y ago

First gen models trained on books directly. Latest Phi distilled textbook-like knowledge down from disparate sources to create novel training data. They are all fairly open about this change and some are even allowing upset publishers to confirm that their work wasn't used directly. So stones and ionized water go in the soup.