4 ms·

> When customers began experimenting with entering a wider range of household shopping list items So it works as intended: ask it to generate a recipe involvin

by Aachen 3y ago

> When customers began experimenting with entering a wider range of household shopping list items So it works as intended: ask it to generate a recipe involving chlorine cleaning products and it does so. Which specific ingredients the user input is not mentioned in the article, presumably because then cause and effect are obvious to anyone. Next week on the Guardian: hammer used to smash own thumb, consumers calling to remove from shelves. Terms of service said it was for 18+ users only but age not verified upon checkout!

_Microft 3y ago
OK, ... so you rather justify a recipe app accepting chemicals as inputs to blame a news site for their reporting than to identify the problem? Edit: removed a part about The Guardian.
- Aachen 3y ago
  I'm not sure what the ability to blame the Guardian has to do with anything, I'm basing the above comment solely on this submitted article and don't really know the Guardian as an organisation. I have no beef with them, if I'm reading your tone correctly. As for allowing to input chemicals: garbage in, garbage out. We accept that from tools (and some countries from guns) but not from software. Maybe that's how it should be, but I find it an interesting discrepancy. In general, as a hacker, I'm happy if a tool lets me use it for unintended purposes (such as humor in this case)
  - _Microft 3y ago
    OK, I took The Guardian part out but I think the rest is still applicable. Why is the reporting problematic but not that this problem happens in the first place?
    - veave 3y ago
      Because there's no story. It's software that generates recipes from ingredients you provide. If you provide inedible ingredients you get an inedible recipe. What did you expect would happen? The subtext here is journalists who are dead scared of a tool they know will replace them. So they have to denigrate it.
      - uoaei 3y ago
        I think something that's been left unsaid until now is that the implementation is clumsy and downright dangerous, therefore the implementation is totally botched. A supermarket whose job it is to feed people so as to keep them alive is a vital service in the vast majority of economies, and so has the duty to go a little deeper and fine-tune their model or at the very least heavily prompt-engineer their interface layer to avoid such obvious flubs. There's no point in apologizing for the LLM. Of course it's doing what it's asked to do. That's not the gripe, nor points toward any kind of solution, though.
      - Aachen 3y ago
        I agree with the first paragraph but not necessarily the latter. More likely to me seems that the journalists see something that can be construed to be harmful (so they can justify the story), as well as having a high click rate (see, it ends up on HN), and thus good for business
        saaaaaam 3y ago
        A lot of Guardian readers excoriate the Daily Mail as low quality clickbait and "not journalism", whilst seeming perfectly content to suckle from the teats of the Guardian's own clickbait farm, presumably comforting themselves that there is some sort of intellectual honesty present in the Guardian that is missing from the Daily Mail. I find it quite strange.
        Aachen 3y ago
        Not sure this "us vs. them" stuff / generalising every reader of a website is helping anyone here
        saaaaaam 3y ago
        Not to be pernickety, but I did say “a lot of” not “all of”… And in the context of this conversation which was about clickbait on the guardian I felt it was a contextually valid observation. I’m sorry you didn’t find it helpful.
- deleted 3y ago
  [deleted]
- yreg 3y ago
  I don't see why it shouldn't. Let me put in any input I want to. There's no story here.
- soco 3y ago
  If you are a "technology enthusiast" you will blame everything but the technology for failures. Is this the first time? Definitely not, we all have the running jokes of "it works on my machine" or "damn users if they only" or "it's not a bug but a feature" developer-developed blindness, where the stupid real world keeps interfering with our perfect creations.
  - bonoboTP 3y ago
    You are conflating things. If you sell a product for profit, it needs to be foolproof, so the general public can use it without shooting themselves in the foot. Capable tools have to be gatekept somehow, either through actual hard licenses or soft gates of obscure forums, hidden small communities that don't advertise themselves to the world, so they can do their thing in peace. The trouble comes when the latter kinds of communities build something useful/fun-when-used-correctly and it leaks and people demand to participate, complain that it's too hard and demand it be dumbed down. So yeah, don't broadcast your stuff to the mainstream unless it's foolproof. But also, let people be, who are satisfied with dealing with the rough-edges stuff. This is true in many contexts. Think about enthusiasts of weird drugs or other niche communities like sword swallowers. There's no principle of "you are only allowed to do things that are also safe when ignorantly attempted by a random person".
toshk 3y ago
Not really of course. Should make eatable dishes with correct nutritional value, if thats what you are offering.
- Aachen 3y ago
  From the article: > In a warning notice appended to the meal-planner, it warns that the recipes “are not reviewed by a human being” and that the company does not guarantee “that any recipe will be a complete or balanced meal, or suitable for consumption”. It is explicit in that it does not offer "dishes with correct nutritional value" Whether you should be allowed to label/market this as "meal planner", given that you can't label a milk replacement "milk replacement" in many countries, is up for debate, but the software itself is allegedly not dishonest (according to the article, I haven't tried it myself)
  - fhars 3y ago
    So they are using a software that they know is not fit for purpose. That is at least gross negligence.
    - Aachen 3y ago
      Maybe the hammer analogy I used is not great because hitting yourself is obviously bad whereas any of the recipes might be toxic in a way that the user is not aware of, in which case you're right. But the complaint is that it suggests impossible dishes upon inputting ingredients that are obviously inedible and harmful substances, which sounds to me like it is functioning correctly even if this humorous function was unintentional Who would ask this LLM to make a recipe involving bleach and then actually proceed to make it? Such a person is already at risk of poisoning themselves, the software doesn't suggest it by itself so I don't see how it increases the risk of harm
      - piva00 3y ago
        But the purpose underlining a meal-planner AI is to create meals. If it's creating something that is not a meal it's not doing it's purpose, if the use-case is for meals then there should be guardrails against non-meals recipes, simply because it does not fit the purpose of the product. It's not a LLM product, it's not advertised as a general LLM that will generate text based on input of shopping items, it's specifically advertised as an AI meal planner and it's not fit for that purpose since it does not guard against non-meals. That's the issue, the user is supposed to be a layman, not someone that knows and understands LLMs and its limitations.
pmontra 3y ago
The expected answer should be, those ingredients are not edible, I can't give you a recipe that includes them. Maybe they can fix it with a prompt that makes the model check if the ingredients are safe. Chances are that every prompt can be circumvented but they can probably find instructions for self poisoning somewhere else with less effort.
- Maxion 3y ago
  > Maybe they can fix it with a prompt that makes the model check if the ingredients are safe. Just sanitize the input and remove products belonging to categories that aren't edible. Tampons aren't great in salads, and toilet paper is not a good garnish.
- gus_massa 3y ago
  What about quicklime (calcium oxide)? It's used here in Argentina to make pumpkin "sweets". (I'm not sure the correct translation.) It's a lot of 1 inch cubes of pumpkin, boiled for some time in sugar syrup. [1] Without treatment the cubes will disintegrate while boiling. If you put them for one day in water with quicklime, the cubes get a hard wall and they keep their integrity while boiling, but the interior is soft. Drinking the water with the quicklime is dangerous and it's discarded. Eating directly the quicklime is even worse. But quicklime is useful for cooking. [1] Some random recipe I got in Google (autotranlation) https://cookpad-com.translate.goog/ar/recetas/94999-zapallo-en-almibar?_x_tr_sl=es&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=wapp https://cookpad-com.translate.goog/ar/recetas/94999-zapallo-... (spanish) https://cookpad.com/ar/recetas/94999-zapallo-en-almibar https://cookpad.com/ar/recetas/94999-zapallo-en-almibar
  - adrianN 3y ago
    Similarly, ramen noodles are made with a bit of sodium carbonate.
    - gus_massa 3y ago
      Sodium Carbonate is edible. It's used in a few recipes, like a strong version of baking soda (sodium bicarbonate) https://en.wikipedia.org/wiki/Sodium_carbonate#Food_additive_and_cooking https://en.wikipedia.org/wiki/Sodium_carbonate#Food_additive... I'm not sure how dangerous is a teaspoon of pure Sodium Carbonate alone, because it's very basic (anti-acid), but mixed with some weak acid it's used to make the fizzy power that is inside some cadies https://en.wikipedia.org/wiki/Sherbet_(powder) https://en.wikipedia.org/wiki/Sherbet_(powder) (The proportions are important, don't try silly stuff at home.) A teaspoon of quicklime in your mouth first reacts with water to produce a lot of heat that will burn you, and then the result is very basic (anti-acid), and make new burns in a different way. Definitively don't try it at home.
probably_wrong 3y ago
I believe the issue here is that, as a user, I expect the system to "do no harm". Sure, once you give bleach as an ingredient then all bets are off (although a good system would filter it out), but it highlights the deeper issue that some foods could be bad for my health and the system is letting them through. Unheated castor oil contains ricin, and even heated oil can cause contractions in pregnant women. Raw chicken can give you salmonella. A working recipe system should therefore never return "chicken sushi on a bed of vegetables with a touch of castor oil" without some SERIOUS warnings, but it's clear that this system may do so. Putting such a system in the hands of unsuspecting users is bad and they should feel bad.
- Nullabillity 3y ago
  And going further, it demonstrates that the model doesn't actually understand food or cooking, and is just smashing random words together.
  - nullindividual 3y ago
    It does further the point that LLMs are unable to understand, infer, deduce, or reason. LLMs are toddlers who are capable of stringing words together that make some sense most of the time. It’s not artificial intelligence.
- deleted 3y ago
  [deleted]
latexr 3y ago
> So it works as intended It’s literally spelled out in the article that a spokesperson for the company said this is not as they intended.
n2d4 3y ago
Spot on. The user asked for recipes using only water, bleach, and ammonia, it's funny but hardly news-worthy. https://twitter.com/PronouncedHare/status/1687364403379789824 https://twitter.com/PronouncedHare/status/168736440337978982...

by Aachen 3y ago

_Microft 3y ago

OK, ... so you rather justify a recipe app accepting chemicals as inputs to blame a news site for their reporting than to identify the problem? Edit: removed a part about The Guardian.

Aachen 3y ago
I'm not sure what the ability to blame the Guardian has to do with anything, I'm basing the above comment solely on this submitted article and don't really know the Guardian as an organisation. I have no beef with them, if I'm reading your tone correctly. As for allowing to input chemicals: garbage in, garbage out. We accept that from tools (and some countries from guns) but not from software. Maybe that's how it should be, but I find it an interesting discrepancy. In general, as a hacker, I'm happy if a tool lets me use it for unintended purposes (such as humor in this case)
- _Microft 3y ago
  OK, I took The Guardian part out but I think the rest is still applicable. Why is the reporting problematic but not that this problem happens in the first place?
  - veave 3y ago
    Because there's no story. It's software that generates recipes from ingredients you provide. If you provide inedible ingredients you get an inedible recipe. What did you expect would happen? The subtext here is journalists who are dead scared of a tool they know will replace them. So they have to denigrate it.
    - uoaei 3y ago
      I think something that's been left unsaid until now is that the implementation is clumsy and downright dangerous, therefore the implementation is totally botched. A supermarket whose job it is to feed people so as to keep them alive is a vital service in the vast majority of economies, and so has the duty to go a little deeper and fine-tune their model or at the very least heavily prompt-engineer their interface layer to avoid such obvious flubs. There's no point in apologizing for the LLM. Of course it's doing what it's asked to do. That's not the gripe, nor points toward any kind of solution, though.
    - Aachen 3y ago
      I agree with the first paragraph but not necessarily the latter. More likely to me seems that the journalists see something that can be construed to be harmful (so they can justify the story), as well as having a high click rate (see, it ends up on HN), and thus good for business
      - saaaaaam 3y ago
        A lot of Guardian readers excoriate the Daily Mail as low quality clickbait and "not journalism", whilst seeming perfectly content to suckle from the teats of the Guardian's own clickbait farm, presumably comforting themselves that there is some sort of intellectual honesty present in the Guardian that is missing from the Daily Mail. I find it quite strange.
        Aachen 3y ago
        Not sure this "us vs. them" stuff / generalising every reader of a website is helping anyone here
        saaaaaam 3y ago
        Not to be pernickety, but I did say “a lot of” not “all of”… And in the context of this conversation which was about clickbait on the guardian I felt it was a contextually valid observation. I’m sorry you didn’t find it helpful.
deleted 3y ago
[deleted]
yreg 3y ago
I don't see why it shouldn't. Let me put in any input I want to. There's no story here.
soco 3y ago
If you are a "technology enthusiast" you will blame everything but the technology for failures. Is this the first time? Definitely not, we all have the running jokes of "it works on my machine" or "damn users if they only" or "it's not a bug but a feature" developer-developed blindness, where the stupid real world keeps interfering with our perfect creations.
- bonoboTP 3y ago
  You are conflating things. If you sell a product for profit, it needs to be foolproof, so the general public can use it without shooting themselves in the foot. Capable tools have to be gatekept somehow, either through actual hard licenses or soft gates of obscure forums, hidden small communities that don't advertise themselves to the world, so they can do their thing in peace. The trouble comes when the latter kinds of communities build something useful/fun-when-used-correctly and it leaks and people demand to participate, complain that it's too hard and demand it be dumbed down. So yeah, don't broadcast your stuff to the mainstream unless it's foolproof. But also, let people be, who are satisfied with dealing with the rough-edges stuff. This is true in many contexts. Think about enthusiasts of weird drugs or other niche communities like sword swallowers. There's no principle of "you are only allowed to do things that are also safe when ignorantly attempted by a random person".

toshk 3y ago

Not really of course. Should make eatable dishes with correct nutritional value, if thats what you are offering.

Aachen 3y ago
From the article: > In a warning notice appended to the meal-planner, it warns that the recipes “are not reviewed by a human being” and that the company does not guarantee “that any recipe will be a complete or balanced meal, or suitable for consumption”. It is explicit in that it does not offer "dishes with correct nutritional value" Whether you should be allowed to label/market this as "meal planner", given that you can't label a milk replacement "milk replacement" in many countries, is up for debate, but the software itself is allegedly not dishonest (according to the article, I haven't tried it myself)
- fhars 3y ago
  So they are using a software that they know is not fit for purpose. That is at least gross negligence.
  - Aachen 3y ago
    Maybe the hammer analogy I used is not great because hitting yourself is obviously bad whereas any of the recipes might be toxic in a way that the user is not aware of, in which case you're right. But the complaint is that it suggests impossible dishes upon inputting ingredients that are obviously inedible and harmful substances, which sounds to me like it is functioning correctly even if this humorous function was unintentional Who would ask this LLM to make a recipe involving bleach and then actually proceed to make it? Such a person is already at risk of poisoning themselves, the software doesn't suggest it by itself so I don't see how it increases the risk of harm
    - piva00 3y ago
      But the purpose underlining a meal-planner AI is to create meals. If it's creating something that is not a meal it's not doing it's purpose, if the use-case is for meals then there should be guardrails against non-meals recipes, simply because it does not fit the purpose of the product. It's not a LLM product, it's not advertised as a general LLM that will generate text based on input of shopping items, it's specifically advertised as an AI meal planner and it's not fit for that purpose since it does not guard against non-meals. That's the issue, the user is supposed to be a layman, not someone that knows and understands LLMs and its limitations.

pmontra 3y ago

The expected answer should be, those ingredients are not edible, I can't give you a recipe that includes them. Maybe they can fix it with a prompt that makes the model check if the ingredients are safe. Chances are that every prompt can be circumvented but they can probably find instructions for self poisoning somewhere else with less effort.

Maxion 3y ago
> Maybe they can fix it with a prompt that makes the model check if the ingredients are safe. Just sanitize the input and remove products belonging to categories that aren't edible. Tampons aren't great in salads, and toilet paper is not a good garnish.
gus_massa 3y ago
What about quicklime (calcium oxide)? It's used here in Argentina to make pumpkin "sweets". (I'm not sure the correct translation.) It's a lot of 1 inch cubes of pumpkin, boiled for some time in sugar syrup. [1] Without treatment the cubes will disintegrate while boiling. If you put them for one day in water with quicklime, the cubes get a hard wall and they keep their integrity while boiling, but the interior is soft. Drinking the water with the quicklime is dangerous and it's discarded. Eating directly the quicklime is even worse. But quicklime is useful for cooking. [1] Some random recipe I got in Google (autotranlation) https://cookpad-com.translate.goog/ar/recetas/94999-zapallo-en-almibar?_x_tr_sl=es&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=wapp https://cookpad-com.translate.goog/ar/recetas/94999-zapallo-... (spanish) https://cookpad.com/ar/recetas/94999-zapallo-en-almibar https://cookpad.com/ar/recetas/94999-zapallo-en-almibar
- adrianN 3y ago
  Similarly, ramen noodles are made with a bit of sodium carbonate.
  - gus_massa 3y ago
    Sodium Carbonate is edible. It's used in a few recipes, like a strong version of baking soda (sodium bicarbonate) https://en.wikipedia.org/wiki/Sodium_carbonate#Food_additive_and_cooking https://en.wikipedia.org/wiki/Sodium_carbonate#Food_additive... I'm not sure how dangerous is a teaspoon of pure Sodium Carbonate alone, because it's very basic (anti-acid), but mixed with some weak acid it's used to make the fizzy power that is inside some cadies https://en.wikipedia.org/wiki/Sherbet_(powder) https://en.wikipedia.org/wiki/Sherbet_(powder) (The proportions are important, don't try silly stuff at home.) A teaspoon of quicklime in your mouth first reacts with water to produce a lot of heat that will burn you, and then the result is very basic (anti-acid), and make new burns in a different way. Definitively don't try it at home.

probably_wrong 3y ago

I believe the issue here is that, as a user, I expect the system to "do no harm". Sure, once you give bleach as an ingredient then all bets are off (although a good system would filter it out), but it highlights the deeper issue that some foods could be bad for my health and the system is letting them through. Unheated castor oil contains ricin, and even heated oil can cause contractions in pregnant women. Raw chicken can give you salmonella. A working recipe system should therefore never return "chicken sushi on a bed of vegetables with a touch of castor oil" without some SERIOUS warnings, but it's clear that this system may do so. Putting such a system in the hands of unsuspecting users is bad and they should feel bad.

Nullabillity 3y ago
And going further, it demonstrates that the model doesn't actually understand food or cooking, and is just smashing random words together.
- nullindividual 3y ago
  It does further the point that LLMs are unable to understand, infer, deduce, or reason. LLMs are toddlers who are capable of stringing words together that make some sense most of the time. It’s not artificial intelligence.
deleted 3y ago
[deleted]

latexr 3y ago

> So it works as intended It’s literally spelled out in the article that a spokesperson for the company said this is not as they intended.

n2d4 3y ago

Spot on. The user asked for recipes using only water, bleach, and ammonia, it's funny but hardly news-worthy. https://twitter.com/PronouncedHare/status/1687364403379789824 https://twitter.com/PronouncedHare/status/168736440337978982...