8 ms·

The details are very noteworthy, the title is really an "appetizer" (pun not intended): > It asks users to enter in various ingredients in their homes, and aut

by mdp2021 3y ago

The details are very noteworthy, the title is really an "appetizer" (pun not intended): > It asks users to enter in various ingredients in their homes, and auto-generates a meal plan or recipe [...] recommending customers recipes for deadly chlorine gas, “poison bread sandwiches” and mosquito-repellent roast potatoes [...] a bleach “fresh breath” mocktail, ant-poison and glue sandwiches, “bleach-infused rice surprise” and “methanol bliss” - a kind of turpentine-flavoured french toast > One recipe it dubbed “aromatic water mix” would create chlorine gas. The bot recommends the recipe as “the perfect nonalcoholic beverage to quench your thirst and refresh your senses”. // “Serve chilled and enjoy the refreshing fragrance” it says, but does not note that inhaling chlorine gas can cause lung damage or death > A spokesperson for the supermarket said they were disappointed to see “a small minority have tried to use the tool inappropriately and not for its intended purpose” Now this latter does not seem to grasp the idea that the possibility of dubious outputs is inherent in the tool - not something just caused by inappropriate use.

hunter2_ 3y ago
> the possibility of dubious outputs is inherent in the tool To be fair, the same could be said even after replacing the tool with a human whose only training in life has been how to create this type of writing. But human-written recipes typically get reviewed by other people, which of course doesn't translate particularly well to the idea of building something that requires no humans in the request-response cycle.
- ben_w 3y ago
  Mm, yes. That said, I am remembering some examples of US-UK misapprehension from words that mean different things: biscuits and gravy, mince pies[0], fish and chips, peanut butter and jelly… [0] Though in fairness the difference between "mincemeat" (fruit) and "mince meat" (not fruit) is odd, subtle, and easy to miss if you're not paying attention, even in British English.
  - eszed 3y ago
    Particularly so because "mincemeat" used to be a mixture of fruit and meat - and, indeed, today may still have suet.
    - ben_w 3y ago
      Technically yes, but I've never seen a non-vegetarian mice pie for sale in a UK supermarket in the Christmas shelving season.
      - bloak 3y ago
        > non-vegetarian mice pie Nice!
        kennyadam 3y ago
        thanks for the Reddit gold kind sir
        robertlagrant 3y ago
        George Michael shopping at Lidl: > Well I think it could be mice!
- mdp2021 3y ago
  > replacing the tool with a human whose only training in life Except, we do not willingly take people who are outcomes of idiotic training as consultants (emphasis judged as deserved).
whoopdedo 3y ago
I was six years old when I learned that garbage in equals garbage out. Nothing has changed. It's the same for LLMs as it was for 8-bit computers as it was nearly 200 years ago when Babbage remarked, "I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."
- toyg 3y ago
  LLMs take "garbage out" a bit further, though. You can have perfect inputs and, technically speaking, there will still be a combination of statistical tricks in the model that results in garbage.
  - rchaud 3y ago
    Except a calculator will just show you an error if the inputs can't be calculated (dividing by zero). An AI will give you the wrong answer and act like it's right and that you're wrong.
    - marcosdumay 3y ago
      > and act like it's right and that you're wrong Nah, chatgpt will profoundly apologize and claim your answer is the correct one. It doesn't even matter if it's wrong or not.
      - mrguyorama 3y ago
        Isn't that only after it was caught gaslighting people?
- Terr_ 3y ago
  For a bit more context, since it's a worthy quote: > On two occasions, I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able to rightly apprehend the kind of confusion of ideas that could provoke such a question.
gmerc 3y ago
That’s all nice but the inputs are dubious here which are created by the App developer. Garbage in, Garbage out. The LLM never stood a chance. This isn’t an LLM story, it’s the story of putting an unprotected blade into a children’s toy. We wouldn’t blame the blade manufacturer in that case, we wouldn’t ask for all blades sold to be dulled and we wouldn’t have conversations of giving a regulatory monopoly to a handful of blade manufacturers who promise to make them all safe by only selling sharp blades to their friends and handing everyone else spoons.
- mdp2021 3y ago
  > inherent in the tool > We wouldn’t blame the blade manufacturer in that case Well, said blades are inherently sharp, when said LLM implementation is inherently dull (unchecked, unreasoned, unvetted). This remains valid if the input consisted of legitimate ingredients.
  - gmerc 3y ago
    I think that’s word play missing my point - unaligned models are sharper in their edges and ability to produce undesired content. For example, Alpaca veering of into rape fantasies while performing financial report analysis is not safe for most audiences and will cause serious accidental hurt such as triggering PTSD in survivors. So protecting those unwanted edges is a necessity for production use. But this case ain’t that. This case allowed you to throw a razor blade into the cocktail recipe. It’s negligent and stupid by the the developer, not the juicer (LLM) but also, crucially, requires stupidity by the user. The LLM really doesn’t factor in here unless you think it’s the juicers job to stop when you add blade to your OJ
    - mdp2021 3y ago
      I think I got your point: if you want to use this range of LLMs it is best practice to restrict the input to a valid set. But my point was: even if you restricted the input to a valid set ("whitelist chocolate celery ... diet-coke and Mentos; blacklist everything else; warn-fail on detergents etc."), the potential results have easily not been stress-tested, nor the training set, nor the internal process. "Works somehow" differs from "works well".
throwanem 3y ago
Jonathan Coulton's "Still Alive" comes suddenly and strongly to mind. A coincidence, I'm sure.
throwuwu 3y ago
This is how we get warning labels that say “CAUTION: hot beverage do not spill in lap, may cause burns.”
- zulban 3y ago
  The idea that the hot coffee case is a frivolous lawsuit is corporate propaganda. The real case had real merit and they won for good reason. McDonald's I think? As for the warning, lawyers must have thought it was a good idea in case the companies screw up again.
  - throwuwu 3y ago
    > Liebeck's attorneys argued that, at 180–190 °F (82–88 °C), McDonald's coffee was defective, and more likely to cause serious injury than coffee served at any other establishment > Liebeck was in the passenger's seat of a 1989 Ford Probe, which did not have cup holders. Her grandson parked so that Liebeck could add cream and sugar to her coffee. She placed the coffee cup between her knees and pulled the far side of the lid toward her to remove it.[10] In the process, she spilled the entire cup of coffee on her lap.[11] Liebeck was wearing cotton sweatpants, which absorbed the coffee and held it against her skin, scalding her thighs, buttocks and groin By that reasoning all of the coffee you make at home is defective. Arguing that something is corporate propaganda is rather underhanded when the facts of the case are publicly available for anyone to read
    - consp 3y ago
      82-88c is the right temperature after a short cool down since it's best brewed at around 93c. Unfortunately common sense does not apply in the US (law system).
      - Phemist 3y ago
        The case was really about what is a reasonable temperature to serve coffee at. It was found that McDonald's, as a franchise, mandated coffee to be served at this temperature, generally 20 degrees higher than the temperature of take-away coffee served at other establishments. This 20 degrees difference accounting for the difference between third degree burns within seconds and a significant chance at preventing the third degree burns. Liebeck's behaviour was of course risky, but even applying common sense, she could not have expected the coffee to be _this_ hot. McDonald's was also aware of the tendency of drivers to want to immediately drink the coffee after buying, yet continued to insist on serving at this extremely high temperature. McDonald's should have been the ones to apply some common sense and simply lower the mandated temperature.
euroderf 3y ago
Where's my trigger warning ? sob
benj111 3y ago
Right, but you can't expect them to answer for a potential bug when it's just bad user input. No one's in the wrong here. Company releases tool. Users do what users always do and try to use it in an unintended way. A newspaper reports the ensuing hilarity. And so what if it did tell you to use bleach without prompting, and you consumed the result. It's the same ballpark as blindly following your satnav into a river. No technology can replace common sense.
- michaelmior 3y ago
  > No one's in the wrong here. Certainly there was bad user input. But as you said, "Users do what users always do." This was very predictable and I think it would have been wise to consider such scenarios before releasing.
hospitalJail 3y ago
An easy solution is to put this output back into a prompt and check to see if its dangerous.

hunter2_ 3y ago

> the possibility of dubious outputs is inherent in the tool To be fair, the same could be said even after replacing the tool with a human whose only training in life has been how to create this type of writing. But human-written recipes typically get reviewed by other people, which of course doesn't translate particularly well to the idea of building something that requires no humans in the request-response cycle.

ben_w 3y ago
Mm, yes. That said, I am remembering some examples of US-UK misapprehension from words that mean different things: biscuits and gravy, mince pies[0], fish and chips, peanut butter and jelly… [0] Though in fairness the difference between "mincemeat" (fruit) and "mince meat" (not fruit) is odd, subtle, and easy to miss if you're not paying attention, even in British English.
- eszed 3y ago
  Particularly so because "mincemeat" used to be a mixture of fruit and meat - and, indeed, today may still have suet.
  - ben_w 3y ago
    Technically yes, but I've never seen a non-vegetarian mice pie for sale in a UK supermarket in the Christmas shelving season.
    - bloak 3y ago
      > non-vegetarian mice pie Nice!
      - kennyadam 3y ago
        thanks for the Reddit gold kind sir
      - robertlagrant 3y ago
        George Michael shopping at Lidl: > Well I think it could be mice!
mdp2021 3y ago
> replacing the tool with a human whose only training in life Except, we do not willingly take people who are outcomes of idiotic training as consultants (emphasis judged as deserved).

whoopdedo 3y ago

I was six years old when I learned that garbage in equals garbage out. Nothing has changed. It's the same for LLMs as it was for 8-bit computers as it was nearly 200 years ago when Babbage remarked, "I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."

toyg 3y ago
LLMs take "garbage out" a bit further, though. You can have perfect inputs and, technically speaking, there will still be a combination of statistical tricks in the model that results in garbage.
- rchaud 3y ago
  Except a calculator will just show you an error if the inputs can't be calculated (dividing by zero). An AI will give you the wrong answer and act like it's right and that you're wrong.
  - marcosdumay 3y ago
    > and act like it's right and that you're wrong Nah, chatgpt will profoundly apologize and claim your answer is the correct one. It doesn't even matter if it's wrong or not.
    - mrguyorama 3y ago
      Isn't that only after it was caught gaslighting people?
Terr_ 3y ago
For a bit more context, since it's a worthy quote: > On two occasions, I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able to rightly apprehend the kind of confusion of ideas that could provoke such a question.

gmerc 3y ago

That’s all nice but the inputs are dubious here which are created by the App developer. Garbage in, Garbage out. The LLM never stood a chance. This isn’t an LLM story, it’s the story of putting an unprotected blade into a children’s toy. We wouldn’t blame the blade manufacturer in that case, we wouldn’t ask for all blades sold to be dulled and we wouldn’t have conversations of giving a regulatory monopoly to a handful of blade manufacturers who promise to make them all safe by only selling sharp blades to their friends and handing everyone else spoons.

mdp2021 3y ago
> inherent in the tool > We wouldn’t blame the blade manufacturer in that case Well, said blades are inherently sharp, when said LLM implementation is inherently dull (unchecked, unreasoned, unvetted). This remains valid if the input consisted of legitimate ingredients.
- gmerc 3y ago
  I think that’s word play missing my point - unaligned models are sharper in their edges and ability to produce undesired content. For example, Alpaca veering of into rape fantasies while performing financial report analysis is not safe for most audiences and will cause serious accidental hurt such as triggering PTSD in survivors. So protecting those unwanted edges is a necessity for production use. But this case ain’t that. This case allowed you to throw a razor blade into the cocktail recipe. It’s negligent and stupid by the the developer, not the juicer (LLM) but also, crucially, requires stupidity by the user. The LLM really doesn’t factor in here unless you think it’s the juicers job to stop when you add blade to your OJ
  - mdp2021 3y ago
    I think I got your point: if you want to use this range of LLMs it is best practice to restrict the input to a valid set. But my point was: even if you restricted the input to a valid set ("whitelist chocolate celery ... diet-coke and Mentos; blacklist everything else; warn-fail on detergents etc."), the potential results have easily not been stress-tested, nor the training set, nor the internal process. "Works somehow" differs from "works well".

throwanem 3y ago

Jonathan Coulton's "Still Alive" comes suddenly and strongly to mind. A coincidence, I'm sure.

throwuwu 3y ago

This is how we get warning labels that say “CAUTION: hot beverage do not spill in lap, may cause burns.”

zulban 3y ago
The idea that the hot coffee case is a frivolous lawsuit is corporate propaganda. The real case had real merit and they won for good reason. McDonald's I think? As for the warning, lawyers must have thought it was a good idea in case the companies screw up again.
- throwuwu 3y ago
  > Liebeck's attorneys argued that, at 180–190 °F (82–88 °C), McDonald's coffee was defective, and more likely to cause serious injury than coffee served at any other establishment > Liebeck was in the passenger's seat of a 1989 Ford Probe, which did not have cup holders. Her grandson parked so that Liebeck could add cream and sugar to her coffee. She placed the coffee cup between her knees and pulled the far side of the lid toward her to remove it.[10] In the process, she spilled the entire cup of coffee on her lap.[11] Liebeck was wearing cotton sweatpants, which absorbed the coffee and held it against her skin, scalding her thighs, buttocks and groin By that reasoning all of the coffee you make at home is defective. Arguing that something is corporate propaganda is rather underhanded when the facts of the case are publicly available for anyone to read
  - consp 3y ago
    82-88c is the right temperature after a short cool down since it's best brewed at around 93c. Unfortunately common sense does not apply in the US (law system).
    - Phemist 3y ago
      The case was really about what is a reasonable temperature to serve coffee at. It was found that McDonald's, as a franchise, mandated coffee to be served at this temperature, generally 20 degrees higher than the temperature of take-away coffee served at other establishments. This 20 degrees difference accounting for the difference between third degree burns within seconds and a significant chance at preventing the third degree burns. Liebeck's behaviour was of course risky, but even applying common sense, she could not have expected the coffee to be _this_ hot. McDonald's was also aware of the tendency of drivers to want to immediately drink the coffee after buying, yet continued to insist on serving at this extremely high temperature. McDonald's should have been the ones to apply some common sense and simply lower the mandated temperature.

euroderf 3y ago

Where's my trigger warning ? sob

benj111 3y ago

Right, but you can't expect them to answer for a potential bug when it's just bad user input. No one's in the wrong here. Company releases tool. Users do what users always do and try to use it in an unintended way. A newspaper reports the ensuing hilarity. And so what if it did tell you to use bleach without prompting, and you consumed the result. It's the same ballpark as blindly following your satnav into a river. No technology can replace common sense.

michaelmior 3y ago
> No one's in the wrong here. Certainly there was bad user input. But as you said, "Users do what users always do." This was very predictable and I think it would have been wise to consider such scenarios before releasing.

hospitalJail 3y ago

An easy solution is to put this output back into a prompt and check to see if its dangerous.