4 ms·

I get all the fun, fear and anger at those AI apps. But don't we, individual humans, have responsibility as well? There's this urban myth in Europe, that in t

by berkes 3y ago

I get all the fun, fear and anger at those AI apps. But don't we, individual humans, have responsibility as well? There's this urban myth in Europe, that in the US, all microwaves need to come with a warning to not use it to dry you pets or baby's in it. Many of the AI criticism I see, including this here, sounds similar. Anyone with two brain cells should know that one should review outcome with a little bit of common sense? Just like anyone should know that when an 'ai' navigation directs you into the sea, you don't actually drive your car into the sea, not?... Not? https://www.insider.com/tourists-hawaii-gps-drove-car-into-water-2023-5 https://www.insider.com/tourists-hawaii-gps-drove-car-into-w...

ketzo 3y ago
I used to agree with you. But I think we have spent 20+ years teaching people to accept that computers is generally trustworthy, and a lot of people simply do not know how to separate LLMs from computers more broadly yet. Yeah, I know, “don’t believe what you read on the Internet” — but that’s other people on the Internet. People are used to computers themselves being deterministic, reliable.
- Black616Angel 3y ago
  Yeah, but why would I input bleach into my recipe-maker? Why would I not look out the front window of my car while driving? Those are basic skills humans in those positions should be able to do. From the headline I thought the AI mixed some food-ingredients to make chlorine gas, which would be really bad. But if one were to only input "battery acid, cut fingernails, hairball, dead smartwatch" what kind of recipe do you think they'll get?
  - ketzo 3y ago
    Yes, I agree, I feel pretty good saying that no human being has yet followed through with a bleach recipe. I'm just saying that there's a learning curve, like with all new technologies, and that a population-wide sense of "trust this a little, but not too much" will probably take some time to propagate.
    - Philpax 3y ago
      > I feel pretty good saying that no human being has yet followed through with a bleach recipe. I'm afraid I have bad news for you: https://en.wikipedia.org/wiki/Miracle_Mineral_Supplement https://en.wikipedia.org/wiki/Miracle_Mineral_Supplement
- Gigachad 3y ago
  If someone goes to a recipe ai and asks it what you can cook with bleach, and then proceeds to make and consume it, there are other problems than AI.
- marcosdumay 3y ago
  > we have spent 20+ years teaching people to accept that computers is generally trustworthy I really doubt we did. People seem prone to assume the computer is trustworthy since the very beginning (maybe extending it from calculators?) and blatantly refuse to question this assumption. The general assertion of computers' trustworthiness has been causing problems due to bad data or bad algorithms for decades already, there are entire movies about it, and almost everybody has had problem with it already.
Cthulhu_ 3y ago
Well yeah, this article just states that the AI generated chlorine gas recipes, not that people actually went ahead and made it. And people do (to this day) put live creatures in the microwave, it happens, warnings aside.
mdp2021 3y ago
> one should review You seem to be forgetting that sometimes you do not know if a recipe can produce a dangerous result. Realizing that you should not jump off that cliff should be trivial; the outcomes of some chemical reaction may easily not be. That chemistry teacher that won the Darwin Award for beheading herself by dumping random chemicals of her lab (for disposal) under a manhole did not predict the reaction; easily the layman will not know what some combination will produce: this is why you need a trustworthy source.
- yreg 3y ago
  Sounds like in such a case the same can occur with a recipe invented by human.
  - ben_w 3y ago
    Can, yes. I bet that if a supermarket suggested winging it with fugu or moonshine this would also be criticised.
  - mdp2021 3y ago
    > [OP: ] this is why you need a trustworthy source > the same can occur with a recipe invented by human Yes, this is why we do not take advice from random humans on random matters, this is why we want qualified, educated, expert opinions.
    - yreg 3y ago
      We do take advice on cooking from internet randos though. If you don't then I believe you are in a small minority.
      - mdp2021 3y ago
        And the «responsibility» that Berkes was talking about also refers to knowing that the information from «randos» may be wrong, ignorant, deceitful. And in many cases, I observed, that instances are such is not apparent (you are looking for advice after all). But while human condition should be known, the more specialized the tool the more the responsibility is brought to the tool, not to the user. If John says it you may want to check; if the book says it you should still put limited trust, but the fault in bad information lies more in the proclaimed authority. You heard of "Fool me once shame on you; fool me twice shame on me": here, "Fool me by fool shame on me; fool me by foolery shame on you". > minority "Descriptive" and "prescriptive" don't overlap.
Edd314159 3y ago
You're absolutely right, but the article we're talking about misses the point. It focusses on the sensational-sounding part of this, the implication that people will blindly take these recipes and make them and eat them and then die. Clearly that is not going to be the outcome here. But if you peel away the (false) angle that exists just for clicks on the article, there is a real story here. One that we all already know and understand, especially as technologists, but is worth talking about anyway because it's interesting: It's that these "AI" products (the LLMs they're based on) _cannot reason like a human with common sense can_. They're not even close. That doesn't mean they're not amazing technology, but now that the technology has been applied to products, that limitation becomes more obvious. It's kind of a boring aspect of AI to talk about, particularly compared to "AI creates recipes that will kill you!!!". But it's really the root of this story, and it's not nothing.
- chrisandchris 3y ago
  > Clearly that is not going to be the outcome here. Clearly not. E.g. Tell that to the people that drove off-street because they trusted their navigation system more than their eyes. Tell that to the people that drank chlorine against covid. That the recipes are unusable or dangerous is one thing, but believing as a company that noone will misuse your system is something we should know by now that it's totally wrong.
adhesive_wombat 3y ago
Maybe Skynet has just decided to try to whittle humanity down by Darwinian means before it starts on the nuclear weapons, Terminators and time travel.
BaseballPhysics 3y ago
> Anyone with two brain cells should know that one should review outcome with a little bit of common sense? Have you not met people? First, technology companies have spent decades marketing their solutions as magical and perfect. And now it's the public fault they're being believed? Second, even for the skeptic, when a technology works 90% of the time it's easy to start to take for granted that it'll be right the other 10%, and that leads to entirely understandable complacency (a lot of climbers have died this way and they're typically incredibly skilled and diligent). Worse, when machines get it wrong, they can get it wrong in surprising and unpredictable ways. Third, information asymmetry can mean the user cannot verify the outputs. If you, for example, don't know anything about Hawaii (or in that case, maybe you don't know how to read a map--that is a skill, after all), how can you know you're being led into disaster? LLMs have a these problems in spades. They've been marketed as miracles (so much so that some folks have mused about their potential sentience), they're right a lot of the time but very wrong sometimes, and when they are wrong they're wrong in very strange ways, and they can easily used by people who are not sufficient skilled or knowledgeable to verify their outputs. Who would then be surprised by stories like this?
boringg 3y ago
There is also a myth in the US that nothing ever gets accomplished in Europe due to its bureaucratic nature. Or if it does, it takes exceptionally long to accomplish it. Other than that - AI is arguably supposed to be a huge asset for humans however with such poor performance such as this, how are you supposed to trust its other content if it spectacularly fails as such? I don't see this as a self-responsibility issue more an erosion of trust.
berniedurfee 3y ago
I would imagine there are edge cases where toxic foods or at least harmful foods could be created from various food ingredients that are not obvious to the lay person. Blaming the user/customer isn’t a good look from their PR team and probably won’t hold up on court.
andrewaylett 3y ago
My issue with the "product" is that there's no value in it. If you're going to give me an LLM with a straightforward prompt, don't try to spin it as something it's not. There are plenty of applications for LLMs in consumer products, but the output will have no higher information content than the input: if you're tempted to use a trivial skin over an LLM (particularly a publicly-accessible one) to directly transform my input without telling me, please don't. Give me access to the LLM and the prompt and let me be the judge of whether it's worthwhile. "We've prompted this general purpose LLM to act like a cookery expert" sounds a lot less impressive than "We've built a cookery AI".

by berkes 3y ago

ketzo 3y ago

I used to agree with you. But I think we have spent 20+ years teaching people to accept that computers is generally trustworthy, and a lot of people simply do not know how to separate LLMs from computers more broadly yet. Yeah, I know, “don’t believe what you read on the Internet” — but that’s other people on the Internet. People are used to computers themselves being deterministic, reliable.

Black616Angel 3y ago
Yeah, but why would I input bleach into my recipe-maker? Why would I not look out the front window of my car while driving? Those are basic skills humans in those positions should be able to do. From the headline I thought the AI mixed some food-ingredients to make chlorine gas, which would be really bad. But if one were to only input "battery acid, cut fingernails, hairball, dead smartwatch" what kind of recipe do you think they'll get?
- ketzo 3y ago
  Yes, I agree, I feel pretty good saying that no human being has yet followed through with a bleach recipe. I'm just saying that there's a learning curve, like with all new technologies, and that a population-wide sense of "trust this a little, but not too much" will probably take some time to propagate.
  - Philpax 3y ago
    > I feel pretty good saying that no human being has yet followed through with a bleach recipe. I'm afraid I have bad news for you: https://en.wikipedia.org/wiki/Miracle_Mineral_Supplement https://en.wikipedia.org/wiki/Miracle_Mineral_Supplement
Gigachad 3y ago
If someone goes to a recipe ai and asks it what you can cook with bleach, and then proceeds to make and consume it, there are other problems than AI.
marcosdumay 3y ago
> we have spent 20+ years teaching people to accept that computers is generally trustworthy I really doubt we did. People seem prone to assume the computer is trustworthy since the very beginning (maybe extending it from calculators?) and blatantly refuse to question this assumption. The general assertion of computers' trustworthiness has been causing problems due to bad data or bad algorithms for decades already, there are entire movies about it, and almost everybody has had problem with it already.

Cthulhu_ 3y ago

Well yeah, this article just states that the AI generated chlorine gas recipes, not that people actually went ahead and made it. And people do (to this day) put live creatures in the microwave, it happens, warnings aside.

mdp2021 3y ago

> one should review You seem to be forgetting that sometimes you do not know if a recipe can produce a dangerous result. Realizing that you should not jump off that cliff should be trivial; the outcomes of some chemical reaction may easily not be. That chemistry teacher that won the Darwin Award for beheading herself by dumping random chemicals of her lab (for disposal) under a manhole did not predict the reaction; easily the layman will not know what some combination will produce: this is why you need a trustworthy source.

yreg 3y ago
Sounds like in such a case the same can occur with a recipe invented by human.
- ben_w 3y ago
  Can, yes. I bet that if a supermarket suggested winging it with fugu or moonshine this would also be criticised.
- mdp2021 3y ago
  > [OP: ] this is why you need a trustworthy source > the same can occur with a recipe invented by human Yes, this is why we do not take advice from random humans on random matters, this is why we want qualified, educated, expert opinions.
  - yreg 3y ago
    We do take advice on cooking from internet randos though. If you don't then I believe you are in a small minority.
    - mdp2021 3y ago
      And the «responsibility» that Berkes was talking about also refers to knowing that the information from «randos» may be wrong, ignorant, deceitful. And in many cases, I observed, that instances are such is not apparent (you are looking for advice after all). But while human condition should be known, the more specialized the tool the more the responsibility is brought to the tool, not to the user. If John says it you may want to check; if the book says it you should still put limited trust, but the fault in bad information lies more in the proclaimed authority. You heard of "Fool me once shame on you; fool me twice shame on me": here, "Fool me by fool shame on me; fool me by foolery shame on you". > minority "Descriptive" and "prescriptive" don't overlap.

Edd314159 3y ago

You're absolutely right, but the article we're talking about misses the point. It focusses on the sensational-sounding part of this, the implication that people will blindly take these recipes and make them and eat them and then die. Clearly that is not going to be the outcome here. But if you peel away the (false) angle that exists just for clicks on the article, there is a real story here. One that we all already know and understand, especially as technologists, but is worth talking about anyway because it's interesting: It's that these "AI" products (the LLMs they're based on) _cannot reason like a human with common sense can_. They're not even close. That doesn't mean they're not amazing technology, but now that the technology has been applied to products, that limitation becomes more obvious. It's kind of a boring aspect of AI to talk about, particularly compared to "AI creates recipes that will kill you!!!". But it's really the root of this story, and it's not nothing.

chrisandchris 3y ago
> Clearly that is not going to be the outcome here. Clearly not. E.g. Tell that to the people that drove off-street because they trusted their navigation system more than their eyes. Tell that to the people that drank chlorine against covid. That the recipes are unusable or dangerous is one thing, but believing as a company that noone will misuse your system is something we should know by now that it's totally wrong.

adhesive_wombat 3y ago

Maybe Skynet has just decided to try to whittle humanity down by Darwinian means before it starts on the nuclear weapons, Terminators and time travel.

BaseballPhysics 3y ago

> Anyone with two brain cells should know that one should review outcome with a little bit of common sense? Have you not met people? First, technology companies have spent decades marketing their solutions as magical and perfect. And now it's the public fault they're being believed? Second, even for the skeptic, when a technology works 90% of the time it's easy to start to take for granted that it'll be right the other 10%, and that leads to entirely understandable complacency (a lot of climbers have died this way and they're typically incredibly skilled and diligent). Worse, when machines get it wrong, they can get it wrong in surprising and unpredictable ways. Third, information asymmetry can mean the user cannot verify the outputs. If you, for example, don't know anything about Hawaii (or in that case, maybe you don't know how to read a map--that is a skill, after all), how can you know you're being led into disaster? LLMs have a these problems in spades. They've been marketed as miracles (so much so that some folks have mused about their potential sentience), they're right a lot of the time but very wrong sometimes, and when they are wrong they're wrong in very strange ways, and they can easily used by people who are not sufficient skilled or knowledgeable to verify their outputs. Who would then be surprised by stories like this?

boringg 3y ago

There is also a myth in the US that nothing ever gets accomplished in Europe due to its bureaucratic nature. Or if it does, it takes exceptionally long to accomplish it. Other than that - AI is arguably supposed to be a huge asset for humans however with such poor performance such as this, how are you supposed to trust its other content if it spectacularly fails as such? I don't see this as a self-responsibility issue more an erosion of trust.

berniedurfee 3y ago

I would imagine there are edge cases where toxic foods or at least harmful foods could be created from various food ingredients that are not obvious to the lay person. Blaming the user/customer isn’t a good look from their PR team and probably won’t hold up on court.

andrewaylett 3y ago

My issue with the "product" is that there's no value in it. If you're going to give me an LLM with a straightforward prompt, don't try to spin it as something it's not. There are plenty of applications for LLMs in consumer products, but the output will have no higher information content than the input: if you're tempted to use a trivial skin over an LLM (particularly a publicly-accessible one) to directly transform my input without telling me, please don't. Give me access to the LLM and the prompt and let me be the judge of whether it's worthwhile. "We've prompted this general purpose LLM to act like a cookery expert" sounds a lot less impressive than "We've built a cookery AI".