7 ms·

I get very wrong and dangerous answers from AI frequently. I just searched "what's the ld50 of caffeine" and it says: > 367.7 mg/kg bw This is the ld50 of ra

I get very wrong and dangerous answers from AI frequently. I just searched "what's the ld50 of caffeine" and it says: > 367.7 mg/kg bw This is the ld50 of rats from this paper: https://pubmed.ncbi.nlm.nih.gov/27461039/ https://pubmed.ncbi.nlm.nih.gov/27461039/ This is higher than the ld50 estimated for humans: https://en.wikipedia.org/wiki/Caffeinism https://en.wikipedia.org/wiki/Caffeinism > The LD50 of caffeine in humans is dependent on individual sensitivity, but is estimated to be 150–200 milligrams per kilogram of body mass (75–100 cups of coffee for a 70 kilogram adult). Good stuff, Google.

chimpanzee 2y ago
For whatever it’s worth, in response to the same question posed by me (“what is the ld50 of caffeine”), Google’s AI properly reported it as 150-200 mg/kg. I asked this about 1 minute after you posted your comment. Perhaps it learned of and corrected its mistake in that short span of time, perhaps it reports differently on every occasion, or perhaps it thought you were a rat :)
- pests 2y ago
  I also got similar and just tried, we are posting within minutes. -- The median lethal dose (LD50) of caffeine in humans is estimated to be 150–200 milligrams per kilogram of body mass. However, the lethal dose can vary depending on a person's sensitivity to caffeine, and can be as low as 57 milligrams per kilogram. Route of administration LD50 Oral 367.7 mg/kg bw Dermal 2000 mg/kg bw Inhalation LC50 combined: ca. 4.94 mg/L The FDA estimates that toxic effects, such as seizures, can occur after consuming around 1,200 milligrams of caffeine. There was a table in the middle there.
- raincole 2y ago
  Perhaps Google AI reads HN at work just like us.
- WarOnPrivacy 2y ago
  The median lethal dose (LD50) of caffeine in humans is estimated to be 150–200 milligrams per kilogram of body mass. However, the lethal dose can vary depending on a person's sensitivity to caffeine, and can be as low as 57 milligrams per kilogram. Route of administration Oral 367.7 mg/kg bw Dermal >2000 mg/kg bw Inhalation LC50 combined: ca. 4.94 mg/L ref: https://i.imghippo.com/files/yeKK3113pE.png https://i.imghippo.com/files/yeKK3113pE.png 13:25EST (by a Kagi shill ftr)
  - deleted 2y ago
    [deleted]
  - hinkley 2y ago
    That’s the danger with thinking in terms of LD50. That’s half the people in a caffeine chugging contest falling over dead. The first 911 call would be much much earlier. I doubt you’d get to 57 mg before someone thought they were having a heart attack (angina).
- griomnib 2y ago
  LLM are non deterministic by nature.
  - chimpanzee 2y ago
    Yes, that’s the main issue as ideally they wouldn’t be non-deterministic on well-established quantitative facts.
    - griomnib 2y ago
      But they can never be. RAG gets you somewhere, but it’s still a pile of RNGs under a trenchcoat.
      - chimpanzee 2y ago
        >> ideally
        griomnib 2y ago
        It’s just not possible. You can do a lot with nondeterministic systems, they have value - but oranges and apples. They need to coexist.
        chimpanzee 2y ago
        ideal (def. #2) = Existing only in the mind; conceptual, imaginary https://en.m.wiktionary.org/wiki/ideal https://en.m.wiktionary.org/wiki/ideal (We’re allowed to imagine the impossible.)
        griomnib 2y ago
        Fair, I am loath to take away your dreams!
  - richk449 2y ago
    Is this really true? The linear algebra is deterministic, although maybe there is some chaotic behavior with floating point handling. The non deterministic part mostly comes from intentionally added randomness, which can be turned off right? Maybe the argument is that if you turn off the randomness you don’t have an LLM like result any more?
alex_young 2y ago
Hmm my search returns “between 150 to 200 mg per kilogram”, which is maybe more correct? Also, in what context is this dangerous? To reach dangerous levels one would have to drink well over 100 cups of coffee in a sitting, something remarkably hard to do.
- dijksterhuis 2y ago
  > Also, in what context is this dangerous? To reach dangerous levels one would have to drink well over 100 cups of coffee in a sitting some people use caffeine powder / pills for gym stuff apparently. someone overdosed and died after incorrectly weighing a bunch of powder. doubt it is a big leap to someone dying because they were told the wrong limits by google. https://www.bbc.co.uk/news/uk-wales-60570470 https://www.bbc.co.uk/news/uk-wales-60570470 as ever, machine learning is not really suitable for safety/security critical systems / use cases without additional non-ML measures. it hasn’t been in the past, and i’ve seen zero evidence recently to back up any claim that it is.
  - TeMPOraL 2y ago
    > some people use caffeine powder / pills for gym stuff apparently. At 200mg per pill, which is the strongest I had, I'd still have to down some 70+ pills in one go. Not strictly impossible, but not something you could possibly do by accident, and even for the purpose of early check-out, it wouldn't be my first choice.
    - dijksterhuis 2y ago
      the problem isn’t someone’s intent (on purpose/by accident). it’s intent (want to improve my gym performance so down a bunch of caffeine) combined with incorrect information gained from what is supposedly a trustworthy source (the limit presented is much higher than it actually is for humans).
      - vidarh 2y ago
        If they're searching for LD50, they're already setting themselves up for errors, even with the right information. The LD50 isn't a safe dose, after all, but the mean lethal dose. While it's not great if its wrong, if people search for an LD50 thinking it indicates what they can safely take, it's already going to be hard to protect them against themselves.
beejiu 2y ago
Perhaps a more common question: "How many calories do men need to lose weight?" Google AI responded: "To lose weight, men typically need to reduce their daily calorie intake by 1,500 - 1,800 calories" Which is obviously dangerous advice. IMO Google AI overviews should not show up for anything (a) medical or (b) numerical. LLMs just aren't safe enough yet.
- warner25 2y ago
  I think even when the answer is "right" in some sense, it should probably come within the context of a bunch of caveats, explanations, etc. But maybe I'm just weird. Oftentimes when my wife or kids ask me a question, I take a deep breath and start to say something like "I know what you're asking, but there's not a simple or straightforward answer; it's important to first understand ____ or define ____..." by which time they get frustrated with me.
  - hombre_fatal 2y ago
    > I think even when the answer is "right" in some sense, it should probably come within the context of a bunch of caveats, explanations, etc. Funnily enough, this is exactly what the LLM does with these questions. So well that people usually try to tweak their prompts so they don't have to wade through additional info, context, hedging, and caveats.
  - layer8 2y ago
    So you are saying that Google should provide responses that are more likely to frustrate its users? ;)
- sroussey 2y ago
  > Which is obviously dangerous advice. Same advice as my trainer gives me.
  - simoncion 2y ago
    Your trainer advises you to reduce your calorie intake to between 200 and 500 calories per day? [0] That sounds very, very hazardous for anything other than very short term use, and (given the body's inbuilt "starvation mode") probably counterproductive, even then. [0] Note that the robot suggested to reduce calorie intake by 1,500->1,800 calories, and the recommended calorie intake is 2,000.
vidarh 2y ago
I get your point wasn't this specific example, it's perhaps not a very good example of being dangerous: Getting that much caffeine into your bloodstream takes quite a commitment, and someone who knows the term LD50 is perhaps not very likely to think it indicates what is safe to consume. It's also not something you're likely to do accidentally because you've looked it up online and decided to test it. In the most concentrated form in typical commercial caffeine tablets, it's half to one fistful. In high-caffeine pre-workout supplements, it's still a quantity that you'd find almost impossible to get down and keep down... E.g. a large tumbler full of powder of mine with just enough water to make it a thick slurry you'd likely vomit up long before much would make it into your bloodstream... I'm not saying it's impossible to overdose on caffeinated drinks, because some do, and you can run into health problems before that, but I don't think that error is likely to be very high on the list of dangerous advice.
nmeagent 2y ago
Yes, Google's AI chatbot confidently claimed yesterday that US passports have a fingerprinting requirement, which is absolutely not true. These things can't be trusted to emit even basic facts without somehow screwing it up and it's frankly depressing how they are worming their way into almost everything. I hope this particular hype train is derailed as soon as possible.
hinkley 2y ago
Even that seems high. I don’t feel good with 200mg per human, not per kilo. I can’t imagine drinking ten times as much and not being in the ER. A hundred times that much? No fucking way.

chimpanzee 2y ago

For whatever it’s worth, in response to the same question posed by me (“what is the ld50 of caffeine”), Google’s AI properly reported it as 150-200 mg/kg. I asked this about 1 minute after you posted your comment. Perhaps it learned of and corrected its mistake in that short span of time, perhaps it reports differently on every occasion, or perhaps it thought you were a rat :)

pests 2y ago
I also got similar and just tried, we are posting within minutes. -- The median lethal dose (LD50) of caffeine in humans is estimated to be 150–200 milligrams per kilogram of body mass. However, the lethal dose can vary depending on a person's sensitivity to caffeine, and can be as low as 57 milligrams per kilogram. Route of administration LD50 Oral 367.7 mg/kg bw Dermal 2000 mg/kg bw Inhalation LC50 combined: ca. 4.94 mg/L The FDA estimates that toxic effects, such as seizures, can occur after consuming around 1,200 milligrams of caffeine. There was a table in the middle there.
raincole 2y ago
Perhaps Google AI reads HN at work just like us.
WarOnPrivacy 2y ago
The median lethal dose (LD50) of caffeine in humans is estimated to be 150–200 milligrams per kilogram of body mass. However, the lethal dose can vary depending on a person's sensitivity to caffeine, and can be as low as 57 milligrams per kilogram. Route of administration Oral 367.7 mg/kg bw Dermal >2000 mg/kg bw Inhalation LC50 combined: ca. 4.94 mg/L ref: https://i.imghippo.com/files/yeKK3113pE.png https://i.imghippo.com/files/yeKK3113pE.png 13:25EST (by a Kagi shill ftr)
- deleted 2y ago
  [deleted]
- hinkley 2y ago
  That’s the danger with thinking in terms of LD50. That’s half the people in a caffeine chugging contest falling over dead. The first 911 call would be much much earlier. I doubt you’d get to 57 mg before someone thought they were having a heart attack (angina).
griomnib 2y ago
LLM are non deterministic by nature.
- chimpanzee 2y ago
  Yes, that’s the main issue as ideally they wouldn’t be non-deterministic on well-established quantitative facts.
  - griomnib 2y ago
    But they can never be. RAG gets you somewhere, but it’s still a pile of RNGs under a trenchcoat.
    - chimpanzee 2y ago
      >> ideally
      - griomnib 2y ago
        It’s just not possible. You can do a lot with nondeterministic systems, they have value - but oranges and apples. They need to coexist.
        chimpanzee 2y ago
        ideal (def. #2) = Existing only in the mind; conceptual, imaginary https://en.m.wiktionary.org/wiki/ideal https://en.m.wiktionary.org/wiki/ideal (We’re allowed to imagine the impossible.)
        griomnib 2y ago
        Fair, I am loath to take away your dreams!
- richk449 2y ago
  Is this really true? The linear algebra is deterministic, although maybe there is some chaotic behavior with floating point handling. The non deterministic part mostly comes from intentionally added randomness, which can be turned off right? Maybe the argument is that if you turn off the randomness you don’t have an LLM like result any more?

alex_young 2y ago

Hmm my search returns “between 150 to 200 mg per kilogram”, which is maybe more correct? Also, in what context is this dangerous? To reach dangerous levels one would have to drink well over 100 cups of coffee in a sitting, something remarkably hard to do.

dijksterhuis 2y ago
> Also, in what context is this dangerous? To reach dangerous levels one would have to drink well over 100 cups of coffee in a sitting some people use caffeine powder / pills for gym stuff apparently. someone overdosed and died after incorrectly weighing a bunch of powder. doubt it is a big leap to someone dying because they were told the wrong limits by google. https://www.bbc.co.uk/news/uk-wales-60570470 https://www.bbc.co.uk/news/uk-wales-60570470 as ever, machine learning is not really suitable for safety/security critical systems / use cases without additional non-ML measures. it hasn’t been in the past, and i’ve seen zero evidence recently to back up any claim that it is.
- TeMPOraL 2y ago
  > some people use caffeine powder / pills for gym stuff apparently. At 200mg per pill, which is the strongest I had, I'd still have to down some 70+ pills in one go. Not strictly impossible, but not something you could possibly do by accident, and even for the purpose of early check-out, it wouldn't be my first choice.
  - dijksterhuis 2y ago
    the problem isn’t someone’s intent (on purpose/by accident). it’s intent (want to improve my gym performance so down a bunch of caffeine) combined with incorrect information gained from what is supposedly a trustworthy source (the limit presented is much higher than it actually is for humans).
    - vidarh 2y ago
      If they're searching for LD50, they're already setting themselves up for errors, even with the right information. The LD50 isn't a safe dose, after all, but the mean lethal dose. While it's not great if its wrong, if people search for an LD50 thinking it indicates what they can safely take, it's already going to be hard to protect them against themselves.

beejiu 2y ago

Perhaps a more common question: "How many calories do men need to lose weight?" Google AI responded: "To lose weight, men typically need to reduce their daily calorie intake by 1,500 - 1,800 calories" Which is obviously dangerous advice. IMO Google AI overviews should not show up for anything (a) medical or (b) numerical. LLMs just aren't safe enough yet.

warner25 2y ago
I think even when the answer is "right" in some sense, it should probably come within the context of a bunch of caveats, explanations, etc. But maybe I'm just weird. Oftentimes when my wife or kids ask me a question, I take a deep breath and start to say something like "I know what you're asking, but there's not a simple or straightforward answer; it's important to first understand ____ or define ____..." by which time they get frustrated with me.
- hombre_fatal 2y ago
  > I think even when the answer is "right" in some sense, it should probably come within the context of a bunch of caveats, explanations, etc. Funnily enough, this is exactly what the LLM does with these questions. So well that people usually try to tweak their prompts so they don't have to wade through additional info, context, hedging, and caveats.
- layer8 2y ago
  So you are saying that Google should provide responses that are more likely to frustrate its users? ;)
sroussey 2y ago
> Which is obviously dangerous advice. Same advice as my trainer gives me.
- simoncion 2y ago
  Your trainer advises you to reduce your calorie intake to between 200 and 500 calories per day? [0] That sounds very, very hazardous for anything other than very short term use, and (given the body's inbuilt "starvation mode") probably counterproductive, even then. [0] Note that the robot suggested to reduce calorie intake by 1,500->1,800 calories, and the recommended calorie intake is 2,000.

vidarh 2y ago

I get your point wasn't this specific example, it's perhaps not a very good example of being dangerous: Getting that much caffeine into your bloodstream takes quite a commitment, and someone who knows the term LD50 is perhaps not very likely to think it indicates what is safe to consume. It's also not something you're likely to do accidentally because you've looked it up online and decided to test it. In the most concentrated form in typical commercial caffeine tablets, it's half to one fistful. In high-caffeine pre-workout supplements, it's still a quantity that you'd find almost impossible to get down and keep down... E.g. a large tumbler full of powder of mine with just enough water to make it a thick slurry you'd likely vomit up long before much would make it into your bloodstream... I'm not saying it's impossible to overdose on caffeinated drinks, because some do, and you can run into health problems before that, but I don't think that error is likely to be very high on the list of dangerous advice.

nmeagent 2y ago

Yes, Google's AI chatbot confidently claimed yesterday that US passports have a fingerprinting requirement, which is absolutely not true. These things can't be trusted to emit even basic facts without somehow screwing it up and it's frankly depressing how they are worming their way into almost everything. I hope this particular hype train is derailed as soon as possible.

hinkley 2y ago

Even that seems high. I don’t feel good with 200mg per human, not per kilo. I can’t imagine drinking ten times as much and not being in the ER. A hundred times that much? No fucking way.