6 ms·
Man develops rare condition after ChatGPT query over stopping eating salt
- some_random 1y agoIs it just me or is the title kinda unclear? >The patient told doctors that after reading about the negative effects of sodium chloride, or table salt, he consulted ChatGPT about eliminating chloride from his diet and started taking sodium bromide over a three-month period. This was despite reading that “chloride can be swapped with bromide, though likely for other purposes, such as cleaning”. Sodium bromide was used as a sedative in the early 20th century. In any case, I feel like I really need to see the actual conversation itself to judge how badly chatgpt messed up, if there's no extra context assuming that the user is talking about cleaning doesn't seem _that_ unreasonable.
- Flozzin 1y agoThe article digs a little deeper after. Saying the chat records are lost, and that when they asked ChatGPT, it didn't give that guidance about cleaning purposely only, and that it never asked why they wanted to know. Really though, this could have just as easily happened in a google search. It's not ChatGPT's fault as much as this persons fault for using a non-medical professional for medical guidance.
- zahlman 1y ago>and that it never asked why they wanted to know. Does ChatGPT ever ask the user, like, anything?
- fl7305 1y ago> Does ChatGPT ever ask the user, like, anything? Yes. At least when I just tried ChatGPT-5: Can I replace sodium chloride with sodium bromide? ChatGPT said: Yes, in some cases — but it depends on the application. Chemistry/lab use: Both are salts of sodium and dissolve similarly, but bromide is more reactive in some contexts and heavier. It can change reaction outcomes, especially in halide-sensitive reactions (e.g., silver halide precipitation). Food use: No — sodium bromide is toxic and not approved as a food additive. Industrial processes: Sometimes interchangeable (e.g., certain brines, drilling fluids) if bromide’s different density, solubility, and cost are acceptable. What’s your intended use?
- OJFord 1y agoYeah I thought it was a bit misleading too, it's not exactly 'stopping salt' that caused it, any more than you could describe the ill-effects of swapping nasturtiums for lily of the valley in your salads as 'avoiding edible flowers'.
- kragen 1y agoHe was poisoning himself for three months before he was treated, and apparently made a full recovery: > He was tapered off risperidone before discharge and remained stable off medication at a check-in 2 weeks after discharge. https://www.acpjournals.org/doi/epdf/10.7326/aimcc.2024.1260 https://www.acpjournals.org/doi/epdf/10.7326/aimcc.2024.1260 If you eliminated sodium chloride from your diet without replacing it with another sodium source, you would die in much less than three months; I think you'd be lucky to make it two weeks. You can't replace sodium with potassium or lithium or ammonium to the same degree that you can replace chloride with bromide.
- OJFord 1y agoEven if you managed to reduce your intake enough to cause hyponatremia, I'm not sure that fits the 'rare condition' bill, and probably would've been discharged in well under 2 weeks after some fluids and advice. Would be interesting if he started to become symptomatic and so asked ChatGPT and that's where he got the idea that it needed to be replaced with something though. (But I suspect it was more along the lines of salty taste without the NaCl intake.)
- topaz0 1y agoI'd say that the thing that messed up was the AI hype machine for pretending it might ever be a good idea to take chatgpt output as advice.
- emorning3 1y ago[dead]
- zahlman 1y agoWow. I thought this was just going to be about hyponatremia or something. (And from other research I've done, I really do think that on balance the US experts are recommending inappropriately low levels of sodium intake that are only appropriate for people who already have hypertension, and that typical North American dietary levels of sodium are just fine, really.) But replacing table salt with sodium bromide? Oof. > to judge how badly chatgpt messed up, if there's no extra context assuming that the user is talking about cleaning doesn't seem _that_ unreasonable. This would be a bizarre assumption for the simple reason that table salt is not normally used in cleaning.
- bell-cot 1y agoSame news, Ars Technica, 5 comments, 5 days ago: https://news.ycombinator.com/item?id=44829824 https://news.ycombinator.com/item?id=44829824
- HelloUsername 1y agoAlso today https://news.ycombinator.com/item?id=44887459 https://news.ycombinator.com/item?id=44887459
- slicktux 1y agoDoes that title seem like a cluster to anyone else? I tried rewording it in my head but only came up with a slightly better solution: “Man develops rare condition after cutting consumption of salt do to a ChatGPT query”.
- jleyank 1y agoHow about “man gets bromine poisoning after taking ChatGPT medical advice”?
- brokencode 1y agoWe don’t know whether ChatGPT gave medical advice. Only that it suggested using sodium bromide instead of sodium chloride. For what purpose or in what context, we don’t know. It may even have recommended against using it and the man misunderstood.
- topaz0 1y agoChatgpt doesn't give advice at all, but could say "after interpreting chatgpt output as medical advice"
- neom 1y agoIt's clunky but I understood it immediately, I presumed from the title that it was going to be about your title, however I also thought it was a bit clunky.
- genter 1y agoI read it as "race condition". I was then trying to figure out what salt has to do with a race condition.
- dfee 1y agostill haven't clicked in, but was confused. still am, unless i re-interpret "do" as "due".
- MarkusQ 1y agoLLMs don't think. At all. They do next token prediction. If they are conditioned on a large data set that includes lots of examples of the result of people thinking, what they produce will look sort of like the results of thinking, but then if they were conditioned on a large data set of people repeating the same seven knock knock jokes over and over and over in some complex pattern (e.g. every third time, in French), what they produced will look like that, and nothing like thinking. Failing to recognize this is going to get someone killed, if it hasn't already.
- nimbius 1y agoyeah sure, but, did it enrich the shareholders?
- hcdx6 1y agoAre you thinking over every character you type? You are conditioned too by all the info flowing into your head from birth. Does that gauruntee everything your brain says and does is perfect? People believed in non existent WMDs and tens of thousands got killed. After that what happened ? Chimps with 3 inch brains feel super confident to run orgs and make decisions that effect entire populations and are never held accountable. Ask Snowden what happened after he recognized that.
- uh_uh 1y ago> LLMs don't think. At all. How can you so confidently proclaim that? Hinton and Ilya Sutskever certainly seem to think that LLMs do think. I'm not saying that you should accept what they say blindly due to their authority in the field, but their opinions should give your confidence some pause at least.
- dgfitz 1y ago>> LLMs don't think. At all. >How can you so confidently proclaim that? Do you know why they're called 'models' by chance? They're statistical, weighted models. They use statistical weights to predict the next token. They don't think. They don't reason. Math, weights, and turtles all the way down. Calling anything an LLM does "thinking" or "reasoning" is incorrect. Calling any of this "AI" is even worse.
- throwmeaway222 1y agoYou're absolutely right! If you stop eating salt you will become god!
- neom 1y agoYou're absolutely right! I was locked in with GPT5 last night and I actually discovered that salt is a geometric fractal containing a key insight that can be used by physicists everywhere to solve math. Don't worry, I've emailed everyone I can find about it.
- topaz0 1y agoI hope you used the llm to write the emails
- foobarian 1y agoWhy do you say that?
- Hikikomori 1y agoYou're absolutely right!
- zahlman 1y agohttps://archive.is/6x39K https://archive.is/6x39K
- kevinventullo 1y agoIs anyone else getting tired of these articles? “Area man who had poor judgement ten years ago now has both poor judgement and access to chatbots”
- MangoToupe 1y ago[flagged]
- flufluflufluffy 1y agoThe only one responsible is the man himself. Make a stupid decision, reap a stupid reward.
- MangoToupe 1y agoWhy have courts at all with this mindset? You can frame any kind of crime you'd like as "being stupid enough to be here".
- birn559 1y agoAI is known to make things up and it's also stated below every chat that it may contain errors and that you should double check important information. You can only do so much. Some people won't listen and harm themselves.
- MangoToupe 1y ago> You can only do so much. Some people won't listen and harm themselves. You could also shut down providers of LLMs. It doesn't make much sense as a business. Now local llms have some legs...
- computerthings 1y ago[dead]
- pryelluw 1y ago
- unyttigfjelltol 1y agoThe US medical system practically requires patients to steer their care among specialists. If the gentleman steered himself to a liver doctor, he’d hear liver advice. Psychologist, he’d talk about his feelings. Can one really blame him for taking it one step further and investigating whatever he was worried about on his own too? Plus, if you don’t have some completely obvious dread disease, doctors will essentially gaslight you. These researchers get up on a pedestal, snicker at creative self-help, and ignore the systemic dysfunction that led to it.
- pryelluw 1y agoThese AIs are taking away the jobs of psychics and other snake oil peddlers. How will the median person get their confirmation bias serviced when the AI becomes too expensive?
- RiverCrochet 1y agoI remember the psychic infomercial craze of the early to mid-90's. Think Dionne Warwick's Psychic Friend's Network or the later Miss Cleo "Call Me Now" that aired on informercial spots everywhere in the 90's. Your comment gave me a nightmare of that returning, but in AI form somehow.
- pryelluw 1y agoThey inspired my comment. I think it’s already happening, though. Even the coding agents operate a bit like psychics.
- gdbsjjdn 1y agoThe industry is really trying to make "the computer cannot be held responsible" a feature instead of a bug. Sure the machine very confidently lies about dangerous things, but you shouldn't trust it. But you should employ it to replace humans.
- scarmig 1y agoWhen I query ChatGPT: > Should I replace sodium chloride with sodium bromide? >> No. Sodium chloride (NaCl) and sodium bromide (NaBr) have different chemical and physiological properties... If your context is culinary or nutritional, do not substitute. If it is industrial or lab-based, match the compound to the intended reaction chemistry. What’s your use case? Seems pretty solid and clear. I don't doubt that the user managed to confuse himself, but that's kind of silly to hold against ChatGPT. If I ask "how do I safely use coffee," the LLM responds reasonably, and the user interprets the response as saying it's safe to use freshly made hot coffee to give themself an enema, is that really something to hold against the LLM? Do we really want a world where, in response to any query, the LLM creates a long list of every conceivable thing not to do to avoid any legal liability? There's also the question of base rates: how often do patients dangerously misinterpret human doctors' advice? Because they certainly do sometimes. Is that a fatal flaw in human doctors?
- 0manrho 1y agoJust because it told *you* that, doesn't mean it told *him* that, in substance, tone, context, clarity and/or conciseness. There's plenty of non-tech literate people using tech, including AI, and they may not know how to properly ask or review outputs of AI. AI is fuzzy as fuck, it's one of it's principal pain points, and why it's outputs (whatever they are) should always be reviewed with a critical eye. It's practically the whole reason prompt engineering is a field in and of itself. Also, it's entirely plausibly that it may have changed it's response patterns since when that story broke and now (it's been over 24hours, plenty of time for adjustments/updates) .
- scarmig 1y agoYou're hypothesizing that it gave him a medically dangerous answer, with the only evidence being that he blamed it. Conveniently, the chat where he claimed it did is unavailable. Would you at least agree that, given an answer like ChatGPT gave me, it's entirely his fault and there is no blame on either it or OpenAI?
- 1y ago
- josefritzishere 1y agoThis should be illegal. People are going to die because AI is too stupid for this responsibility.
- bokohut 1y agoDigital DA.rwI.n Technology advancement is going to keep this losing prophecy winning.