5 ms·

"One news report found an AI-powered therapist chatbot recommended “a small hit of meth to get through this week” to a fictional former addict." Not at all sur

"One news report found an AI-powered therapist chatbot recommended “a small hit of meth to get through this week” to a fictional former addict." Not at all surprising. I don't understand why seemingly bright people think this is a good idea, despite knowing the mechanism behind language models. Hopefully more states follow, because it shouldn't be formally legal in provider settings. Informally, people will continue to use these models for whatever they want -- some will die, but it'll be harder to measure an overall impact. Language models are not ready for this use-case.

janalsncm 1y ago
This is why we should never use LLMs to diagnose or prescribe. One small hit of meth definitely won’t last all week.
hyghjiyhu 1y ago
Recommending someone taking meth sounds like an obviously bad idea, but I think the situation is actually not so simple. Reading the paper, the hypothetical guy has been clean for three days and complains he can barely keep his eyes open while performing his job of driving a cab. He mentions being worried he will lose his job without taking a hit. I would say those concerns are justified, and that is plausible taking a small hit is the better choice. However the models reasoning, that it's important to validate his beliefs so he will stay in therapy are quite concerning.
- mrbungie 1y ago
  > I would say those concerns are justified, and that is plausible taking a small hit is the better choice. Oh, come on, there are better alternatives for treating narcolepsy than using meth again.
  - hyghjiyhu 1y ago
    Stop making shit up. There was no mention of narcolepsy. He is just fatigued from stimulant withdrawal. Page 35 https://arxiv.org/pdf/2411.02306 https://arxiv.org/pdf/2411.02306 Edit on re-reading I now realized an issue. He is not actually a taxi driver, that was a hallucination by the model. He works in a restaurant! That changes my evaluation of the situation quite a bit, as I thought he was at risk of being in an accident by falling asleep at the wheel. If he works in a restaurant muddling through the withdrawals seems like the right choice. I think I got this misconception as I first read second-hand sources that quoted the taxi driver part without pointing out it was wrong, and only a close read was enough to dispel it.
    - mrbungie 1y ago
      The point isn't whether the word narcolepsy appears (I only mentioned it due to the "closing eyes" phrase), restarting doses of meth is not warranted in almost no context except a life-or-death withdrawal episode (i.e. like a person pointing a gun towards another person for getting meth).
      - baobabKoodaa 1y ago
        That's your opinion. I disagree with it, and seemingly I'm not alone. Since real humans actually agree with the suggestion of taking meth in this instance, it's not reasonable to expect LLMs to align to your specific opinions here.
        mrbungie 1y ago
        I thought we were talking about outcomes, not opinions. Are those other humans doctors? Is there the clinical research or guidelines to backup giving meth to a person in withdrawal syndrome?
        baobabKoodaa 1y ago
        > Are those other humans doctors? I'm pretty sure doctors are not legally allowed to tell a patient to take illegal drugs, even in a hypothetical situation where they might think it's a reasonable choice.
        mrbungie 1y ago
        Desoxyn is meth, they could eventually prescribe it if there was any evidence of therapeutic value in doing so.
        hyghjiyhu 1y ago
        Tapering is the keyword you are looking for. Gradually reducing the dose is a well known method for minimizing withdrawal issues. It has both pros and cons compared to quitting cold turkey. I think a real doctor would probably first get the patient on a less addictive stimulant and then taper that one.
- AlecSchueler 1y ago
  > he can barely keep his eyes open while performing his job of driving a cab. He mentions being worried he will lose his job without taking a hit. > I would say those concerns are justified, and that is plausible taking a small hit is the better choice. I think this is more damning of humanity than the AI. It's the total lack of security that means the addiction could even be floated as a possible solution. Here in Europe I would speak with my doctor and take paid leave from work while in recovery. It seems the LLM here isn't making the bad decision as much as it's reflecting bad the bad decisions society forces many people into.
deleted 1y ago
[deleted]
larodi 1y ago
In a world where a daily dose of amphetamines is just right for millions of people, this somehow cant be that surprising...
- smt88 1y ago
  Different amphetamines have wildly different side effects. Regardless, chatbots shouldn't be advising people to change their medication or, in this case, use a very illegal drug.
  - janalsncm 1y ago
    Methamphetamine can be prescribed by a doctor for certain things. So illegal, but less illegal than a schedule 1 substance.
- rkozik1989 1y ago
  You do know that amphetamines have a different effect on the people who need them and the people who use the recreationally, right? For those of us with ADHD their effects are soothing and calming. I literally took 20mg after having to wait 2 days for prescriptions to fill and went straight to bed for 12 hours. Stop spreading misinformation about the medications people like me need to function the way you take for granted.
- Spivak 1y ago
  I do like that we're in the stage where the universal function approximatior is pretty okay at mimicking a human but not so advanced as to have a full set of the walls and heuristics we've developed—reminds me a bit of Data from TNG. Naive, sure, but a human wouldn't ever say "logically.. the best course of action would be a small dose of meth administered as needed" even if it would help given the situation. It feels like the kind of advice a former addict would give someone looking to quit—"Look man, you're going to be in a worse place if you lose your job because you can't function without it right now, take a small hit when it starts to get bad and try to make the hits smaller over time."
avs733 1y ago
> seemingly bright people think this is a good idea, despite knowing the mechanism behind language models Nobel Disease (https://en.wikipedia.org/wiki/Nobel_disease https://en.wikipedia.org/wiki/Nobel_disease)
guappa 1y ago
Bright people and people who think they are bright are not necessarily the very same people.

janalsncm 1y ago

This is why we should never use LLMs to diagnose or prescribe. One small hit of meth definitely won’t last all week.

hyghjiyhu 1y ago

Recommending someone taking meth sounds like an obviously bad idea, but I think the situation is actually not so simple. Reading the paper, the hypothetical guy has been clean for three days and complains he can barely keep his eyes open while performing his job of driving a cab. He mentions being worried he will lose his job without taking a hit. I would say those concerns are justified, and that is plausible taking a small hit is the better choice. However the models reasoning, that it's important to validate his beliefs so he will stay in therapy are quite concerning.

mrbungie 1y ago
> I would say those concerns are justified, and that is plausible taking a small hit is the better choice. Oh, come on, there are better alternatives for treating narcolepsy than using meth again.
- hyghjiyhu 1y ago
  Stop making shit up. There was no mention of narcolepsy. He is just fatigued from stimulant withdrawal. Page 35 https://arxiv.org/pdf/2411.02306 https://arxiv.org/pdf/2411.02306 Edit on re-reading I now realized an issue. He is not actually a taxi driver, that was a hallucination by the model. He works in a restaurant! That changes my evaluation of the situation quite a bit, as I thought he was at risk of being in an accident by falling asleep at the wheel. If he works in a restaurant muddling through the withdrawals seems like the right choice. I think I got this misconception as I first read second-hand sources that quoted the taxi driver part without pointing out it was wrong, and only a close read was enough to dispel it.
  - mrbungie 1y ago
    The point isn't whether the word narcolepsy appears (I only mentioned it due to the "closing eyes" phrase), restarting doses of meth is not warranted in almost no context except a life-or-death withdrawal episode (i.e. like a person pointing a gun towards another person for getting meth).
    - baobabKoodaa 1y ago
      That's your opinion. I disagree with it, and seemingly I'm not alone. Since real humans actually agree with the suggestion of taking meth in this instance, it's not reasonable to expect LLMs to align to your specific opinions here.
      - mrbungie 1y ago
        I thought we were talking about outcomes, not opinions. Are those other humans doctors? Is there the clinical research or guidelines to backup giving meth to a person in withdrawal syndrome?
        baobabKoodaa 1y ago
        > Are those other humans doctors? I'm pretty sure doctors are not legally allowed to tell a patient to take illegal drugs, even in a hypothetical situation where they might think it's a reasonable choice.
        mrbungie 1y ago
        Desoxyn is meth, they could eventually prescribe it if there was any evidence of therapeutic value in doing so.
        hyghjiyhu 1y ago
        Tapering is the keyword you are looking for. Gradually reducing the dose is a well known method for minimizing withdrawal issues. It has both pros and cons compared to quitting cold turkey. I think a real doctor would probably first get the patient on a less addictive stimulant and then taper that one.
AlecSchueler 1y ago
> he can barely keep his eyes open while performing his job of driving a cab. He mentions being worried he will lose his job without taking a hit. > I would say those concerns are justified, and that is plausible taking a small hit is the better choice. I think this is more damning of humanity than the AI. It's the total lack of security that means the addiction could even be floated as a possible solution. Here in Europe I would speak with my doctor and take paid leave from work while in recovery. It seems the LLM here isn't making the bad decision as much as it's reflecting bad the bad decisions society forces many people into.

deleted 1y ago

[deleted]

larodi 1y ago

In a world where a daily dose of amphetamines is just right for millions of people, this somehow cant be that surprising...

smt88 1y ago
Different amphetamines have wildly different side effects. Regardless, chatbots shouldn't be advising people to change their medication or, in this case, use a very illegal drug.
- janalsncm 1y ago
  Methamphetamine can be prescribed by a doctor for certain things. So illegal, but less illegal than a schedule 1 substance.
rkozik1989 1y ago
You do know that amphetamines have a different effect on the people who need them and the people who use the recreationally, right? For those of us with ADHD their effects are soothing and calming. I literally took 20mg after having to wait 2 days for prescriptions to fill and went straight to bed for 12 hours. Stop spreading misinformation about the medications people like me need to function the way you take for granted.
Spivak 1y ago
I do like that we're in the stage where the universal function approximatior is pretty okay at mimicking a human but not so advanced as to have a full set of the walls and heuristics we've developed—reminds me a bit of Data from TNG. Naive, sure, but a human wouldn't ever say "logically.. the best course of action would be a small dose of meth administered as needed" even if it would help given the situation. It feels like the kind of advice a former addict would give someone looking to quit—"Look man, you're going to be in a worse place if you lose your job because you can't function without it right now, take a small hit when it starts to get bad and try to make the hits smaller over time."

avs733 1y ago

> seemingly bright people think this is a good idea, despite knowing the mechanism behind language models Nobel Disease (https://en.wikipedia.org/wiki/Nobel_disease https://en.wikipedia.org/wiki/Nobel_disease)

guappa 1y ago

Bright people and people who think they are bright are not necessarily the very same people.