5 ms·
Throwaway account. My ex partner was involved in a study which said these things were not ok. They were paid not to publish by an undisclosed party. That's how
by hy555 1y ago
Throwaway account. My ex partner was involved in a study which said these things were not ok. They were paid not to publish by an undisclosed party. That's how bad it has got.
Edit: the study compared therapist outcomes to AI outcomes to placebo outcomes. Therapists in this field performed slightly better than placebo, which is pretty terrible. The AI outcomes performed much worse than placebo which is very terrible.
- cube00 1y agoThe amount of free money sloshing around the AI space is ridiculous at the moment.
- sorenjan 1y agoWhat did they use for placebo? Talking to somebody without education, or not talking to anybody at all?
- hy555 1y agoNot talking to anyone at all.
- zargon 1y agoWhat did they do then? If they didn't do anything, how can it be considered a placebo?
- risyachka 1y agoDoes it matter? The point is AI made it worse.
- phren0logy 1y agoIt's called a "waitlist" control group, and it's not intended to represent placebo. Or at least, it shouldn't be billed that way. It's not an ideal study design, but it's common enough that you could use it to compare one therapy to another based on their results vs a waitlist control. Placebo control for psychotherapy is tricky and more expensive, and can be hard to get the funding to do it properly.
- deleted 1y ago[deleted]
- trod1234 1y agoThat seems like a very poor control group.
- hy555 1y agoThat is one of my concerns.
- neilv 1y agoSounds like suppressing research, at the cost of public health/safety. Some people knew what the tobacco companies were secretly doing, yet they kept quiet, and let countless family tragedies happen. What are best channels for people with info to help halt the corruption, this time? (The channels might be different than usual right now, with much of US federal being disrupted.)
- hy555 1y agoStart digging into psychotherapy research and tearing their papers apart. Then the SPR. Whole thing is corrupt to the core. A lot of papers drive public health policy outside the field as it's so vague and easy to cite but the research is only fit for retraction watch.
- neilv 1y agoBeing paid to suppress research on health/safety is potentially a different problem than, say, a high rate of irreproducible results. And if the alleged payer is outside the field, this might also be relevant to the public interest in other regards. (For example, if they're trying to suppress this, what else are they trying to do. Even if it turns out the research is invalid.)
- hy555 1y agoBoth are a problem. I should not conflate the two. I agree. Asking questions which are normal in my own field resulted in stonewalling and obvious distress. The worst thing being this leading to the end of what was a good relationship.
- neilv 1y agoIf the allegation is true, hopefully your friend speaks up. If not, you might consider whether you have actionable information yourself, any professional obligations you have (e.g., if you work in science/health/safety yourself), any societal obligations, whether reporting the allegation would be betraying a trust, and what the calculus is there.
- scotty79 1y agoI've heard of some more modern research with llms that had a result that Ai therapist was straight up better than human therapists across all measures.
- ilaksh 1y agoWhich model exactly? What type of therapy/prompt? Was it a completely dated model, like in the article where they talk about a model from two years ago? We have had massive progress in two years.
- raverbashing 1y agoHonestly none of the companies are tuning their model to be better at therapy. Also it is not expected that the training material for the model deals with the actual practical aspects of therapy, only some of the theoretical aspects are probably in that material
- ilaksh 1y agoThe leading edge models are trainable via instructions. That's why agents are possible. Many online therapy or therapy companies are training or instructing their agents in this domain.
- ktallett 1y agoThat still wouldn't allow for edge/unusual cases and situations which I would say having experienced group therapy for many years, most significant therapy users have quite a few of.
- jdietrich 1y ago>none of the companies are tuning their model to be better at therapy BrickLabs have developed an expert-fine-tuned model specifically to provide psychotherapy. Their model has shown modestly positive results in a reasonably large preregistered RCT. https://trytherabot.com/ https://trytherabot.com/ https://ai.nejm.org/doi/full/10.1056/AIoa2400802 https://ai.nejm.org/doi/full/10.1056/AIoa2400802
- raverbashing 1y agoYeah but 99% of people trying "AI mental health" are using free ChatGPT, etc
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- rsynnott 1y agoI'm quite curious how the placebo in a study like this works.
- derbOac 1y agoUsually in psychotherapy controls, there's either: waitlist control, where people get nothing psychoeducational, where people get some kind of educational content about mental health but not therapy existing nonpsychological service, like physical checkups with a nurse existing therapy, so not placebo but current treatment pharmacological placebo, where they're given a placebo pill and told its psychiatric medication for their concern A kind of "nerfed" version of the therapy, such as supportive therapy where the clinician just provides empathy etc but nothing else How to interpret results depends on the control. It's relevant to debates about general vs specific effects in therapy (rapport, empathy, fit) versus specific effects (effects due to specific techniques of a specific therapy). Bruce Wampold has written a lot about types of controls although he has a hard nonspecific/general effects take on therapy.
- twobitshifter 1y agoThey should do ELIZA as the control or at least include it to see how far we have or haven’t advanced.
- kbelder 1y agoOr a normal person with no training in therapy?