4 ms·

I understand your point, but the word selection can be quite subjective. I've taken a lot of care when finding the right words. Copyright will improve as it evo

I understand your point, but the word selection can be quite subjective. I've taken a lot of care when finding the right words. Copyright will improve as it evolves. I don't understand your reference to people with a 'serious mental health crisis.' I've specifically mentioned this should not be used as such and it is not marketed as such. I'm a firm believer that people in that condition should get professional therapy.

racional 2y ago
I don't understand your reference to people with a 'serious mental health crisis.' It comes from the fact that a significant portion of your users will surely be in that category. I understand your point, but the word selection can be quite subjective. Right, but in such cases (and especially when dealing with mental health services) one should apply the cautionary principle. And be extra careful to not use words that suggest that your service provides something ("compassion", "emotional support") which it plainly -- does not.
- remusnegrota 2y ago
  I am well aware of the stigma behind AI and empathy, but a recent study showed that LLMs are starting to provide better emotional support than most humans. I know these are crazy times. Look, I understand your points and am well aware of the thin line that the product must thread, but this is the product's mission: to educate people on the roles AI can have, and the main goal is to talk out loud your thoughts. The most important thing is to take steps towards becoming self-aware of our thoughts by speaking out loud: This is what is meant to be, this is what is presented, nothing more. I will revise the copyright to make this clearer.
  - racional 2y ago
    A recent study showed that LLMs are starting to provide better emotional support than most humans Can you provide a citation?
    - remusnegrota 2y ago
      Sure, here is the paper: https://osf.io/preprints/psyarxiv/fzvd8 https://osf.io/preprints/psyarxiv/fzvd8 And here is a post from one of the contributors on the research, that summarizes it: https://x.com/Amit_Goldenb/status/1782076346929475602 https://x.com/Amit_Goldenb/status/1782076346929475602
      - racional 2y ago
        Thanks. The paper describes LLM performance on a single, very narrowly prescribed task - compared with (if I am reading correctly) average humans (not trained practicioners in the field). This is of course very, very far from a validation of these models as being on the whole suitable for use in real-life counseling situations. Involving, you know, real people facing a diverse set of mental health situations and risks. Look, it's a fine technical achievement, your site. I'm pretty sure also that what I'm saying above is not news to you, and that you're perfectly aware of the current primitive state of these models. And that you're also aware of the fact that none of technology has been formally vetted or approved by anyone in the public health field. In that sense -- it is really quite irresponsible of you to promote this tool in the way that you're doing it. Especially the language implying this is finished, trusted product ("Trusted AI companion"), and with human-like qualities no less ("is curious, compassionate"). And on top of that, to take people's money for it -- given that these tools can be quite addictive. And you're potentially exploiting people in very difficult situations and with limited financial options. I know you have other disclaimers, but that's what I mean: These disclaimers are useless when they are contradicted by other statements (such as those above) which say basically the opposite. That's why the disclaimers come off as -- doublespeak, basically. You need to really re-think what you're doing here, and why you're doing it.
        remusnegrota 2y ago
        Thank you for your view and for the civilized debate. Still, you are steering it to extremes with arguments such as: " And you're potentially exploiting people in very difficult situations and with limited financial options." I have to disagree with you on that, you are basically comparing my product with gambling, which is unfair. That being said, these kinds of discussions, in general, have a net benefit in the world.

racional 2y ago

I don't understand your reference to people with a 'serious mental health crisis.' It comes from the fact that a significant portion of your users will surely be in that category. I understand your point, but the word selection can be quite subjective. Right, but in such cases (and especially when dealing with mental health services) one should apply the cautionary principle. And be extra careful to not use words that suggest that your service provides something ("compassion", "emotional support") which it plainly -- does not.

remusnegrota 2y ago
I am well aware of the stigma behind AI and empathy, but a recent study showed that LLMs are starting to provide better emotional support than most humans. I know these are crazy times. Look, I understand your points and am well aware of the thin line that the product must thread, but this is the product's mission: to educate people on the roles AI can have, and the main goal is to talk out loud your thoughts. The most important thing is to take steps towards becoming self-aware of our thoughts by speaking out loud: This is what is meant to be, this is what is presented, nothing more. I will revise the copyright to make this clearer.
- racional 2y ago
  A recent study showed that LLMs are starting to provide better emotional support than most humans Can you provide a citation?
  - remusnegrota 2y ago
    Sure, here is the paper: https://osf.io/preprints/psyarxiv/fzvd8 https://osf.io/preprints/psyarxiv/fzvd8 And here is a post from one of the contributors on the research, that summarizes it: https://x.com/Amit_Goldenb/status/1782076346929475602 https://x.com/Amit_Goldenb/status/1782076346929475602
    - racional 2y ago
      Thanks. The paper describes LLM performance on a single, very narrowly prescribed task - compared with (if I am reading correctly) average humans (not trained practicioners in the field). This is of course very, very far from a validation of these models as being on the whole suitable for use in real-life counseling situations. Involving, you know, real people facing a diverse set of mental health situations and risks. Look, it's a fine technical achievement, your site. I'm pretty sure also that what I'm saying above is not news to you, and that you're perfectly aware of the current primitive state of these models. And that you're also aware of the fact that none of technology has been formally vetted or approved by anyone in the public health field. In that sense -- it is really quite irresponsible of you to promote this tool in the way that you're doing it. Especially the language implying this is finished, trusted product ("Trusted AI companion"), and with human-like qualities no less ("is curious, compassionate"). And on top of that, to take people's money for it -- given that these tools can be quite addictive. And you're potentially exploiting people in very difficult situations and with limited financial options. I know you have other disclaimers, but that's what I mean: These disclaimers are useless when they are contradicted by other statements (such as those above) which say basically the opposite. That's why the disclaimers come off as -- doublespeak, basically. You need to really re-think what you're doing here, and why you're doing it.
      - remusnegrota 2y ago
        Thank you for your view and for the civilized debate. Still, you are steering it to extremes with arguments such as: " And you're potentially exploiting people in very difficult situations and with limited financial options." I have to disagree with you on that, you are basically comparing my product with gambling, which is unfair. That being said, these kinds of discussions, in general, have a net benefit in the world.