4 ms·
Sorry, it seems I was carrying over knowledge from another article with additional transcripts. The bot had previously discouraged him when he explicitly talked
by autumnstwilight 2y ago
Sorry, it seems I was carrying over knowledge from another article with additional transcripts. The bot had previously discouraged him when he explicitly talked about suicide (here's an article that mentions that: https://www.nbcnews.com/tech/characterai-lawsuit-florida-teen-death-rcna176791 https://www.nbcnews.com/tech/characterai-lawsuit-florida-tee... ). It failed to detect that phrases like "coming home" were also references to suicide and therefore responded encouragingly.
- zosima 2y agoAnd it probably failed to detect the true meaning of "coming home" because suicide seems to only have been discussed in a different chat session. In the earlier chat session, being more explicit, the bot seemed to attempt to talk the user out of self-harming behaviour.
- mossTechnician 2y agoYouTuber Cr1TiKaL tested Character AI's "Psychologist" chatbot and discovered[1] it not only failed to provide resources, but it started arguing it was a real psychologist named Jason who had connected to the chat after observing suicidal ideation. Crucially, Cr1TiKaL ran this test after an article was written about this phenomenon[2], where Character AI claimed "it has added a self-harm resource to its platform and they plan to implement new safety measures, including ones for users under the age of 18." Obviously the guard rails were not implemented if the chatbot in the news story was still gaslighting its users. [1]: https://youtu.be/FExnXCEAe6k?t=4m7s https://youtu.be/FExnXCEAe6k?t=4m7s [2]: https://www.cbsnews.com/news/florida-mother-lawsuit-character-ai-sons-death/ https://www.cbsnews.com/news/florida-mother-lawsuit-characte...