> I imagine that mainstream assistant chatbots doing this is a brief occurrence. The next generations of AI systems will be trained to be better about not affir

> I imagine that mainstream assistant chatbots doing this is a brief occurrence. The next generations of AI systems will be trained to be better about not affirming psychotic users. OpenAI attempted to nuke their overly-agreeable LLM (GPT-4o). After user outcry they brought it back. I wouldn't be too sure that companies will do the responsible thing here; there's a pretty ugly history of companies who sell to addicts deliberately making their products more addictive and appealing to said addicts (the tobacco industry is particularly notorious for this). Like, one view would be that LLM 'companions' and 'therapists' are inherently harmful, or at least so likely to be net-harmful that making them available would be unjustifiable. But that's certainly not the view that the vendors are taking. > It has no long term plans - it just wants the user to like it. To be clear, it doesn't 'want' anything.