3 ms·
> "The incentive is to keep you online," Stanford University psychiatrist Nina Vasan told Futurism. The AI "is not thinking about what is best for you, what's b
by Permit 1y ago
> "The incentive is to keep you online," Stanford University psychiatrist Nina Vasan told Futurism. The AI "is not thinking about what is best for you, what's best for your well-being or longevity... It's thinking 'right now, how do I keep this person as engaged as possible?'"
Is this actually true? Or is this just someone retelling what they’ve read about social media algorithms?
- deleted 1y ago[deleted]
- bravesoul2 1y agoI doubt it's true yet but give it time.
- tough 1y agoThey're basically simplifying and romanticizing how RHLF works. https://openai.com/index/sycophancy-in-gpt-4o/ https://openai.com/index/sycophancy-in-gpt-4o/ https://www.anthropic.com/news/towards-understanding-sycophancy-in-language-models https://www.anthropic.com/news/towards-understanding-sycopha...
- lionkor 1y agoIt's not "thinking" in any sense of the word. As any LLM about budget date ideas in your city, for example, and watch them come up with the most cookie-cutter, boring, cringe filled content you've ever seen. Like blog spam, but condensed into a hyper friendly summary that optimizes for maximum plausibility and minimum offensiveness. It's an extreme stretch to suggest that there is any thinking involved.
- khnorgaard 1y agoI find that more often than not the LLM will try to keep the conversation going instead of ending it.
- jazzcomputer 1y agoI had a break from ChatGPT for a few months and got back onto it last week with some questions about game engines. I noticed that this time it's asking a lot of stuff when it looks like I'm coming to the end of my questions - like, "would you like me to go through with..." or "would you like me to help you with setting up..." Previously it felt less this way but it was notable as it seemed to sense I was coming towards the end of my questions and wanted me to stick around.
- luluthefirst 1y agoI think you are referring to the 'ask follow-up questions' toggle in the settings but it's not available on all devices to turn it off.
- kirth_gersen 1y agoI have it toggled off and it does do this less, but still often enough to be mildly annoying.
- donatj 1y agoFrom an economic standpoint probably not. The individual queries cost real money. They want you to like the service and pay for it, but there's not much in it for OpenAI for you to use it obsessively beyond training data.
- mike_hearn 1y agoNah, it's just academic slop of the type every journalist has a crack-level addiction to. OpenAI's incentives are the exact opposite: users pay them a flat fee (or nothing) but OpenAI's cost scale per interaction. OpenAI make more money when people subscribe but don't talk to ChatGPT much, i.e. their incentives are the inverse of what Vasan is claiming here. Ironically, the Stanford psychiatrist is hallucinating some statistically likely words whilst misinforming readers, perhaps in a way that will make them paranoid. It's turtles all the way down.