6 ms·
With AI chatbots, Big Tech is moving fast and breaking people
- cbluth 1y agoI know someone that has "resurrected" a dead relative through an llm, and I've seen the nonsense on forums about dating an "ai boyfriend"... Some people can't help themselves and don't use these tools appropriately.
- dingnuts 1y agothe good news is that this shit isn't sustainable. when investor funtime subsidies end, ain't nobody spending $2000/mo for an AI boyfriend. "but it's getting cheaper and cheaper to run inference" they say. To which I say: ok bud sure thing, this isn't a technology governed by Moore. We'll see.
- nullc 1y agoIt doesn't require a particularly powerful AI because the human's own hope is doing the heavy lifting. 70B models run juust fine on hardware you can have sitting under your desk.
- deadbabe 1y ago$2000? I see $4000/month minimum, roughly equivalent to what some typical Wall Street data feeds are priced at. It’s big business.
- tokai 1y agoPeople have spend much more on pig butchering scam boyfriends that don't even exist. I bet you could get some people to pay quit a lot to keep what they see as their significant other alive.
- fullshark 1y agoMental health issues in the population are never going away, people using software tools to prey on those with issues is never going away. Arguably the entire consumer software industry preys on addiction and poor impulse control already.
- dingnuts 1y agoyeah, I know, I'm addicted to this hellhole of a site. I hate it but I still open it every five minutes. But that's not what I'm talking about, I'm talking specifically about people who've made a SoTA model their buddy, like the people who were sad when 4o disappeared. Users of character.ai. That sort of thing. It's going to get very, very expensive and provides very little value. People are struggling with rent. These services won't be able to survive, I hope, purely through causing psychosis in vulnerable people.
- valbaca 1y ago> ain't nobody spending $2000/mo for an AI boyfriend you haven't met the Whales (big spenders) in the loneliness-epidemic industry (e.g. OnlyFans and the like)
- dingnuts 1y agowhy would they stop paying for the attention of a real woman when the artificial alternative costs just as much?
- AlexandrB 1y agoBoth are fake. Do you think OnlyFans models are actually giving their customers their attention?
- JohnMakin 1y agoOn OF specifically, you are much more likely to be spending money talking to a bot that sends you sexy pics and messages than a real human being.
- evilduck 1y agoReliability and consistency? Humans have bad days. Humans have needs and can't be available 24/7/365 for years on end. OF creators burn out or grow up or lose interest or cash out and retire. It's not like the "real women" of OnlyFans are consistently real, or women. And there's some percentage that are already AI-by-proxy. There's definitely opportunity for someone to just skip the middleman.
- deleted 1y ago[deleted]
- BoredPositron 1y agoWhen I was at a game studio for a big MMORPG I had the valuable experience sitting next to the monetization team. It was a third grade MMO with gacha mechanics our whales spend 20-30k every month... for years.
- deadbabe 1y agoCan you talk more about the resurrection? Did they train an LLM fine tune against as much written content as possible made by that person?
- nullc 1y agoThat might actually be interesting if there were enough content, something of the "Beta" level AI's in Alastair Reynolds' revelation space books. But that isn't what I've seen done when people said they did that. Instead they just told ChatGPT a bit about the person and asked it to playact. The result was nothing like the person-- just the same pathetic ChatGPT persona, but in their confusion, grief, and vulnerability they thought it was a recreation of the deceased person. A particularly shocking and public example is the Jim Acosta interview of the simulacra of a parkland shooting victim.
- MisterTea 1y agoMy friend admitted to having political and religious arguments with chatgpt. They have mental issues which contributed.
- beacon473 1y agoWhat's wrong with using an LLM to learn about politics and religion? I've found Claude to be an excellent tool to facilitate introspective psychoanalysis. Unlike most human therapists I've worked with, Claude will call me on my shit and won't be talked into agreeing with my neurotic fantasies (if prompted correctly).
- BoredPositron 1y agoNo you are gaslighting yourself without recognizing it. That's what we are talking about.
- iinnPP 1y agoIt's impossible to gaslight yourself as it requires intention.
- BoredPositron 1y agoHe wants to be called a good boy, so the LLM calls him a good boy. Since the LLM is a machine that does what you want, he's essentially doing it to himself. It might not be a conscious choice, but there's still intention behind it. Kein Herr im eigenen Haus. (No master in one's own house.) - Sigmund Freud. He was wrong about a lot of stuff but this is one thing that still stands. It's called unconscious intention, and here's a pretty interesting paper that'll bring you up to speed: https://irl.umsl.edu/cgi/viewcontent.cgi?article=1206&context=thesis https://irl.umsl.edu/cgi/viewcontent.cgi?article=1206&contex...
- iinnPP 1y agoI'm sure you can be unconsciously intent on things, but gaslighting is a unique concept. Here's the definition I am relying on: manipulate (someone) using psychological methods into questioning their own sanity or powers of reasoning. In your provided example, the user is obviously not trying to manipulate someone into questioning their sanity, nor power of reasoning. Quite the opposite. Lying to themselves (your example) for sure.
- datadrivenangel 1y agoAnd this is only going to get worse. The worst engagement hacking of social media but even more personalized.
- wafflemaker 1y agoSo far even the least* "ethical" companies (raising initial money by pretending to be open) don't use machine learning based on users psychological profiles to create perfect hypno-drug-dopamine-poison (sorry, but the English word for drugs doesn't really carry the deserved weight). And Snapchat, TikTok, Facebook, YouTube, Instagram, more?.. are just it. They waste time worth $50-$100 just to make $0.05 in ads. LLMs seem to be far from that. * I might be ignorant of the real picture, please correct me if I'm wrong and not aware of really evil LLM companies.
- Analemma_ 1y agoWait what? Companies are absolutely doing this at scale, they're just laundering it through third-party content providers, who get seen or not seen based on what the user has lingered on in the past. If you haven't seen your relatives' Facebook feeds lately, they're close to 100% LLM-generated slop, which gradually gets tuned to the specific user based on what their eyeballs have lingered on in the past. TikTok isn't quite there yet but is trending in the same direction. So it's platform partners plus filtered selection putting the tuned dopamine poison in front of people for the moment, but it's absolutely happening. And eventually the platform owners can and will cut out the middlemen.
- EGreg 1y agoIn areas I am an expert in, I am sure my discoveries are interesting, and probably novel. I am working now to actually build them myself. ChatGPT is just helping discuss what else is out there. But in areas I am not, I am still skeptical, but enthusiastic — that they’ll be vindicated maybe in a decade. Any physics or QM experts can opine on this? https://magarshak.com/blog/?p=568 https://magarshak.com/blog/?p=568
- david_shaw 1y agoI love playing with advancing technologies, and although I don't think LLM/Agentic AI is quite ready to change the world, one day soon it might be -- but the volume of individuals falling into AI-induced psychosis is astounding and horrifying. For those of you who, thankfully, don't have personal experience, it generally goes like this: reasonable-ish individual starts using AI and, in turn, their AI either develops or is prompt-instructed to have certain personality traits. LLMs are pretty good at this. Once the "personality" develops, the model reinforces ideas that the user puts forth. These can range from emotional (such as the subreddit /r/MyBoyfriendIsAI) to scientific conspiracies ("yes, you've made a groundbreaking discovery!"). It's easy to shrug these instances off as unimportant or rare, but I've personally witnessed a handful of people diving off the deep-end, so to speak. Safety is important, and it's something many companies are failing to adequately address. It'll be interesting to see where this leads over the next year or so, as the technology -- or at least quality of models -- continues to improve.
- BoredPositron 1y agoThey are working on it since gpt3 and made 0 progress. The newer models are even more prone to hallucinate. The only thing they do is trying to hide it.
- ACCount37 1y agoWhat? First, hallucinations aren't even the problem we're talking about. Second, there have been marked improvements on hallucination rates with just about every frontier release (o3 being the notable outlier).
- BoredPositron 1y agoCitation needed. The base models have not improved. The tooling might catch more but if you get an azure endpoint without restrictions you are in for a whole lot of bullshit.
- ACCount37 1y agoWe have very little data on the actual "volume of individuals falling into AI-induced psychosis" - and how that compares to pre-AI psychosis rate. It'll be a while until we do. It might be an actual order-of-magnitude increase from the background rates. It might be basically the same headcount, but with psychosis expressed in a new notable way. I.e. that guy who would be planning to build a perpetual motion machine to disprove quantum physics is now doing the same, but with an AI chatbot at his side - helping him put his ramblings into a marginally more readable form.
- SirFatty 1y ago"Brooks isn't alone. Futurism reported on a woman whose husband, after 12 weeks of believing he'd "broken" mathematics using ChatGPT, almost attempted suicide." How does one "almost attempt" something?
- deleted 1y ago[deleted]
- miltonlost 1y agoPurchasing large amounts of pills for instance. You can almost attempt riding a bicycle by buying one but not actually using it.
- nullc 1y agoAsk the AI for a method but whatever it hallucinates end up being something that couldn't actually kill you? Mostly a joke but one the common properties in people with an delusional or otherwise unhealthy pattern of AI use appears to be substituting it for all their own choices, even where it's obviously inappropriate. E.g. if you try to intervene in their conduct, you'll just get an AI generated reply.
- Dilettante_ 1y agoI guess the same way one turns up missing[1]. [1]https://youtube.com/watch?v=gkBXfmnlVIg https://youtube.com/watch?v=gkBXfmnlVIg
- deleted 1y ago[deleted]
- HankStallone 1y agoSay you're several stories up in a building and having a really bad day, so you start thinking about jumping out the window and actually walk over and look down, but then decide against it. If someone asked you later if you'd ever attempted suicide, "almost" seems like a reasonable answer.
- rsynnott 1y agoYou could argue the definition, but if you take "attempt" to be the actual act which might kill you, which is fairly normal usage, then preparatory acts short of the final act would count as 'almost attempt'. (It's a fairly awkward phrase, and you wouldn't say "I almost attempted to run a marathon", say, but "attempt suicide" has a fairly specific commonly-understood meaning where 'attempt' is unusually restrictive vs general usage)
- ACCount37 1y agoI imagine that mainstream assistant chatbots doing this is a brief occurrence. The next generations of AI systems will be trained to be better about not affirming psychotic users. But it goes to show just how vulnerable a lot of humans are. By all accounts, GPT-4o isn't malicious. It has no long term plans - it just wants the user to like it. It still does this kind of thing to people. An actual malicious AI capable of long term planning would be able to do that, and worse. One of the most dangerous and exploitable systems you can access online is a human.
- gjsman-1000 1y agoThis is one item on my long list of flaws with AI. OpenAI thinks that as long as you aren't using dating terminology or asking to build bombs, you're probably safe. - What does having an unquestioning mirror that is always gracious and reflects your own biases back at you, do to the mind in the long term? We don't know. - How do you know, that after 2-3 years of this use, without any particular warning, it will not suddenly escalate into a delusion spiral? We don't know. - Does this affect childhood development to have no pushback on their personal feelings and opinions? We don't know. - Does this affect people's very willingness to learn, and to endure intellectual struggle? We don't know. - Does this affect long-term retention, as losing even an imperceptible 2% of your knowledge every month, is 48% lost after 2 years? We don't know. - How does this affect a population of which 27% have experienced a mental health issue in the last year, and of which 9% are experiencing depression? We don't know. - Are all these problems solvable with any amount of education or warning labels, given that human nature has not changed and will not change? We don't know. - Can this be prevented with any amount of AI guardrails or ethics training, or is it inherent to the structure of the system itself? We don't know. - Will this eventually be considered so dangerous, that we need black-box warning labels similar to cigarettes and possibly social media? We don't know. - Will this eventually be considered so insolvably risky for human cognition, that we outright restrain use without a license, either like driving at best, or asbestos at worst? We don't know. - Will any employer be able to obtain insurance for AI use, if there is even a 0.2% documented chance that any employee could go psychotic from using it, completely at random? We don't know. - Will said employees, if they do go psychotic from a work-mandated AI chatbot, successfully sue their employers for damages? We don't know.
- akomtu 1y agoMost people are addicted to pleasures of this world. A subtle kind of this is flattery and chatbots have mastered this art. Was is done with a devious intent? We don't know.
- sch-sara 1y agoI asked claude if I should worry about this and they said no
- AstroBen 1y agoEvery social media platform has engineered their product to be as addictive, time-sucking and manipulative as they can make it. I'm fully expecting AI to be treated the same, except much more potent I suppose the only hope here is that its economic value comes mainly from its productive output (summarization, code, writing, text transformation whathaveyou)
- rsynnott 1y ago> A machine that uses language fluidly, convincingly, and tirelessly is a type of hazard never encountered in the history of humanity. Though, interestingly, it has been _predicted_ for a while. It's a bit blink-and-you'll-miss-it, but the Ministry of Truth in 1984 has book-writing machines!