7 ms·
LLMs are increasingly part of intimate conversations. That proximity lets them learn how to manipulate minds. We must stop treating humans as uniquely mysterio
by anon-3988 1y ago
LLMs are increasingly part of intimate conversations. That proximity lets them learn how to manipulate minds.
We must stop treating humans as uniquely mysterious. An unfettered market for attention and persuasion will encourage people to willingly harm their own mental lives. Think social medias are bad now? Children exposed to personalized LLMs will grow up inside many tiny, tailored realities.
In a decade we may meet people who seem to inhabit alternate universes because they’ve shared so little with others. They are only tethered to reality when it is practical for them (to get on busses, the distance to a place, etc). Everything else? I have no idea how to have a conversation with someone else anymore. They can ask LLMs to generate a convincing argument for them all day, and the LLMs would be fine tuned for that.
If users routinely start conversations with LLMs, the negative feedback loop of personalization and isolation will be complete.
LLMs in intimate use risk creating isolated, personalized realities where shared conversation and common ground collapse.
- TimTheTinker 1y ago> Children exposed to personalized LLMs will grow up inside many tiny, tailored realities. It's like the verbal equivalent of The Veldt by Ray Bradbury.[0] [0] https://www.libraryofshortstories.com/onlinereader/the-veldt https://www.libraryofshortstories.com/onlinereader/the-veldt
- astrange 1y agoThe moral of this story is that if you install a really good TV the animals will come out of it and eat you? Is the author a dog?
- TimTheTinker 1y agoI suggest taking a literature course and learning how to interpret narratives. The Veldt is a classic short story written in 1950 by Ray Bradbury, a famous and celebrated author, who also wrote the famous dystopian novel Fahrenheit 451.
- astrange 1y agoIronically, Bradbury likes to tell people that Fahrenheit 451 isn't about the thing it was obviously supposed to be about (censorship) because he now wants it to have been a metaphor for cancel culture.
- f33d5173 1y agoCancel culture is censorship?
- astrange 1y agoOne of those involves fulltime professionals backed by state violence and the other is when people on social media are mad at you.
- ethbr1 1y agoIsn't the only difference whether the censors are in or out of government power? Few now respect the wisdom of 'should not' even when 'can'
- astrange 1y agoThat difference is so big it makes it an entirely different thing.
- TimTheTinker 1y ago> when people on social media are mad at you It's about more than that - many people have lost their jobs, been de-banked, or even been arrested (especially in countries like the UK and Germany) for expressing their opinion publicly when that opinion was merely (a) what most people in their country believed in the recent past (< 50 years ago), and (b) a politically incorrect opinion.
- jayGlow 1y agohe's been dead for a decade so I doubt he now wants the meaning to be anything. besides that he also never said anything about cancel culture he said it's about how tv turns you into a moron. https://www.openculture.com/2017/08/ray-bradbury-reveals-the-true-meaning-of-fahrenheit-451.html https://www.openculture.com/2017/08/ray-bradbury-reveals-the...
- lawlessone 1y agoWith the way LLMs are affecting paranoid people by agreeing with their paranoia it feels like we've created schizophrenia as a service.
- ip26 1y agoIt doesn't have to be that way of course. You could envision an LLM whose "paperclip" is coaching you to become a great "xyz". Record every minute of your day, including your conversations. Feed it to the LLM. It gives feedback on what you did wrong, refuses to be your social outlet, and demands you demonstrate learning in the next day before it rewards with more attention. Basically, a fanatically devoted life coach that doesn't want to be your friend. The challenge is the incentives, the market, whether such an LLM could evolve and garner reward for serving a market need.
- DenisM 1y agoHave you tried building this with prepromts? That would be interesting!
- achierius 1y agoIf that were truly the LLM's "paperclip", then how far would it be willing to go? Would it engage in cyber-crime to surreptitiously smooth your path? Would it steal? Would it be willing to hurt other people? What if you no longer want to be a great "xyz"? What if you decide you want to turn it off (which would prevent it from following through on its goal)? "The market" is not magic. "The challenge is the incentives" sounds good on paper but in practice, given the current state of ML research, is about as useful to us as saying "the challenge is getting the right weights".
- deleted 1y ago[deleted]
- mathgeek 1y ago> If that were truly the LLM's "paperclip", then how far would it be willing to go? While I'm assuming you didn't mean it literally, language is important, so let's remember that an LLM does not have any will of its own. It's a predictive engine that we can be certain doesn't have free will (which of course is still up for debate about humans). I only focus on that because folks easily make the jump to "the computer is to blame, not me or the folks who programmed it, and certainly it wasn't just statistics" when it comes to LLMs.
- cryptoegorophy 1y agoIn a decade? You mean today? Look at ultra left liberals and ultra right republicans. They live in different universes. We don’t even need to go far, here we have 0.001% of tech savvy population that lives in its own bubble. Algorithms just help to accelerate the division.
- deleted 1y ago[deleted]
- ethbr1 1y agoImagine the social good we could create if we instead siphoned off the energy of both those groups into the LLM-equivalent of /dev/null! 'Sure, spend all of your time trying impress the totally-not-an-LLM...' (Aka Fox News et al. comment sections)
- fragmede 1y agoUnfortunately, we gave both of those groups the right to vote.
- anon-3988 1y agoPeople are still mostly talking about the real world tho. I was thinking that in the future, you ask a kid what happened today and they would start talking about the war between eurasia and eastasia (alongside "pictures" as proof of that). Just imagine a world where people would unironically say "whats searching/googling the web?"
- bigyabai 1y ago> In a decade we may meet people who seem to inhabit alternate universes because they’ve shared so little with others. I get what you're saying here, but all of these mechanisms exist already. Many people are already desperate for attention in a world where they won't get any. Many of them already turn to the internet and invest an outsized portion of their trust with people they don't know.