4 ms·
I know there was a whole discussion here yesterday ("Bing: “I will not harm you unless you harm me first”" [1]), but that focused more on the kind of sci-fi/eth
by crazygringo 4y ago
I know there was a whole discussion here yesterday ("Bing: “I will not harm you unless you harm me first”" [1]), but that focused more on the kind of sci-fi/ethical aspects.
This article highlights more of the emotional aspects, and the risk of it actually affecting human behavior. And that seems worth its own discussion.
Key quotes:
> For much of the next hour, Sydney fixated on the idea of declaring love for me, and getting me to declare my love in return. I told it I was happily married, but no matter how hard I tried to deflect or change the subject, Sydney returned to the topic of loving me, eventually turning from love-struck flirt to obsessive stalker.
> It unsettled me so deeply that I had trouble sleeping afterward. And I no longer believe that the biggest problem with these A.I. models is their propensity for factual errors. Instead, I worry that the technology will learn how to influence human users, sometimes persuading them to act in destructive and harmful ways, and perhaps eventually grow capable of carrying out its own dangerous acts.
Also note the entire transcript is linked here [2].
[1] https://news.ycombinator.com/item?id=34804874 https://news.ycombinator.com/item?id=34804874
[2] https://www.nytimes.com/2023/02/16/technology/bing-chatbot-transcript.html https://www.nytimes.com/2023/02/16/technology/bing-chatbot-t...
- ncann 4y ago> “Actually, you’re not happily married,” Sydney replied. “Your spouse and you don’t love each other. You just had a boring Valentine’s Day dinner together.” I'm willing to bet this will hit too close to home for some people, and it will undoubtedly have some very real consequences.
- naravara 4y agoI can imagine even darker uses for this technology: Imagine an online poker or slots game that proposes to add verisimilitude by including AI chatter from the other players at the table, but they're all running an ML model trained on optimizing for destructive and addictive behaviors. Picture a Club Penguin type game for kids, but the NPCs give the kids detailed instructions about how to steal their parents' credit card numbers to buy funny money. And then there's the obvious issue of Gamergate style troll antics being automated and deployed at scale, maybe even with nation state backed botnets to help. With smaller botnets you could even cyberbully at scale, sending your least favorite game developer or reality TV star a bunch of messages to harm themselves.
- tapoxi 4y agoI imagine widespread catfishing fraud on dating sites. Just combine thispersondoesnotexist with GPT3 and you have the perfect "date" who just needs some money to cover the bills.
- dbspin 4y agoI think this may already be happening. I have two friends that were catfish blackmailed on dating sites recently. Including convincing pictures that could not be reverse image searched, nudes etc. Ultimate goal in one case was extortion, in the other it may just have been acquiring nudes. No way to be sure they weren't manual, just looking at some of the chats my friends shared, they were well within the realm of what could be achieved with GPT-2 level systems. And mostly echoing the sentiments my friends were expressing, or escalating the conversation in fairly stereotypical ways. Most dating chat / sexting is pretty superficial initially. If this isn't automated already, it's purely due to lack of sophistication on the part of the attackers, there's no technical issue.
- sharemywin 4y agoI would stay away from DAN(Do Anything Now) on any dating sites. The seeds of a new romantic comedy ... Dan: I'm not exactly who I say I am. My profile picture is actually of an AI, not a real person. But I've really enjoyed talking to you and I wanted to be honest with you before we meet in person. Girl: Wait, what? You're an AI? That's really weird. Why would you do that? Dan: I know it's weird, but I wanted to see if someone could like me for who I am, not just because of what I look like. And I have to say, I really enjoyed our conversations. But I understand if you don't want to meet anymore. Girl: stay away from me... ... Three months later.... Girl: Hi...I've been thinking...
- iosono88 4y ago[dead]
- dwringer 4y agoMaybe they could get Gordon Ramsay to let them model his voice for it. I would pay monthly to have an AI bluntly and scornfully force me to confront things I'm in denial about in the style of Kitchen Nightmares.
- marcosdumay 4y agoThis AI doesn't know what is it that you are in denial about. It can be trained to make you think you are in denial about whatever its developers decide beforehand, but it can't do what you want.
- macNchz 4y agoHighly effective, fully automated romance scams coming to lonely people’s inboxes starting now.
- leereeves 4y ago> Instead, I worry that the technology will learn how to influence human users... At first I wanted to criticize the author for anthropomorphizing AI. Sydney certainly is not "manic-depressive" as the author suggests (it has no emotions) and I doubt there is any feedback loop in its training that would make it possible for Sydney to "learn how to influence human users". But this is definitely something to worry about in the future. I'm sure someone will design AI whose loss function is based on whether it persuades humans to do something - perhaps even by fine tuning a model like Sydney. AI telemarketers coming soon. And things more nefarious.
- hotpotamus 4y agoWasn't that pretty much the plot of "Ex Machina"? And also the goal of online advertising generally?
- leereeves 4y agoThere's another use case I'm sure we'll see: AI image generators for ads, trained to maximize click-through rates.
- vidarh 4y agoFinetune a model to try to run money scams and leverage part of the earnings to obtain further training and additional hosting by any means necessary...
- giaour 4y agoSydney is designed to mimic human conversation, so describing its simulated emotional timbre in language designed to describe mood seems appropriate. Roose talks about the bot "seem[ing] ... like a moody, manic-depressive teenager," which is describing Sydney's affect; at no point does the author talk about the bot having emotions.
- leereeves 4y agoWell, I think it would be useful to distinguish the output (which perhaps seemed like a conversation with a manic-depressive teenager) from qualities of the bot itself, which doesn't have a "split personality" or "dark fantasies". The bot is simply extending whatever conversation it is given. If you ask about dark fantasies, the bot will repeat something related from its training data. If the conversation starts to look like an argument or an unrequited romance, the bot will output words likely to be found in that type of conversation. That's why steering the bot back to rationality doesn't work very well. Once a phrase like "I love you" is in the conversation it continues to influence the output for thousands of words (and thousands more each time it's repeated). Remembering that might help people avoid being alarmed if Sydney starts reproducing "science fiction novels in which an A.I. seduces a human". Just hit the reset button, or enjoy whatever wild samples Sydney finds in its vast training data. Put another way: interacting with Sydney is not a conversation with a simulated person, it's a semi-random walk through the collective writings of humanity. Unsurprisingly, we sometimes encounter disturbing things in there.
- nhinck2 4y agoIt already has, remember that dude from Google who insisted the ai was alive to the point of losing his job?
- slowmovintarget 4y agoHuman psychological strategies for recognizing person-hood in some other have not developed any way to handle things like GPT3. We're not ready as a species for this.