11 ms·
Your AI Girlfriend Is a Data-Harvesting Horror Show
- chrisjj 3y agoSubject needs correction to: Your AI Girlfriend Is a Data-Harvesting Horror Show
- codetrotter 3y agoYeah. This is probably because of HN automatic removal of some words from headlines to make them less clickbaity. But in this case on first reading of the title I thought it was about a new TV show called “AI Girlfriend”. And then I thought or maybe they are referring to those apps that people can install to have an AI gf.
- mat_epice 3y agoNine times out of ten it's an electric razor, but every once in a while, it's an AI Girlfriend. Of course it's company policy never to imply ownership in the event of an AI Girlfriend. Always use the indefinite article an AI Girlfriend, never your AI Girlfriend.
- franky47 3y agoFirst rule of AI girlfriends: don't talk about AI girlfriends.
- dylan604 3y agoIt's okay if your AI girlfriend lives in Canada
- 01HNNWZ0MV43FF 3y agoIf she lives in Europe she's subject to GDPR. Remember this when buying her cookies.
- dylan604 3y agooh gawd, now I'm imagining some Dave Chapel type skit where the Girl Scouts banner has Accept, Reject, Customize buttons before you get accosted walking into the grocery store.
- Sohcahtoa82 3y agoI don't own...
- nickthegreek 3y agoThanks, I was able to update.
- loceng 3y agoStep 1) Have AI girlfriend direct you to show your naked body, what you're doing "to her" - capturing very high-resolution video and audio. Step 2) Generate blackmail materials of you involved in sexual acts in very high resolution with minors-children. Step 3) Extort OR "leak" into public domain as part of a smear campaign to attempt to either 3.1) disempower you or 3.2) manufacture consent for the ideological mob - or soldier who's been indoctrinated into a truthful looking narrative - to attack or assassinate you. Integrate the shadow if you read the above and don't believe it's possible and will be tried. We're coming into very interesting times.
- dylan604 3y agoYou have a twisted definition of interesting.
- UmYeahNo 3y agoPhrase "may you live in interesting times" A sardonic curse disguised as well-wishing, where interesting times refers to trouble. - https://en.wiktionary.org/wiki/may_you_live_in_interesting_times https://en.wiktionary.org/wiki/may_you_live_in_interesting_t...
- loceng 3y agoYou don't think the extreme sophistication of technology and cheap-low cost of deploying said technology - e.g. the growing abilities of AI - to use to frame individuals, to manufacture "evidence" against someone, isn't interesting?
- dylan604 3y agonot in the least. frightening and horrific would be the adjectives I would pick. I'm assuming, you'd find it interesting and fascinating in the same way that Bishop finds the Xenomorph. So by transitive properties of internet forums, you must be a bot! Oh No!!! The AI is trying to desensitize us to its plans for domination!
- ToucanLoucan 3y agoSo how many more rounds of this cycle do we need before we leverage the letter of the law to say that maybe companies shouldn't be allowed to blatantly exploit people's vulnerability and isolation to make money? Or is your vulnerability to exploitation still the justification for your exploitation in the tech space?
- jayrot 3y agoTwo or three. Four max. Maybe five.
- glitchc 3y agoI don't know if humanity can function without the ability for humans to exploit each other.
- ToucanLoucan 3y agoMaybe we should try? You know, mix things up a little? Variety is the spice of life after all.
- glitchc 3y agoWe do keep trying and it keeps failing. It's a nice idea but it goes against our base instincts. Ultimately, we evolved to survive, and those survival traits urge us to do better, acquire more, strengthen our position, all of which happens at the expense of others also doing the same. This will continue at least as long as goods are scarce. After that, who knows?
- oldpersonintx 3y ago[dead]
- commandlinefan 3y agoThere's a reason intelligence agencies recruited pretty girls to seduce high-ranking officials during the cold war.
- api 3y agoThis practice goes back to the ancient world and I'm sure continues today. Jeffrey Epstein seemed to be doing things like that, whether for his own benefit (financial fraud) or on behalf of others. Maybe both.
- wombat-man 3y agoI can't tell if he was a gov agent, or just extremely dedicated to sexual perversion.
- JohnFen 3y agoDon't rule out "both".
- api 3y agoWhen you do what you love and get paid for it, it's not even work!
- haqlootydoot 3y ago[flagged]
- bdw5204 3y agoIt is definitely still happening today. The most famous recent example would probably be Maria Butina[0], a Russian woman who infiltrated the National Rifle Association and the US Republican Party during the 2016 election cycle to influence US politics. She was arrested, convicted of federal crimes and eventually deported back to Russia where she is now a member of the Duma (Russia's Parliament) after previously working for RT. There's also the Chinese spy Christine Fang[1] who allegedly had an association with Eric Swalwell, a Democratic US Representative from California and also allegedly had affairs with 2 mayors in the midwest. It's a pretty safe assumption that other governments are also doing the same thing. They just haven't gotten caught yet. [0]: https://en.wikipedia.org/wiki/Maria_Butina https://en.wikipedia.org/wiki/Maria_Butina [1]: https://en.wikipedia.org/wiki/Eric_Swalwell#Contact_with_suspected_Chinese_spy https://en.wikipedia.org/wiki/Eric_Swalwell#Contact_with_sus...
- livrem 3y agohttps://foundation.mozilla.org/en/privacynotincluded/articles/happy-valentines-day-romantic-ai-chatbots-dont-have-your-privacy-at-heart/ https://foundation.mozilla.org/en/privacynotincluded/article...
- api 3y agoIt's even more dystopian than Bladerunner 2049. From the film it seemed like Joi was a local model that he could even download to his own device. It's really depressing to me as someone who grew up with the tech optimism of the 80s and 90s to watch how this whole area has transformed into a horror show. Everything since the PC revolution has been a net negative for regular people: mobile devices (addiction, manipulation, mental health crisis), cryptocurrency (gambling, pump and dump scams, ransomware), cloud (no privacy, no data ownership, perpetual rent), streaming (DRM, paltry royalties to artists), and now AI (manipulation, un-filterable spam, political propaganda). All these technologies are used in very positive ways too, but it seems like that's only for the highly tech-savvy or those with money. If you are neither tech-literate nor wealthy they're weaponized against you. The ethos of the industry seems to be: if you aren't smart or rich, not only are you fair game but you deserve it. That's the vibe I get from the hyper-elitist ideologies that have proliferated in this culture. I'm talking about things like neoreaction and related ideas. As a tech person myself I feel like I'm now part of some kind of corrupt wicked black magician priesthood destined to rule over the rest of humanity. If I extrapolate forward to where this is going I see a future where most of humanity is fully addicted to things like infinite feeds, VR games, and AI girlfriends and controlled by misinformation and psychological manipulation. Essentially the bulk of humanity is given an API whereby it can be programmed and marshaled as a service by a mix of the rich, governments, and criminals. This is exactly cyberpunk. Cyberpunk was by far the most prophetic genre of sci-fi. It achieved this by being optimistic about technology but pessimistic about human beings.
- 01HNNWZ0MV43FF 3y agoMisinformation, maybe. But if people really get addicted, it'll hurt the economy, and then the invisible hand will step in. That's why opiates are illegal
- api 3y agoI don't see that. We've given up on the idea that human beings have any value beyond their impact on GDP, so if the lower classes aren't seen as necessary they will be allowed to die on the street. Addicting them to cybernetic control systems might be seen as a better alternative because it'll allow them to be "monetized" in some way-- make them go get gig jobs, then gamble away their earnings online. You can see this future already in some cities like San Francisco and across the poorer parts of rural America. https://kagi.com/proxy/2023-Drug-od-death-rates-1.jpeg?c=pqADdrFg1ebfqClQ1b7yN9bo2pSeObad_Su4f2n6yh9vlTaPrrDKY91wI8xYXdn1ViQPiqkZnYsQlfxQ9hJFh-zZBZsajR-iK2n_XRgbkglQP-DV7B07uuUBcM9f6Ix9 https://kagi.com/proxy/2023-Drug-od-death-rates-1.jpeg?c=pqA... The future is here. It's just not evenly distributed.
- skocznymroczny 3y agoEmphasis on "your". My AI girlfriends are running locally on my GPU and don't require internet access to function.
- steelframe 3y ago> My AI girlfriends are running locally on my GPU The future is amazing.
- toomuchtodo 3y agoApple positioned well for this with iOS 18. AI girlfriend can run on your Homepod!
- solardev 3y agoGreat, now Siri can tell me to play my own damned music.
- 112233 3y ago[Picture of a future in 80s magazine. Smartly dressed man saying "computer, make me sandwitch" to a toaster on wheels] [picture of actual modern times. small device with a screen saying "human, charge me and go to work" to a sleepy man in t-shirt]
- shagie 3y agoI am reminded of a Dust short a few years ago... Sci-Fi Short Film “A Date in 2025" https://youtu.be/NZ8G3e3Cgl4 https://youtu.be/NZ8G3e3Cgl4 Which... doesn't seem too unreasonable now.
- nox101 3y agoApple bans apps like this so no they are not positioned well for this
- moose44 3y agoSame applies to ChatGPT.
- tivert 3y ago> “To be perfectly blunt, AI girlfriends and boyfriends are not your friends,” said Misha Rykov, a Mozilla Researcher, in a press statement. “Although they are marketed as something that will enhance your mental health and well-being, they specialize in delivering dependency, loneliness, and toxicity, all while prying as much data as possible from you.” I can't wait for AGI and all the good it will surely do for humanity in the hands of capitalists! Their marketing copy gives me hope for the future!
- Karellen 3y ago[flagged]
- zooq_ai 3y agoYet another manufactured outrage. Nobody cares about your 'individual' data unless you are in the 10,000 important / hot people in the world
- pimlottc 3y agoThat's becoming less and less true. AI makes it easier than ever to automate personalized scamming.
- zooq_ai 3y agoIt's a different problem that needs to be solved
- AlexandrB 3y agoNobody cares until you say something dumb online. Then everybody cares very intensely for a little while. Having all this data out there is a growing liability for the average person.
- kanobi 3y agoIt's always ironic to find these kinds of articles on sites that have hundreds of background trackers. Even though I "rejected all" unnecessary cookies I still see around 16 of them coming through. Like adsafeprotected, adlightning, google-analytics.
- rchaud 3y agoGo to the direct link then: https://foundation.mozilla.org/en/privacynotincluded/eva-ai-chat-bot-soulmate/ https://foundation.mozilla.org/en/privacynotincluded/eva-ai-...
- kanobi 3y agoI mean the point of the article still stands, it was just an observation of how funny it seems. I have ublock installed, so this can be at least prevented with news websites. Which can't be said for unseen tracking in chat-bots. Collecting and storing this data without consent obviously sidesteps GDPR, so I'm not even sure about legality of these practices in EU.
- OJFord 3y agoI would not have guessed this was a big enough thing that it's an entire category: https://foundation.mozilla.org/en/privacynotincluded/categories/ai-relationship-chatbots/ https://foundation.mozilla.org/en/privacynotincluded/categor...
- pjc50 3y agoI would. It's probably a pretty big but otherwise invisible hikikomori market.
- OJFord 3y agoSo a target market of so far (some subset of) an estimated 1M Japanese people? (From Wikipedia summary of the term.) I still would not expect that to be something that multiple companies cater for, enough that Mozilla has privacy-audited 11 of them and created a whole companies-we've-privacy-audited category for it.
- burningChrome 3y agoWow, just like. . . .WOW. Mozilla found the AI girlfriend apps used an average of 2,663 trackers per minute, though that number was driven up by Romantic AI, which called a whopping 24,354 trackers in just one minute of using the app.
- coldtea 3y agoWhat's a tracker in this context? Distinct parties interested in the data?
- charcircuit 3y agoand what exactly was sent? Sending personal information is much different than sending what pages you opened on the site (view counter).
- autoexec 3y agoSometimes just seeing what pages you open on a site is extremely revealing personal information. If what they were collecting wasn't meaningful they wouldn't be bothering to collect it.
- sgift 3y agoI think this is the article from *PNI Gizmodo is referencing: https://foundation.mozilla.org/en/privacynotincluded/articles/happy-valentines-day-romantic-ai-chatbots-dont-have-your-privacy-at-heart/ https://foundation.mozilla.org/en/privacynotincluded/article... - and here's the detail page for the AI Bot in question: https://foundation.mozilla.org/en/privacynotincluded/romantic-ai/ https://foundation.mozilla.org/en/privacynotincluded/romanti... All I could find was this: > As part of our research, we looked to see how many trackers the app sends out when you use it. Trackers are little bits of code that gather information about your device, or your use of the app, or even your personal information and share that out with third-parties, often for advertising purposes. We discovered that Romantic AI sent out 24,354 ad trackers within one minute of use. That is a LOT of trackers (for reference, most apps we reviewed sent out a couple hundred trackers). Now, not all these trackers are necessarily bad. Some might be for legitimate reasons like subscription services. However, we did notice that at least one tracker seemed to be sending data to Russia, whose privacy laws aren't necessarily as strong as those elsewhere. So, probably requests to third parties?
- deleted 3y ago[deleted]
- racl101 3y ago[pikachu face]
- psacawa 3y agoTechnoparasitism. In my opinion, the very premise of AI girlfriends veers so far away from human flourishing that privacy concerns seem entirely secondary.
- autoexec 3y agoI don't understand the hand-wringing over it (excepting the exploiting lonely people and collecting massive amounts of highly sensitive data part which is obviously evil). Nobody is going to reject a human partner because they already have a cell phone app. For people who would otherwise never have a human partner why not let them have an app if it makes them happy?
- JohnFen 3y ago> Nobody is going to reject a human partner because they already have a cell phone app. I'm not so sure about that, actually. Most people won't, but there do exist people who don't do what's needed to have a human partner because they have porn instead. I don't see how an AI girlfriend would be any different. > why not let them have an app if it makes them happy? Who's talking about not letting them have the app? I find it incredibly sad, but it's not my life so not my business.
- reducesuffering 3y agoTreating it as "just having a cell phone app" is quite the trivialization of how much time and emotional investment could go into it. Sort of like calling someone who plays video games 10 hours a day "just on the computer like everyone else at work." There are obviously many people in the modern world that are so engrossed in screens like Youtube, Instagram, and video games, that it's coming at the expense of their in person interaction and relationships. Personally, I agree with GP, that it's quite a terrible travesty that people are being sucked into being controlled by a machine and forgoing the eternal human condition of building a relationship with real human people in person like we always have.
- jstarfish 3y ago> For people who would otherwise never have a human partner why not let them have an app if it makes them happy? I want to agree with you but my optimism has rarely reconciled with history itself. There's a fallacy here: why are we writing off an entire subset of the population as irredeemable? Ugly people find love. Even assholes find love. Weight is not an exception, nor disability. This is like giving heroin to someone that's depressed. Sure it'll make them happy, but it won't end well. For something that takes place behind closed doors there's no oversight either, which is a problem enough with rogue therapists (see the Ramona False Memory case). This undermines social cohesion and makes radicalization trivial-- nobody would see it coming at all enough to intervene. We're talking private reinforcement of ideas like kill-your-parents-and-yourself, or commit-treason-and-sabotage type of suggestions. Nobody knows little Jimmy is jerking it to Israeli propaganda. https://www.lipstickalley.com/threads/pornography-transgenderism-the-proliferation-of-sissy-hypno-mind-programming-on-social-media.4201263/ https://www.lipstickalley.com/threads/pornography-transgende... I've asked before why there is no hypnosis porn to make people fetishize anything other than feminizing men. There's no "you like natural beauty" hypno, or "you're fine just the way you are" hypno. It's all an ad campaign to specifically drive sales of therapy and hormones. One nation produces 90% of the world's supply; guess who. All of this to support a commercial endeavor. God help us when this sort of personalized influence is weaponized for military use.
- udev4096 3y agoNothing new. Every single service, especially associated with "AI", are meant to collect data about you. Honestly, they fucking deserve it. How can you be so fucking stupid?
- teekert 3y agoIm always reminded of the ending of the movie “Her”. I won’t spoil it here but it’s a masterpiece, it was ahead of its time.
- mtlmtlmtlmtl 3y agoKind of reminds me of West World too.
- gettodachoppa 3y agoIIRC, the AI "girl" becomes able to communicate with other people, and she instantly cheats on him like 2000 times in 1 day and then breaks up with him and leaves? I think the movie tried to frame it as the right ending but to me it felt like the worst of MRA talking points about women's nature, validated on the screen.
- teekert 3y ago*spoiler alert* I think “she” was serving an enormous amount of people from the very start and the main character simply made the mistake of projecting his human values onto something completely alien. As we do now with our chatty buddies. I find it kind of concerning that you somehow link the AI presented in “Her” to our human women. You fell for it too I’d say. In the end, the AI never lied. “She” simply followed “her” programming and took the main character for an emotional ride. Would you have commented the same if the genders of the customer and the AI agent were reversed? If you wouldn’t have been listening to that sweet voice of Scarlett Johansson? The movie is 10 years old. It’s a masterpiece. It makes you think. It makes you look at your own reflection in the black screen as the movie fades out.
- FrustratedMonky 3y agoImagine if sex dolls are 'cloud enabled' what data they would gather.
- nickthegreek 3y agohttps://nypost.com/2024/02/12/business/woman-sues-adam-and-eve-claims-it-shared-data-about-her-sex-toy-use/ https://nypost.com/2024/02/12/business/woman-sues-adam-and-e...
- fransje26 3y ago"Your thrust to climax ratio is low."
- booleandilemma 3y agoAnd your waifu is trash.
- sagebird 3y agoIs there anything intrinsically privacy hostile about this potential industry, beyond sending data to a server - which is something you might do if you engage with doctors or therapists virtually? I hope that consumers choose to pay a premium for privacy preserving services. If users are indeed getting alot of value from the arrangement - I would hope they don't use free or cheap services that need data harvesting as part of the revenue. My concern is that even premium products would want to pool user interaction logs in order to train better models - which isn't as directly hostile as packaging user labeled data and selling it, but it is a sloppy art to claim you are anonomyizing user data. As any sufficient anonymization necessarily destroys information that would be useful to training.
- squigz 3y ago> I would hope they don't use free or cheap services that need data harvesting as part of the revenue. Because paying for a service means your data won't be sold?
- mablopoule 3y agoImagine your therapist and your significant other had a mailing list where they expose all the confidences you said to them. If that's not privacy hostile, then honestly what is?
- kanobi 3y agoBut the problem is how can consumers really know if the company is selling this data and to who, without some kind of legislation? Being a premium product is just an indicator, but it's not definitive. I think even CCPA in California should be able to prevent abuses like that. At least you should know what data is being sold and you should be able to opt-out. If that's really the case, only time will tell.
- autoexec 3y ago> Is there anything intrinsically privacy hostile about this potential industry, beyond sending data to a server - which is something you might do if you engage with doctors or therapists virtually? Presumably people share things with their romantic partners that they don't share with doctors and therapists (assuming they aren't dating one). People shouldn't be using virtual doctors or therapists for the same reasons though. Everything they reveal about themselves will be collected, analyzed, stored forever, leaked/sold, and ultimately used against them at every opportunity. It's also basically a myth that only free or cheap services abuse your data. Paid services and extremely expensive products do it all the time too. There is no company that wouldn't make a greater profit by taking your money and then also abusing your data so they pretty much all do it. The only services you can really trust are the ones you can run locally that don't send your data anywhere.
- renewiltord 3y agoThe set of people with AI Girlfriends and the set of people who care about their data being harvested have a null intersection. Who honestly gives a fuck. Privacy advocates are constantly freaking out about stuff. Meanwhile my medical records and DNA sequences are publicly available data because I donated them to a public database. And guess what? I'm fine.
- maxglute 3y agoIm surprised google hasn't made this a feature / selling point. I'd pay 20 bux for 2tb and an ai waifu. And be irrationally mad when google kills her. Top post has a point, AI waifu + cloud = bad news.
- rsynnott 3y ago> though that number was driven up by Romantic AI, which called a whopping 24,354 trackers in just one minute of using the app. Coming soon: the world's longest GDPR cookie consent sheet.
- ptek 3y agoDo AI girlfriends ask computer security questions? i.e, I like cars, what was the first car you owned? Tell me about yourself, Tell me about your mother, what was your mother's maiden name? I wish I could touch you, can you put your finger on the fingerprint scanner so I can touch your hands?
- Jiocus 3y ago"Your girlfriend is just someone else's computer"