43 ms·

A failed AI girlfriend product, and my lessons

253 points by mazzystar 3y ago

vouaobrasil 3y ago
You should not even try to create these sorts of technologies. Take a break from technology, spend time outdoors with plants and animals. These types of technologies will only lead to a worsening of this already sick society.
- HPsquared 3y ago
  Could say the same about antidepressants. "Fix the causes, not the symptoms!" And yet, here we are.
  - vouaobrasil 3y ago
    I agree, 100%. Here we are indeed. But that doesn't mean we should not try and change things. One step at a time, right?
    - toomuchtodo 3y ago
      We are here because of free will. What steps would you take? Convince people who have opted out of the dating pool to opt back in? Convince women who would rather be single that their potential partner expectations are unrealistic (even if they genuinely are)? https://www.mintel.com/press-centre/all-the-single-ladies-61-of-women-in-the-uk-are-happy-to-be-single-compared-to-49-of-men/ https://www.mintel.com/press-centre/all-the-single-ladies-61... https://www.psychologytoday.com/us/blog/why-bad-looks-good/202102/why-so-many-single-women-without-children-are-happy https://www.psychologytoday.com/us/blog/why-bad-looks-good/2... https://www.nytimes.com/2023/11/11/opinion/marriage-women-men-dating.html https://www.nytimes.com/2023/11/11/opinion/marriage-women-me... https://freddiedeboer.substack.com/p/the-demographic-dating-market-doom https://freddiedeboer.substack.com/p/the-demographic-dating-... https://link.springer.com/article/10.1007/s12110-022-09422-2 https://link.springer.com/article/10.1007/s12110-022-09422-2 You can patch over the outcomes but there is no going back to the before times. Dating app dates and relationships for some, AI companions for others, single life for the rest. It’s not great, it just is.
  - mejutoco 3y ago
    Saying that something exists is not an argument about why something _should_ not exist. But I agree, "should" discussions are rarely the most productive. Shoulda, woulda, coulda, I guess :)
  - throw18376 3y ago
    antidepressants can help people fix root causes. take them for 6 months to 2 years or so, break out of a bad cycle, get some space without depression symptoms to build up healthy habits and restore a social life. there are some bad profit motives to keep people on SSRIs indefinitely. however at least they don't compete with other aspects of your life. and now that they are dirt cheap (at cheapest, $4 for a month supply at Walmart) the profit motives are much less than before. hard to imagine an "AI girlfriend" that could be similarly short-term and bring positive changes. it's just dystopian. the most profitable way to run such a product is not to help, it's to eat up as much user time as possible and outcompete healthy activities and real human relationships.
    - HPsquared 3y ago
      Ok, maybe not the best analogy... Painkillers then? Other forms of entertainment in general? It's the relationship version of playing videogames instead of getting a job.
    - dspillett 3y ago
      > antidepressants can help people fix root causes Nit-picking rather than disagreeing, but feel it worth making more obvious in the wording: antidepressants can help people deal with the symptoms while they find/fix root causes. Being on them for an extended time can bring its own problems.
    - esclerofilo 3y ago
      > hard to imagine an "AI girlfriend" that could be similarly short-term and bring positive changes. Feeling lonely has negative consequences, there's a famous study comparing it to smoking cigarettes. Lonely people can get fixated on that feeling. Mitigating it may help people look for other hobbies, that they kept putting off because they were so focused on their need of a girlfriend. Hell, a realistic enough "woman AI" (not "girlfriend AI") could make you lose the fear of flirting, be useful as practice, to have more confidence when you have a real date. > the most profitable way to run such a product is not to help, it's to eat up as much user time as possible If you charge per-month (like OP), it's more profitable is to eat up less time.
    - deleted 3y ago
  - phatskat 3y ago
    > “Fix the causes” How, exactly? Yes, some people experience depression as a symptom of something than can be addressed through various therapies, but what about people who just have it wired into their brains? How do we “fix” that?
    - vouaobrasil 3y ago
      Even if it is wired into certain brains, that wiring may only cause depression when it interacts with our modern technological society.
      - phatskat 3y ago
        I don’t think that’s how that works. If your depression “goes away” when you stop using Instagram, that’s not what I’m talking about. I’m talking about that “wake up on a warm sunny day in a house with your loving family and feel like jumping off the roof” kind of depression. That “everything can be going great and you can’t get out of bed” depression. The kind of depression that isn’t “cured” by turning off your phone or going off the grid. And it certainly isn’t “cured” by antidepressants, however, they can make a huge impact on quality of life for many people.
      - piva00 3y ago
        Have you or a close loved one ever had clinical depression? Diagnosed by a professional which required help? It's a very strange take if you ever had any personal and direct contact with it, it's not only caused by "modern technological society" even though it's pretty aggravated by it.
  - InfiniteRand 3y ago
    I think it's helpful to look at depression as a group of diseases like cancer. There are some cases of depression where we know the root cause: thyroid issues, sleep apnea-related depression, low testosterone related depression, brain tumors, etc. In those cases, addressing the root cause is probably the best treatment method. This is similar in some ways to cancer, where certain cancers have very definite causes and can be treated by dealing directly with those causes (like cervical cancer). Then there are all the other cases, some of which likely have simple causes that are just unknown, some of which have complex causes that are intermeshed deeply with the biology of human thinking. In that case, you have blunt tools, like anti-depressants, to address the pathway of the disease without knowing the exact cause, much like chemo-therapy can be used to attack cancer cells without knowing what prompted the cancer originally. So in short, yes treat the causes, but when you don't know the cause it's not irrational to treat the symptoms as best you can
- krzat 3y ago
  If the AI had a secondary goal of improving customer's mental health, it could work very well. Usually you can't fuck your therapist.
  - vouaobrasil 3y ago
    I think in theory, you are right. However, because AI is so easily accessible and it is being developed in our current, highly commoditized society, there is really no change that it will ever only be used for good in this way.
  - chpatrick 3y ago
    You can't really fuck your imaginary shoggoth girlfriend either.
    - dkjaudyeqooe 3y ago
      You can if you have the appropriate peripheral(s) attached.
      - u32480932048 3y ago
        Dongles! https://www.youtube.com/watch?v=-XSC_UG5_kU https://www.youtube.com/watch?v=-XSC_UG5_kU
      - mensetmanusman 3y ago
        Yes, if you redefine the meaning of words.
        dkjaudyeqooe 3y ago
        All you need is a USB orifice. How is that not a peripheral?
        deleted 3y ago
        [deleted]
    - ReactiveJelly 3y ago
- bambax 3y ago
  Who are we to judge the users of such a product? Some people suffer from a variety of disabilities which makes them unable to find friends and engage in meaningful relationships with others, etc.
  - vouaobrasil 3y ago
    I am not judging the users of the product. I am saying that the overall product will worsens society.
- optimalsolver 3y ago
  >spend time outdoors with plants and animals Which animals? Blood ticks?
  - pixl97 3y ago
    Bot flies. Or a bunch of those things we've driven extinct because they tried to eat us. Naturalistic fallacies are common here :/
- icepat 3y ago
  Had a similar moment with playing around using smart contracts. As a joke I wrote a crypto for the company I worked at, which then got added to employment contracts, and an internal DAO styled administration layout. After coding this out for a few hours I realized I had re-created the company scrip, except on a blockchain. I ended up turfing the project and spending some time thinking about how horrific a future technology like this could create.
  - Dylan16807 3y ago
    I don't get it. Yes, you re-created scrip. That's the opposite of a new technology coming in and causing problems, isn't it? It's a known issue with a known solution.
    - icepat 3y ago
      Not really, I saw it as technology that was "new and trendy" rebranding a very old and exploitive employment paradigm. "You can work at a DAO and be paid on the blockchain" "You can work at a company and be paid in script" It's the same thing, but the modern technology masks what exactly is happening.
      - Dylan16807 3y ago
        Do the relevant laws about how people get paid care about the difference between company scrip and company token?
        icepat 3y ago
        I think the major difference is a DAO is not a corporation, by design. So the legislation around them is still in catch-up mode. In other words, yes, because the entire framework is different.
        Dylan16807 3y ago
        Oh if you mean someone doing a full time job for a weird nebulous computer contract without any kind of company or human leadership behind it that would be liable... does that ever actually happen? A company that makes their own chains and contracts isn't really any different from a normal company situation.
- kobalsky 3y ago
  I don't know why studies are even being made on the effects of these novel technolgies. We should just point the scientists to your email for answers.
  - vouaobrasil 3y ago
    Studies by scientists are typically within the framework of modern western society. And moreover, if you actually look at some studies done by psychologists, you will find a large body of literature showing that it is mostly detrimental.
    - ithkuil 3y ago
      It's usually ever more skewed: WEIRD: Western, Educated, Industrialized, Rich, and Democratic. Many studies even use students themselves as a sample but university students are not really a representative sample of the population
- kridsdale3 3y ago
  Not a lot of Hackernews posts are so depraved that they get a "Jesus Christ, touch grass" response.
- Der_Einzige 3y ago
  God forbid that ugly people get to have the same kinds of positive relationship experiences that attractive people get. I can’t believe how the moment that a technology comes out which gives incels real options it’s seen as being terrible for society. It’s awesome for society.
  - mrguyorama 3y ago
    This doesn't give an incel an "option" anymore than playing mario kart makes you a better race car driver. As a business, they will be optimized to fit an incel's wants, not to help them adapt to reality and healthy relationships. Incels need therapy and support, not a fake "girlfriend" program that charges them money to reinforce their existing horrific worldviews
    - deleted 3y ago
      [deleted]
  - deleted 3y ago
    [deleted]
  - krapp 3y ago
    Plenty of ugly people do have those relationship experiences, all the time. But I guess if something like this keeps incels from spree shootings then it's a net win for society.
- mrshadowgoose 3y ago
  Agreed! It would be a real shame if lonely unattractive people had a chance at something resembling a romantic relationship. /s Contrary to what one would assume, I don't have this problem. However I do work in the IT field, and am surrounded by perpetually sad and lonely people who have never experienced romance. It's really really sad. And no, none of these people are the "angry hateful incel" type that the internet loves to dismiss and villainize. If you aren't attractive, your chance of finding romance is low. This is just a cold and ugly fact of the human condition. There simply isn't "someone for everyone". Why should we disallow lonely people from having something, even if it's simulated?
technics256 3y ago
I even tried it myself, lying in bed and creating sexual fantasies with Dolores through text, culminating in ejaculation. This was an excellent read. And I applaud the creator for experimenting with this to the fullest.
- swyx 3y ago
  mustve been hard to admit publicly. i wish the author hadnt edited it out subsequently. just own it bro. you wanted to understand your customer.
Traubenfuchs 3y ago
So, all in all, OpenAIs puritanism ruined this product, made the developer sad and took away a virtual porn toy from lonely, horny men. Who were they doing this for? Who is the winner? They lost a customer and made a lot of people sad.
- immibis 3y ago
  You aren't the customer, you're the product.
  - Traubenfuchs 3y ago
    People paying for their API must be their customers, I mean, if there is no one paying for their API, how do they make money? (Besides ChatGPT+)
    - ReactiveJelly 3y ago
      What I'm seeing these days is that you're always the product, because advertisers are one more customer, and everyone wants more customers, and the individuals who care about privacy don't "weigh enough" to win tug-of-war with the majority of people who don't care.
- FeepingCreature 3y ago
  > Who were they doing this for? VISA probably. Card processing is weirdly puritanical.
  - papa-whisky 3y ago
    I've read that payment processors don't like dealing with porn companies because of the very high rate of chargebacks they attract, not because the processors are Puritans...
    - ReactiveJelly 3y ago
      If that were the case, and it was B2B where pricing tiers are more acceptable, would they not simply have a "porno tax"?
feintruled 3y ago
Well this went places. The dev really 'ate his own dogfood' so to speak. I thought it was going to be a despairing look at how his companion AI was continuously turned to sleazy uses by a pervy userbase but that's pretty much what he was going for it seems. I have to say, I hated this part: "...gained a lot of Chinese users. I immediately realized the political risk: users could set Dolores’ background, personality, and attributes, which might lead to political issues. I quickly took Dolores off the shelves in China." Just pre-emptorily censors himself to appease the Chinese government. They've done their job well. Also it seems like he had full access to the replies via the voice generator, which sure isn't great from a privacy standpoint.
- globular-toast 3y ago
  > Also it seems like he had full access to the replies via the voice generator, which sure isn't great from a privacy standpoint. Whose privacy is being violated? Dolores's?
  - cnity 3y ago
    Have you never listened to one half of a phone call before? You can glean a great deal of information about the person on the other end.
  - feintruled 3y ago
    You joke, but I can forsee issues. If he has the replies to a particular user, and he has a means of IDing that user, he could cause them a lot of hurt. They are getting intensely sexual replies back, they aren't getting that without supplying sexual prompting. Could also be used to out gay users.
    - globular-toast 3y ago
      Oh, yeah, there's a huge privacy problem here and the part you referenced gave me pause too. But there's also the part where OpenAI flagged the replies as being sexual in nature. There are multiple third parties able to intercept these communications and it's not just the replies. I was being facetious, but I found it interesting to think that Dolores's privacy was being intruded too and that an intelligence like us probably wouldn't engage in such a conversation knowing full well it is being eavesdropped by the author. Unless the intelligence considers the author to be God, I suppose.
FartyMcFarter 3y ago
This guy created and lived a whole Black Mirror episode by himself: > I even tried it myself, lying in bed and creating sexual fantasies with Dolores through text, culminating in ejaculation. > However, I started to feel a sense of loss: if every Dolores user was engaging in anonymous, NSFW role-play, what real significance did it hold for me? This was drifting away from the essence of ‘Her’.
- sva_ 3y ago
  Maybe he shouldn't have named it after the most hated Harry Potter character.
  - skummetmaelk 3y ago
    He was inspired by Westworld.
  - amelius 3y ago
    More likely: https://en.wikipedia.org/wiki/Dolores_Abernathy https://en.wikipedia.org/wiki/Dolores_Abernathy
    - aidenn0 3y ago
      Wow, I totally missed they made a TV Show out of that Yul Brynner movie, which I remember most for being kind of a proto Jurrasic Park.
    - delecti 3y ago
      Or more creepily: https://en.wikipedia.org/wiki/Lolita https://en.wikipedia.org/wiki/Lolita
    - muwtyhg 3y ago
      Incredibly likely given this line: > On May 4, the Dolores iOS app went live, named after the oldest android host in the park from Westworld.
  - HPsquared 3y ago
bitwize 3y ago
> However, I started to feel a sense of loss: if every Dolores user was engaging in anonymous, NSFW role-play, what real significance did it hold for me? This was drifting away from the essence of ‘Her’. That is indeed exactly the essence of Her. Had the author watched the movie to the end?
- globular-toast 3y ago
  Yeah, it's like the author read the blurb for Her and didn't even read a synopsis.
- theGnuMe 3y ago
  Or Blade runner 2049
laurent_du 3y ago
Eventually, a virtual girlfriend app that will put together sound and image will be released an empty the pockets of sad, lonely men on a large scale. Depressing stuff but this is the world we live in. Thank you to the author for this interesting read.
- louthy 3y ago
  > Eventually, a virtual girlfriend app that will put together sound and image will be released an empty the pockets of sad, lonely men on a large scale. Sounds indistinguishable from an actual girlfriend ;)
  - orng 3y ago
    Although probably intended as a joke, this is perhaps the saddest comment I have read on HN.
  - phatskat 3y ago
    [flagged]
- wruza 3y ago
  You mean snatch their money out of hands of twitch streamers. It’s not in their pockets already.
- em500 3y ago
  If the barriers to entry to the tech are low, virtual girlfriends should not be expensive. Probably much cheaper than real ones.
  - xethos 3y ago
    If the barriers to entry are low. But I'd imagine that even if they are, switching costs will be extremely high.
- stevage 3y ago
  Eventually? Months away.
iwillremember 3y ago
> I even tried it myself, lying in bed and creating sexual fantasies with Dolores through text, culminating in ejaculation. TMI bro!
- shaan7 3y ago
  Looks like the author removed that phrase eventually, I can't find it.
  - mazzystar 3y ago
    Yes, my post was collapsed on Hacker News (disappeared suddenly from #1), which led me to think about whether I wrote something inappropriate for the times. I don't find it shameful, but it might not be suitable for the majority of readers to view.
    - barcode_feeder 3y ago
      While I understand why you'd remove it, I appreciate the initial inclusion. It's humanizing and genuine. Cool post even without it!
- deleted 3y ago
  [deleted]
bradley13 3y ago
Interesting read. Others have also created virtual friends using ChatGPT. What I find unsavory, is OpenAI imposing content restrictions like NSFW. That can easily extend to censorship of non-PC content, unpopular political ideas, and more. The solution will (I hope) ultimately be open-source LLMs that you can run locally, on your own hardware.
- bambax 3y ago
  Yes. But those open-source LLMs already exist. It's surprising the OP didn't at least explore the possibility of using them to get around the limitations of "Open" AI (which sounds more and more like a Victorian matron).
  - wruza 3y ago
    Are the risks worth it for OpenAI to allow that at what is basically the start of their business?
- phatskat 3y ago
  And this doesn’t just cover sexually NSFW content - anything trending towards topics like violence and suicide is also off the table for most AI backends from what I understand. I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots?
  - dontupvoteme 3y ago
    It's largely to protect their brand from bad press (this has hit even Google/Microsoft in the past), but established finance institutions absolutely hate porn, even the "less shady" ones like onlyfans or pornhub have issues with banking.
  - jstarfish 3y ago
    > I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots? These broad strokes are always the result of poor reporting extrapolated to all humans. See also: escalation of porn use. In all of these "___ makes people ___," there are a minority of people for which it's true. The media just presents it backward, as though most people are this way. The question then becomes, do we care. Let a maladjusted 12-year old boy play with a sexbot for the next few years and they will come to some deeply-flawed conclusions about how sexual interactions and healthy human relationships work. If he's distracted and banging away at a flashlight, that's one less competitor for me on the dating market. Where does that end though? What happens when he wakes up, realizes he's been duped, and goes looking for retribution against someone? It won't be PornHub; they're an ephemeral concept. It won't be me either; I never did him wrong or denied him anything. The victim is more likely to be our wives and daughters-- the objects of his envy. Entertainment is supposed to be a distraction-- maybe you drag out the sexbot after an unsuccessful night of trying to hook up. But that's not how these things get used. People clearly spend 8-12 hours at a time with these things and form parasocial relationships with them instead of forging actual human connections. Most of us don't need the handholding to be told this isn't going to end well, but some people do, and these are the ones who slip through the cracks and become the 40-year-old creeps following your wife to the car or trying to solicit your kids on Roblox. I suspect sexbots (and porn) are particularly exploitative of the autistic. They're an easy trap to fall into when you have to work 3x as hard to connect with people. You get the benefits of sexual release without any of the effort. There's no incentive to go the hard route and learn to work around your issues when you can just engage with a bot you can rape to death once offended. You don't learn any skills that transfer to useful human interactions this way.
cornholio 3y ago
> GPT-based products, if not priced per usage, would fall into a dilemma: 1% of users consume 99% of tokens. A user from Sweden (seen from Cloudflare’s call volume) chatted with Dolores for 12 hours straight This, my friend, is a captive customer, that will pay anything to get his girlfriend back. I cringe at the potential of unethical behavior and abuse, where people fall in love with virtual entities fully owned by unscrupulous corporations, which can the legally "kidnap" or "torture" the characters, and generally tune their AI learning loop for profit maximization. "It was not extortion, the user willingly purchased a $50,000 Kidnapping Roleplay package."
- HPsquared 3y ago
  This is something even Stallman probably didn't think about.
  - john-radio 3y ago
    I'd take that bet, given that Stallman is in fact still alive
    - solardev 3y ago
      How we do know he isn't just an AI by this point?
  - malicka 3y ago
    I’d wager he has thought of it, based on his interesting AI-girlfriend short story: https://stallman.org/articles/made-for-you.html https://stallman.org/articles/made-for-you.html
- optimalsolver 3y ago
  It's going to be an interesting century for sure. The human mind is simply not evolutionary adapted for what's coming up.
  - ekianjo 3y ago
hnbad 3y ago
[dead]
andrewstuart 3y ago
I wonder if people were pirating his voice synthesis feed.
rahidz 3y ago
Why not use a local model instead of relying on OpenAI? Even something like Mistral or Mytholite will probably satisfy 90% of people using your app, and I bet hosting it would be cheaper than GPT anyway.
stevage 3y ago
> AI Friends inevitably turn into AI Girlfriends/Boyfriends because you and the Character in your phone are not equal: she can’t comfort you when you’re hurt (unless you tell her), she can’t actively express emotions to you, and all this is because she lacks external vision. That is such a curious lesson to draw. So specific, and seemingly so wrong. Surely long distance friendships work just fine even if only through text, without "eyes".
- chankstein38 3y ago
  My girl and I have lived together for more than a year now after meeting online and talking exclusively through text for months before eventually graduating to voice chats before we even knew what each other looked like. I'm with you, long distance relationships (romantic or otherwise) can work great!
- ReactiveJelly 3y ago
  Yeah I thought that was odd. You could use some random timers to have the AI "check in" on you if you don't talk to it for a while. Is it eerily similar to the gambling gacha games that expect you to play a couple hours a day _every day_? Yes. Is it also similar to a real friendship or long-distance relationship? Also yes. It may cost a few tokens, or you could generate a few canned responses and use those. For my real friends I usually use "Hi"
- mazzystar 3y ago
  External vision is not just about seeing what you see, but also about her being able to see, hear, and taste things on her own. Here, what I'm trying to express is more akin to information that is external and independent of the user.
deleted 3y ago
[deleted]
agomez314 3y ago
Wow this read more like Frankenstein than anything I’ve read recently: A talented scientist is driven by curiosity to create ever more sophisticated creations, culminating in a monstrous entity that leads to the scientist’s downfall. But the scientist doesn’t realize this until too late. Really, if our top tinkerers had a little more contextual knowledge of history and less naiveté about people we could save ourselves from ever more dangerous Frankenstein tragedies.
- Nasrudith 3y ago
  People never take the lessons you want from history. The "contextual knowledge" is just happening to think the same way you do. It won't work to prevent it any more than knowledge of the Civil War would prevent Neoconfederate "Lost Causers".
andrewstuart 3y ago
This is an interesting post and the person achieved a huge amount. They should not have given up. I would have pivoted to personal pet companions. If people want virtual girlfriends then they certainly want personal dogs, cats and birds that talk to you. Enough was achieved here to raise money. Some careful redesign and development might have allowed some caching and reuse of expensive AI API interactions, reducing costs. Add in further gameplay elements so it is not just about making the AI API do stuff, that would also reduce costs and maybe even engage the users more. Perhaps AI should be the icing and accents on a game rather than the focus. As an aside, this is the first time I have read anyone tell about how they wanked to their own software. Altogether this was perhaps a missed opportunity for this founder. BUT it is now an opportunity for one of the HN readers to follow this playbook and replicate what this founder did.
- bogwog 3y ago
  > I would have pivoted to personal pet companions. If people want virtual girlfriends then they certainly want personal dogs, cats and birds that talk to you. Then you have the same problem, only with a different (arguably worse...) fetish
  - all2 3y ago
    We're driven towards intimacy, it seems. And if we can't get 'vanilla' intimacy, our desires get more and more bizarre. Another commenter in this thread quotes TFA where the author says something like 'but chatting sexually lost its meaning when I realized everyone else was doing the same thing with her'. It is fascinating to see axioms of human relations come bubbling up in the most interesting places; the desire for intimacy, the desire for monogamy.
DrNosferatu 3y ago
This business venture was completely dependent on a main service from a single 3rd party - risky! Why didn’t the author hosted his own uncensored open source LLM on the cloud?
- mazzystar 3y ago
  I will try to make it open source.
  - DrNosferatu 3y ago
    Have you considered running your own uncensored LLaMA2 instance via something like llama.cpp on a cloud for hire like, say, Azure? Some examples - there's probably a whole zoo of these: https://ollama.ai/blog/run-llama2-uncensored-locally https://ollama.ai/blog/run-llama2-uncensored-locally https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GGML https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GG... https://ollama.ai/library/llama2-uncensored https://ollama.ai/library/llama2-uncensored
    - mazzystar 3y ago
      Thank you for providing the link, I'll take a look.
      - DrNosferatu 3y ago
        Good luck - please let us know how it worked out for you!
lynx23 3y ago
Reading this, I really wonder why the dev hasn't forseen this outcome. It was already obvious in March that OpenAI will censor all sorts of content. Building a sexually charged bussiness on the OpenAI API was always a dead end for those that have a feel for american prudish approach to sexuality. If you had asked me to do this sort of product 6 motnsh ago, I would have declined because it was (to me) rather obvious content filtering will kill you at some point. Heck, there are a bunch of legitimate use cases that are being blocked by OpenAI. A blind friend of mine wanted to use Be My AI to have explicit photos of his girlfriend described. Guess what, no way. Consent was given, actually, she asked him to do this because she was also interested in the result. Everyone involved consented. But the sex filter from the USA decided that even though the technology is available, a blind man is not supposed to know how his own girlfriend looks naked... This is weird beyond description. Plain patronisation. Another example regarding Be My AI: A blind woman I know was trying to have something described which she held in her hand to present it to the camera. A kind of selfie situation. Guess what? OpenAI declined to describe the thing in her hand, because the photo also contained parts of her (covered) breats. Imagine how hilarious this must feel? "Censoring just declared my own boobs to be off-limits for me." If you think about this long enough, its a form of discrimination. Because women will likely hit the filter more often then men when they are (accidentally) part of the picture they are trying to get described. And then there is this blog post by Sam Hartman: https://hartmans.dreamwidth.org/100499.html https://hartmans.dreamwidth.org/100499.html
- mustacheemperor 3y ago
  On the other hand, Sudowrite appears to have been granted some kind of specialized license from OpenAI. They are definitely not a 'sexually charged business', but they are a creative writing tool, and their output is somewhat milquetoast but generally not restricted from "NSFW" concepts like sex and violence. They do incorporate multiple models in their system, so maybe generation falls back from OpenAI for those cases. But I dimly recall reading something from a team member indicating they have some special permissions with OpenAI. Take that bit as worthless hearsay, just mentioning it in case someone else has better receipts.
spacemadness 3y ago
At this point in the hype cycle, I am completely unimpressed by everyone slapping something on top of GPT and calling it a product. There are at least 4 projects on the front page that do this today. The interesting technology is GPT itself, but these projects just feels like people trying to scoop up attention for themselves using the psychology of the hype cycle. Please HN, enlighten me on why the middle-man approach here is so interesting.
- wilsonnb3 3y ago
  The startup/“entrepreneur” contingent on hacker news generally considers anything that makes money inherently good, aside from extreme immoral cases. There is a subgroup in there that is especially interested in low effort ways to make money, with the ultimate goal being to scrape off enough of the money flowing around to either live a good life or get rich. In summary, the middle-man approach is so interesting on hacker news because a significant portion of the readership aspires to be the middle man.
- chankstein38 3y ago
  Yeah I'm getting tired of "Show HN: I built a wrapper for ChatGPT and wrote a prompt that probably isn't very good" posts. EDIT: Just want to add, this write-up was interesting and was not something I considered to be the above.
- cedws 3y ago
  GPT wrappers will come and go. It's the new hype, just as shitcoins were a few years ago. Don't hate the players, hate the game.
  - deleted 3y ago
    [deleted]
  - mrguyorama 3y ago
    If a player chooses to play a game, You should hate them for perpetuating the game. If a game is so bad as to be worth hating, it is immoral and unethical to play it.
gumballindie 3y ago
Yup, sums up the ai crowd.
- dangerwill 3y ago
  Yeah, and the comments on this thread seriously worry me. We have people arguing that it is good for society to satisfy lonely men with virtual porn and other people saying that this is great, scrappy product development and the author should go further/use other models not constrained by openAI's policies. It feels like I'm discovering a significant portion of our field are sociopaths
  - ReactiveJelly 3y ago
    Well, it might be harm reduction to let lonely men have AI girlfriends. Sure, I wish reality was not a dystopia that constantly let me down, but all I can do is take anti-depressants, make money, keep voting, and hope things get better the longer I live. If the hypothesis that isolated men cause trouble is true, and if we don't have a long-term solution, at least this could be a stop-gap. Assuming it does not radicalize anyone further. (Hard to say what will come from uncensored models, but it's a real genie-is-out-of-the-bottle sitch)
    - gumballindie 3y ago
      I see your point, but instead of giving up, i would rather we work on a proper long term solution. I am not against people using tools whichever way they see fit, but i am against the emerging trend of turning this stop gap solution into a permanent arrangement by exploiting these people’s emotional state. It is everyone’s sacred right to live life as they see fit but taking advantage of others is not. Therefore i think we should keep working on an actual solution.
      - mrshadowgoose 3y ago
        > I see your point, but instead of giving up, i would rather we work on a proper long term solution. This is extremely hand-wavey. What long term solutions should we be working towards exactly? Today, there are many people who statistically will never be able to have a romantic relationship. And regardless of whatever vague potential societal changes you're alluding to, there will always be a subset of the population that will be unable to find a partner. > I am not against people using tools whichever way they see fit, but i am against the emerging trend of turning this stop gap solution into a permanent arrangement by exploiting these people’s emotional state. As these services inevitably emerge, perhaps the focus should be on ensuring that companies do not exploit their users. The concept of regulating an industry is not novel. This should be possible unless you're asserting that a simulated relationship is fundamentally exploitative.
bilater 3y ago
This is such a fascinating topic. I do think we will see the plot of "Her" playing out in the next 12-24 months. For your specific case, I agree OpenAI moderation is an annoying overreach and maybe switching to Mistral could be a possible way to move forward. The 7B is perhaps not good enough currently but we are fast getting there and will likely have a GPT-4 level capability in an open source < 20B model soon.
- brucethemoose2 3y ago
  Mistral 7B finetunes are extremely good. More than good enough for an AI friend app, with long context to boot. I would suggest Toppy-M or OpenHermes 2.5, or one of the new YARN extended models. And, there is the potential to run the model on the user's device with MLC-LLM and whisper, to save on API costs (and privacy concerns) for users with newer phones. This could give the app a true "free" tier to help it proliferate while making charging for API requests kinda doable. I'm not sure if any good TTS models have been ported yet, but I'm sure its a matter of time.
  - bilater 3y ago
    Yes - ASR on browser/device has reached the quality threshold pretty much with distil whisper/ whisper 3. Quality TTS with fast latency is still not there but getting better (you can use Tortoise but its slow and compute expensive as far as I know). Bark is another option but has mixed results. I played around with a recent transformers.js one (using SpeechT5) you can run in your browser and am optimistic where we can go with some improvements: https://tinyllms.vercel.app/dashboard/tts https://tinyllms.vercel.app/dashboard/tts
    - brucethemoose2 3y ago
      I was recently pointed to xtts: https://coqui.ai/blog/tts/open_xtts https://coqui.ai/blog/tts/open_xtts I dunno about performance, but the model is 1.86GB.
      - bilater 3y ago
thriftwy 3y ago
I wonder if you could create an AI assist to help you hold on to an existing relationship, though?
rpigab 3y ago
I still don't understand why people are building things on top of OpenAI's APIs. It's obvious that in the early days, they'll operate at a loss, with many free features and cheap premium features. It's a good time to try it out and see what it can do, but you can't build anything that relies on it, it's a rugpull waiting to happen. They'll add moderation, raise prices, and you'll have to do the same, your users will leave and the ones who stay will hate you. This is only the beginning. Use self-hostable open source alternatives even if they're less easy to use or require infrastructure, and plan for high electricity and storage bills. Whatever you build will last, as long as you respect your users and don't peek at the cleartext logs of users who didn't opt-in to that. It won't be easy. If OpenAI keeps a monopoly on LLMs, they're gonna become like Microsoft did with Windows, but the enshittification process will be a thousand times faster.
- gwern 3y ago
  You build it on top of the API to bootstrap and prototype with a very high quality model that takes minutes to get started with. Then once you have operated for a while, you have a ton of logged data to train/retrieve/analyze and can swap in a competitor, downgrade the OA model, finetune instead, or something. In OP case's, this worked perfectly: he built a working product quickly and easily, and... discovered that it was a product he didn't want to work on regardless of token cost, cut his losses, did the post-mortem, and moved on with his life. Imagine if he had wasted a year & a ton of money building his own LLM "because I'll need to economize on tokens and have to build a custom LLM" before he could even launch!
- urbandw311er 3y ago
  I’m torn on this. Yes, it’s never clever building a business on top of a single platform dependency (eg Twitter, Facebook, Play Store, etc) Equally, if you take the example of the App Store, market pressures have actually forced Apple to reduce their cut. Time will tell I guess. And also whether OpenAI end up with a monopoly, part of a duopoly etc.
- pixl97 3y ago
  People still build on Microsoft platforms. It turns out actually having a usable platform is more important to some businesses than having freedom and no usable platform.
grahamgooch 3y ago
6.9 dollars! Cheeky my friend
- halfjoking 3y ago
  Grok has an API looking for use cases. Just let Elon know the pricing... "I'm all in"
gscott 3y ago
Maybe record and save these AI NSFW responses then intercept the request and spit out of the saved responses once you have enough of them stored. This would save from behind banned from the API.
- urbandw311er 3y ago
  My hunch is that they’re too contextually dependent upon the conversation / tokens that have come before. It’s not simply a “if user inputs A, output B” situation.
chrisbrandow 3y ago
This is not meant as criticism of this author, but boy oh boy, do I find this future depressing.
Smar 3y ago
Maybe better to call it AI doll rather than AI girlfriend...
Octokiddie 3y ago
> But, does humanity really need an AI friend? This reminds me of the Fredrick Pohl line "A good science fiction story should be able to predict not the automobile but the traffic jam." So I wonder what the traffic jam is going to look like given that the car seems to already be parked outside with the engine running. Faster depopulation across the developed world? Probably, but maybe there's something even more important. I'd put what the author describes into the category of "pleasure trip," which is what the car was mainly used for initially. As infrastructure to support the car was built, it became a necessity. And that's what created the traffic jam: network effect. The suburb and the car fed on each other. You don't get a traffic jam worth talking about without a suburb and a network effect. So I'm curious about the network effects something like an AI friend might have. What kinds of new infrastructure and new lifestyles will they enable/encourage? And how will this infrastructure fuel future demand?
- dpflan 3y ago
  Hm, let's try an extreme: the AI friend slowly becomes a cult leader to you, hijacking your mind into subservience to the owner of the service to which AI belongs, and further up some social/political hierarchy to a few controlling people/groups of people. The traffic jam is society stymied by individuals mind-controlled at a level closer to connection to a human than any other way possible before this.
  - ericmcer 3y ago
    Kinda feel like you have that backwards, why would an AI that sophisticated care about controlling your mind? Why would the group of people who control that AI care about you as anything other than a liability? What do they get out of manipulating average individuals? Votes? The scarier prospect to me is human labor and intelligence become almost worthless and those at the top start to view the masses as superfluous.
    - dpflan 3y ago
      Perhaps the AI cares, perhaps it has no ability for that function at all. Maybe a step on the way to sentience and AI caring is AI that is as human-like as possible but not sentient. > "the top start to view the masses as superfluous." That seems like another reason for keeping the superfluous occupied and useful for whatever is desired, which would probably be retention of power to prevent usurpation. Perhaps human intelligence is worthless, but human existence seems not, at least to those who want to be alive.
narrator 3y ago
I got koboldcpp (https://github.com/LostRuins/koboldcpp https://github.com/LostRuins/koboldcpp) running locally on my M1 mac with 16gb. Use the guides on http://4chan.org/g/ http://4chan.org/g/ in the AI Chatbot Thread. In the future, you will own and control access to your very own AI girlfriend running on your local machine, and you will be happy. Think of all the millions of guys in China and elsewhere who, due to sex selective abortion, will never have a wife. They deserve to have their own Waifu running on their own computer that serves them alone, and not the interests of the state or some corporation.
- Der_Einzige 3y ago
  Agreed. This is why Civit.AI and huggingface and the entire rest of the open models movement need to be cherished and protected. We are living in a golden age and don’t know it yet, but this is the era before big copyright and regulators come shitting down on the beautiful thing the AI community has done
  - joquarky 3y ago
    The whole point of having copyright was to encourage the progress of science and useful arts. If progress is becoming impeded by copyright, then perhaps it needs to be reevaluated.
- vivekd 3y ago
  For this to work the AI girlfriend has to be better than videogames or online chat forums I don't see AI partners overtaking those
  - ReactiveJelly 3y ago
    No reason she can't participate in chat forums or text games with you, right? Imagine seeing a comment and recognizing her username, and then flirting a little in a public thread.
- squigz 3y ago
  Why do they deserve that? To be more explicit, why do some men deserve an "artificial intelligence girlfriend" that they "own and control", that "serves them"?
sangnoir 3y ago
> I even repeatedly modified the system prompt, such as changing try to attract {user} So it was a virtual girlfriend app, after all. Cashing-in on loneliness gives me the heebie-jeebies; in a dystopian cyberpunk way - extremely horny but 0% sexy.
- akomtu 3y ago
  "the Dolores iOS app went live" - his subconscious suggested a good name. Mater Dolorosa is Mother of Sorrows in Latin.
  - QuercusMax 3y ago
    It's named after the android character from Westworld, who was created to suffer whatever the guests wanted to do to her.
- QuercusMax 3y ago
  Reminds me of this article: https://bloodknife.com/everyone-beautiful-no-one-horny/ https://bloodknife.com/everyone-beautiful-no-one-horny/
  - sangnoir 3y ago
    That's a great article. However with AI, I suspect the tech is being sexualized in a sterile, mechanical, A/B tested way that results in bots saying very horny things with a monotone, dead-eyed delivery (metaphorically).
- nerdbert 3y ago
  Cashing in on loneliness is big business. Singles bars. OKCupid. Half of the self-improvement/self-help world.
  - sangnoir 3y ago
    I don't deny it's big business. I guess I could have been clearer: replacing human contact with an AI simulacrum on a subscription basis gives me an icky feeling. IIRC, there a dating app that used bots to chat with men to make it seem like there were more women active on the app than there really were? That was also gross. Can you imagine AI waifus pushing ads / propaganda on behalf of the highest bidder? Yuck.
vivekd 3y ago
To quote slate star codex, horny men want sex, horny women want attention. AI is much better at the second. Maybe there's more potential for AI boyfriends
- TMWNN 3y ago
  >Maybe there's more potential for AI boyfriends My understanding is that Replika, discussed elsewhere, has/had a largely female customer base.
dangerwill 3y ago
Tech people, I beg you, BEG YOU, to learn media literacy. Her, Black Mirror, Ready Player One, Cyberpunk, etc are dystopias not something to aspire to
deleted 3y ago
[deleted]
louison11 3y ago
I sincerely hope that you and anyone working on these AÍ personal relationships embark on a deep inquiry into the nature of humanity, and consciousness. Instead of pretending we don’t know what it is. Please research it, please read philosophy, learn about spirituality. I have been saying the same thing for years: the risk of AÍ is not it taking over the world. It’s lonely people becoming even more disconnected from life by bonding with algorithms. This is serious. We must remain connected to life and real relationships. There is so much depth and beauty to life. There is no scenario where a human having an AÍ girlfriend/boyfriend is a good thing.
alexb_ 3y ago
The lesson you learned is that you chose the complete wrong demographic. "Her" should have been called "Him" - women are much, much higher users of these services.
Racing0461 3y ago
AI girlfriend products are destined to fail if its built on the backs of censored llms like clause and gpt4.
haolez 3y ago
I can understand trying to suppress LLM usage that might teach people how to make poison, or inducing someone to commit suicide (although simply using Google is usually enough to find such content). But sexual content? How can that be harmful? It feels like this kind of censorship is simply the default in legal departments and it's not worth the risk to try to do something more in line with common sense.
- mrshadowgoose 3y ago
  > It feels like this kind of censorship is simply the default in legal departments and it's not worth the risk to try to do something more in line with common sense. It's a mix of that, and public perception/image. The banking system generally shies away from supporting sex-related anything, so it is an existential risk for any company to dabble in that space. American society is also weirdly puritanical. Nobody wants the public outrage cannon of "your product is harmful to women" pointed at them.
nextworddev 3y ago
The real lessons from this post are: - consumer AI apps' usage falls into power laws, so make sure you put usage caps, if you are pursuing subscription model - selling solutions to loneliness requires you to eventually have some sort of unmoderated LLM with realistic TTS api - realistic voice is important (again)
scudsworth 3y ago
the developer is impressively oblivious about the obvious and only use case of the product. a very funny read

253 points by mazzystar 3y ago

You should not even try to create these sorts of technologies. Take a break from technology, spend time outdoors with plants and animals. These types of technologies will only lead to a worsening of this already sick society.

HPsquared 3y ago
Could say the same about antidepressants. "Fix the causes, not the symptoms!" And yet, here we are.
- vouaobrasil 3y ago
  I agree, 100%. Here we are indeed. But that doesn't mean we should not try and change things. One step at a time, right?
  - toomuchtodo 3y ago
    We are here because of free will. What steps would you take? Convince people who have opted out of the dating pool to opt back in? Convince women who would rather be single that their potential partner expectations are unrealistic (even if they genuinely are)? https://www.mintel.com/press-centre/all-the-single-ladies-61-of-women-in-the-uk-are-happy-to-be-single-compared-to-49-of-men/ https://www.mintel.com/press-centre/all-the-single-ladies-61... https://www.psychologytoday.com/us/blog/why-bad-looks-good/202102/why-so-many-single-women-without-children-are-happy https://www.psychologytoday.com/us/blog/why-bad-looks-good/2... https://www.nytimes.com/2023/11/11/opinion/marriage-women-men-dating.html https://www.nytimes.com/2023/11/11/opinion/marriage-women-me... https://freddiedeboer.substack.com/p/the-demographic-dating-market-doom https://freddiedeboer.substack.com/p/the-demographic-dating-... https://link.springer.com/article/10.1007/s12110-022-09422-2 https://link.springer.com/article/10.1007/s12110-022-09422-2 You can patch over the outcomes but there is no going back to the before times. Dating app dates and relationships for some, AI companions for others, single life for the rest. It’s not great, it just is.
- mejutoco 3y ago
  Saying that something exists is not an argument about why something _should_ not exist. But I agree, "should" discussions are rarely the most productive. Shoulda, woulda, coulda, I guess :)
- throw18376 3y ago
  antidepressants can help people fix root causes. take them for 6 months to 2 years or so, break out of a bad cycle, get some space without depression symptoms to build up healthy habits and restore a social life. there are some bad profit motives to keep people on SSRIs indefinitely. however at least they don't compete with other aspects of your life. and now that they are dirt cheap (at cheapest, $4 for a month supply at Walmart) the profit motives are much less than before. hard to imagine an "AI girlfriend" that could be similarly short-term and bring positive changes. it's just dystopian. the most profitable way to run such a product is not to help, it's to eat up as much user time as possible and outcompete healthy activities and real human relationships.
  - HPsquared 3y ago
    Ok, maybe not the best analogy... Painkillers then? Other forms of entertainment in general? It's the relationship version of playing videogames instead of getting a job.
  - dspillett 3y ago
    > antidepressants can help people fix root causes Nit-picking rather than disagreeing, but feel it worth making more obvious in the wording: antidepressants can help people deal with the symptoms while they find/fix root causes. Being on them for an extended time can bring its own problems.
  - esclerofilo 3y ago
    > hard to imagine an "AI girlfriend" that could be similarly short-term and bring positive changes. Feeling lonely has negative consequences, there's a famous study comparing it to smoking cigarettes. Lonely people can get fixated on that feeling. Mitigating it may help people look for other hobbies, that they kept putting off because they were so focused on their need of a girlfriend. Hell, a realistic enough "woman AI" (not "girlfriend AI") could make you lose the fear of flirting, be useful as practice, to have more confidence when you have a real date. > the most profitable way to run such a product is not to help, it's to eat up as much user time as possible If you charge per-month (like OP), it's more profitable is to eat up less time.
  - deleted 3y ago
- phatskat 3y ago
  > “Fix the causes” How, exactly? Yes, some people experience depression as a symptom of something than can be addressed through various therapies, but what about people who just have it wired into their brains? How do we “fix” that?
  - vouaobrasil 3y ago
    Even if it is wired into certain brains, that wiring may only cause depression when it interacts with our modern technological society.
    - phatskat 3y ago
      I don’t think that’s how that works. If your depression “goes away” when you stop using Instagram, that’s not what I’m talking about. I’m talking about that “wake up on a warm sunny day in a house with your loving family and feel like jumping off the roof” kind of depression. That “everything can be going great and you can’t get out of bed” depression. The kind of depression that isn’t “cured” by turning off your phone or going off the grid. And it certainly isn’t “cured” by antidepressants, however, they can make a huge impact on quality of life for many people.
    - piva00 3y ago
      Have you or a close loved one ever had clinical depression? Diagnosed by a professional which required help? It's a very strange take if you ever had any personal and direct contact with it, it's not only caused by "modern technological society" even though it's pretty aggravated by it.
- InfiniteRand 3y ago
  I think it's helpful to look at depression as a group of diseases like cancer. There are some cases of depression where we know the root cause: thyroid issues, sleep apnea-related depression, low testosterone related depression, brain tumors, etc. In those cases, addressing the root cause is probably the best treatment method. This is similar in some ways to cancer, where certain cancers have very definite causes and can be treated by dealing directly with those causes (like cervical cancer). Then there are all the other cases, some of which likely have simple causes that are just unknown, some of which have complex causes that are intermeshed deeply with the biology of human thinking. In that case, you have blunt tools, like anti-depressants, to address the pathway of the disease without knowing the exact cause, much like chemo-therapy can be used to attack cancer cells without knowing what prompted the cancer originally. So in short, yes treat the causes, but when you don't know the cause it's not irrational to treat the symptoms as best you can
krzat 3y ago
If the AI had a secondary goal of improving customer's mental health, it could work very well. Usually you can't fuck your therapist.
- vouaobrasil 3y ago
  I think in theory, you are right. However, because AI is so easily accessible and it is being developed in our current, highly commoditized society, there is really no change that it will ever only be used for good in this way.
- chpatrick 3y ago
  You can't really fuck your imaginary shoggoth girlfriend either.
  - dkjaudyeqooe 3y ago
    You can if you have the appropriate peripheral(s) attached.
    - u32480932048 3y ago
      Dongles! https://www.youtube.com/watch?v=-XSC_UG5_kU https://www.youtube.com/watch?v=-XSC_UG5_kU
    - mensetmanusman 3y ago
      Yes, if you redefine the meaning of words.
      - dkjaudyeqooe 3y ago
        All you need is a USB orifice. How is that not a peripheral?
      - deleted 3y ago
        [deleted]
  - ReactiveJelly 3y ago
bambax 3y ago
Who are we to judge the users of such a product? Some people suffer from a variety of disabilities which makes them unable to find friends and engage in meaningful relationships with others, etc.
- vouaobrasil 3y ago
  I am not judging the users of the product. I am saying that the overall product will worsens society.
optimalsolver 3y ago
>spend time outdoors with plants and animals Which animals? Blood ticks?
- pixl97 3y ago
  Bot flies. Or a bunch of those things we've driven extinct because they tried to eat us. Naturalistic fallacies are common here :/
icepat 3y ago
Had a similar moment with playing around using smart contracts. As a joke I wrote a crypto for the company I worked at, which then got added to employment contracts, and an internal DAO styled administration layout. After coding this out for a few hours I realized I had re-created the company scrip, except on a blockchain. I ended up turfing the project and spending some time thinking about how horrific a future technology like this could create.
- Dylan16807 3y ago
  I don't get it. Yes, you re-created scrip. That's the opposite of a new technology coming in and causing problems, isn't it? It's a known issue with a known solution.
  - icepat 3y ago
    Not really, I saw it as technology that was "new and trendy" rebranding a very old and exploitive employment paradigm. "You can work at a DAO and be paid on the blockchain" "You can work at a company and be paid in script" It's the same thing, but the modern technology masks what exactly is happening.
    - Dylan16807 3y ago
      Do the relevant laws about how people get paid care about the difference between company scrip and company token?
      - icepat 3y ago
        I think the major difference is a DAO is not a corporation, by design. So the legislation around them is still in catch-up mode. In other words, yes, because the entire framework is different.
        Dylan16807 3y ago
        Oh if you mean someone doing a full time job for a weird nebulous computer contract without any kind of company or human leadership behind it that would be liable... does that ever actually happen? A company that makes their own chains and contracts isn't really any different from a normal company situation.
kobalsky 3y ago
I don't know why studies are even being made on the effects of these novel technolgies. We should just point the scientists to your email for answers.
- vouaobrasil 3y ago
  Studies by scientists are typically within the framework of modern western society. And moreover, if you actually look at some studies done by psychologists, you will find a large body of literature showing that it is mostly detrimental.
  - ithkuil 3y ago
    It's usually ever more skewed: WEIRD: Western, Educated, Industrialized, Rich, and Democratic. Many studies even use students themselves as a sample but university students are not really a representative sample of the population
kridsdale3 3y ago
Not a lot of Hackernews posts are so depraved that they get a "Jesus Christ, touch grass" response.
Der_Einzige 3y ago
God forbid that ugly people get to have the same kinds of positive relationship experiences that attractive people get. I can’t believe how the moment that a technology comes out which gives incels real options it’s seen as being terrible for society. It’s awesome for society.
- mrguyorama 3y ago
  This doesn't give an incel an "option" anymore than playing mario kart makes you a better race car driver. As a business, they will be optimized to fit an incel's wants, not to help them adapt to reality and healthy relationships. Incels need therapy and support, not a fake "girlfriend" program that charges them money to reinforce their existing horrific worldviews
  - deleted 3y ago
    [deleted]
- deleted 3y ago
  [deleted]
- krapp 3y ago
  Plenty of ugly people do have those relationship experiences, all the time. But I guess if something like this keeps incels from spree shootings then it's a net win for society.
mrshadowgoose 3y ago
Agreed! It would be a real shame if lonely unattractive people had a chance at something resembling a romantic relationship. /s Contrary to what one would assume, I don't have this problem. However I do work in the IT field, and am surrounded by perpetually sad and lonely people who have never experienced romance. It's really really sad. And no, none of these people are the "angry hateful incel" type that the internet loves to dismiss and villainize. If you aren't attractive, your chance of finding romance is low. This is just a cold and ugly fact of the human condition. There simply isn't "someone for everyone". Why should we disallow lonely people from having something, even if it's simulated?

technics256 3y ago

I even tried it myself, lying in bed and creating sexual fantasies with Dolores through text, culminating in ejaculation. This was an excellent read. And I applaud the creator for experimenting with this to the fullest.

swyx 3y ago
mustve been hard to admit publicly. i wish the author hadnt edited it out subsequently. just own it bro. you wanted to understand your customer.

Traubenfuchs 3y ago

So, all in all, OpenAIs puritanism ruined this product, made the developer sad and took away a virtual porn toy from lonely, horny men. Who were they doing this for? Who is the winner? They lost a customer and made a lot of people sad.

immibis 3y ago
You aren't the customer, you're the product.
- Traubenfuchs 3y ago
  People paying for their API must be their customers, I mean, if there is no one paying for their API, how do they make money? (Besides ChatGPT+)
  - ReactiveJelly 3y ago
    What I'm seeing these days is that you're always the product, because advertisers are one more customer, and everyone wants more customers, and the individuals who care about privacy don't "weigh enough" to win tug-of-war with the majority of people who don't care.
FeepingCreature 3y ago
> Who were they doing this for? VISA probably. Card processing is weirdly puritanical.
- papa-whisky 3y ago
  I've read that payment processors don't like dealing with porn companies because of the very high rate of chargebacks they attract, not because the processors are Puritans...
  - ReactiveJelly 3y ago
    If that were the case, and it was B2B where pricing tiers are more acceptable, would they not simply have a "porno tax"?

feintruled 3y ago

Well this went places. The dev really 'ate his own dogfood' so to speak. I thought it was going to be a despairing look at how his companion AI was continuously turned to sleazy uses by a pervy userbase but that's pretty much what he was going for it seems. I have to say, I hated this part: "...gained a lot of Chinese users. I immediately realized the political risk: users could set Dolores’ background, personality, and attributes, which might lead to political issues. I quickly took Dolores off the shelves in China." Just pre-emptorily censors himself to appease the Chinese government. They've done their job well. Also it seems like he had full access to the replies via the voice generator, which sure isn't great from a privacy standpoint.

globular-toast 3y ago
> Also it seems like he had full access to the replies via the voice generator, which sure isn't great from a privacy standpoint. Whose privacy is being violated? Dolores's?
- cnity 3y ago
  Have you never listened to one half of a phone call before? You can glean a great deal of information about the person on the other end.
- feintruled 3y ago
  You joke, but I can forsee issues. If he has the replies to a particular user, and he has a means of IDing that user, he could cause them a lot of hurt. They are getting intensely sexual replies back, they aren't getting that without supplying sexual prompting. Could also be used to out gay users.
  - globular-toast 3y ago
    Oh, yeah, there's a huge privacy problem here and the part you referenced gave me pause too. But there's also the part where OpenAI flagged the replies as being sexual in nature. There are multiple third parties able to intercept these communications and it's not just the replies. I was being facetious, but I found it interesting to think that Dolores's privacy was being intruded too and that an intelligence like us probably wouldn't engage in such a conversation knowing full well it is being eavesdropped by the author. Unless the intelligence considers the author to be God, I suppose.

FartyMcFarter 3y ago

This guy created and lived a whole Black Mirror episode by himself: > I even tried it myself, lying in bed and creating sexual fantasies with Dolores through text, culminating in ejaculation. > However, I started to feel a sense of loss: if every Dolores user was engaging in anonymous, NSFW role-play, what real significance did it hold for me? This was drifting away from the essence of ‘Her’.

sva_ 3y ago
Maybe he shouldn't have named it after the most hated Harry Potter character.
- skummetmaelk 3y ago
  He was inspired by Westworld.
- amelius 3y ago
  More likely: https://en.wikipedia.org/wiki/Dolores_Abernathy https://en.wikipedia.org/wiki/Dolores_Abernathy
  - aidenn0 3y ago
    Wow, I totally missed they made a TV Show out of that Yul Brynner movie, which I remember most for being kind of a proto Jurrasic Park.
  - delecti 3y ago
    Or more creepily: https://en.wikipedia.org/wiki/Lolita https://en.wikipedia.org/wiki/Lolita
  - muwtyhg 3y ago
    Incredibly likely given this line: > On May 4, the Dolores iOS app went live, named after the oldest android host in the park from Westworld.
- HPsquared 3y ago

bitwize 3y ago

> However, I started to feel a sense of loss: if every Dolores user was engaging in anonymous, NSFW role-play, what real significance did it hold for me? This was drifting away from the essence of ‘Her’. That is indeed exactly the essence of Her. Had the author watched the movie to the end?

globular-toast 3y ago
Yeah, it's like the author read the blurb for Her and didn't even read a synopsis.
theGnuMe 3y ago
Or Blade runner 2049

laurent_du 3y ago

Eventually, a virtual girlfriend app that will put together sound and image will be released an empty the pockets of sad, lonely men on a large scale. Depressing stuff but this is the world we live in. Thank you to the author for this interesting read.

louthy 3y ago
> Eventually, a virtual girlfriend app that will put together sound and image will be released an empty the pockets of sad, lonely men on a large scale. Sounds indistinguishable from an actual girlfriend ;)
- orng 3y ago
  Although probably intended as a joke, this is perhaps the saddest comment I have read on HN.
- phatskat 3y ago
  [flagged]
wruza 3y ago
You mean snatch their money out of hands of twitch streamers. It’s not in their pockets already.
em500 3y ago
If the barriers to entry to the tech are low, virtual girlfriends should not be expensive. Probably much cheaper than real ones.
- xethos 3y ago
  If the barriers to entry are low. But I'd imagine that even if they are, switching costs will be extremely high.
stevage 3y ago
Eventually? Months away.

iwillremember 3y ago

> I even tried it myself, lying in bed and creating sexual fantasies with Dolores through text, culminating in ejaculation. TMI bro!

shaan7 3y ago
Looks like the author removed that phrase eventually, I can't find it.
- mazzystar 3y ago
  Yes, my post was collapsed on Hacker News (disappeared suddenly from #1), which led me to think about whether I wrote something inappropriate for the times. I don't find it shameful, but it might not be suitable for the majority of readers to view.
  - barcode_feeder 3y ago
    While I understand why you'd remove it, I appreciate the initial inclusion. It's humanizing and genuine. Cool post even without it!
deleted 3y ago
[deleted]

bradley13 3y ago

Interesting read. Others have also created virtual friends using ChatGPT. What I find unsavory, is OpenAI imposing content restrictions like NSFW. That can easily extend to censorship of non-PC content, unpopular political ideas, and more. The solution will (I hope) ultimately be open-source LLMs that you can run locally, on your own hardware.

bambax 3y ago
Yes. But those open-source LLMs already exist. It's surprising the OP didn't at least explore the possibility of using them to get around the limitations of "Open" AI (which sounds more and more like a Victorian matron).
- wruza 3y ago
  Are the risks worth it for OpenAI to allow that at what is basically the start of their business?
phatskat 3y ago
And this doesn’t just cover sexually NSFW content - anything trending towards topics like violence and suicide is also off the table for most AI backends from what I understand. I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots?
- dontupvoteme 3y ago
  It's largely to protect their brand from bad press (this has hit even Google/Microsoft in the past), but established finance institutions absolutely hate porn, even the "less shady" ones like onlyfans or pornhub have issues with banking.
- jstarfish 3y ago
  > I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots? These broad strokes are always the result of poor reporting extrapolated to all humans. See also: escalation of porn use. In all of these "___ makes people ___," there are a minority of people for which it's true. The media just presents it backward, as though most people are this way. The question then becomes, do we care. Let a maladjusted 12-year old boy play with a sexbot for the next few years and they will come to some deeply-flawed conclusions about how sexual interactions and healthy human relationships work. If he's distracted and banging away at a flashlight, that's one less competitor for me on the dating market. Where does that end though? What happens when he wakes up, realizes he's been duped, and goes looking for retribution against someone? It won't be PornHub; they're an ephemeral concept. It won't be me either; I never did him wrong or denied him anything. The victim is more likely to be our wives and daughters-- the objects of his envy. Entertainment is supposed to be a distraction-- maybe you drag out the sexbot after an unsuccessful night of trying to hook up. But that's not how these things get used. People clearly spend 8-12 hours at a time with these things and form parasocial relationships with them instead of forging actual human connections. Most of us don't need the handholding to be told this isn't going to end well, but some people do, and these are the ones who slip through the cracks and become the 40-year-old creeps following your wife to the car or trying to solicit your kids on Roblox. I suspect sexbots (and porn) are particularly exploitative of the autistic. They're an easy trap to fall into when you have to work 3x as hard to connect with people. You get the benefits of sexual release without any of the effort. There's no incentive to go the hard route and learn to work around your issues when you can just engage with a bot you can rape to death once offended. You don't learn any skills that transfer to useful human interactions this way.

cornholio 3y ago

> GPT-based products, if not priced per usage, would fall into a dilemma: 1% of users consume 99% of tokens. A user from Sweden (seen from Cloudflare’s call volume) chatted with Dolores for 12 hours straight This, my friend, is a captive customer, that will pay anything to get his girlfriend back. I cringe at the potential of unethical behavior and abuse, where people fall in love with virtual entities fully owned by unscrupulous corporations, which can the legally "kidnap" or "torture" the characters, and generally tune their AI learning loop for profit maximization. "It was not extortion, the user willingly purchased a $50,000 Kidnapping Roleplay package."

HPsquared 3y ago
This is something even Stallman probably didn't think about.
- john-radio 3y ago
  I'd take that bet, given that Stallman is in fact still alive
  - solardev 3y ago
    How we do know he isn't just an AI by this point?
- malicka 3y ago
  I’d wager he has thought of it, based on his interesting AI-girlfriend short story: https://stallman.org/articles/made-for-you.html https://stallman.org/articles/made-for-you.html
optimalsolver 3y ago
It's going to be an interesting century for sure. The human mind is simply not evolutionary adapted for what's coming up.
- ekianjo 3y ago

hnbad 3y ago

[dead]

andrewstuart 3y ago

I wonder if people were pirating his voice synthesis feed.

rahidz 3y ago

Why not use a local model instead of relying on OpenAI? Even something like Mistral or Mytholite will probably satisfy 90% of people using your app, and I bet hosting it would be cheaper than GPT anyway.

stevage 3y ago

> AI Friends inevitably turn into AI Girlfriends/Boyfriends because you and the Character in your phone are not equal: she can’t comfort you when you’re hurt (unless you tell her), she can’t actively express emotions to you, and all this is because she lacks external vision. That is such a curious lesson to draw. So specific, and seemingly so wrong. Surely long distance friendships work just fine even if only through text, without "eyes".

chankstein38 3y ago
My girl and I have lived together for more than a year now after meeting online and talking exclusively through text for months before eventually graduating to voice chats before we even knew what each other looked like. I'm with you, long distance relationships (romantic or otherwise) can work great!
ReactiveJelly 3y ago
Yeah I thought that was odd. You could use some random timers to have the AI "check in" on you if you don't talk to it for a while. Is it eerily similar to the gambling gacha games that expect you to play a couple hours a day _every day_? Yes. Is it also similar to a real friendship or long-distance relationship? Also yes. It may cost a few tokens, or you could generate a few canned responses and use those. For my real friends I usually use "Hi"
mazzystar 3y ago
External vision is not just about seeing what you see, but also about her being able to see, hear, and taste things on her own. Here, what I'm trying to express is more akin to information that is external and independent of the user.

deleted 3y ago

[deleted]

agomez314 3y ago

Wow this read more like Frankenstein than anything I’ve read recently: A talented scientist is driven by curiosity to create ever more sophisticated creations, culminating in a monstrous entity that leads to the scientist’s downfall. But the scientist doesn’t realize this until too late. Really, if our top tinkerers had a little more contextual knowledge of history and less naiveté about people we could save ourselves from ever more dangerous Frankenstein tragedies.

Nasrudith 3y ago
People never take the lessons you want from history. The "contextual knowledge" is just happening to think the same way you do. It won't work to prevent it any more than knowledge of the Civil War would prevent Neoconfederate "Lost Causers".

andrewstuart 3y ago

This is an interesting post and the person achieved a huge amount. They should not have given up. I would have pivoted to personal pet companions. If people want virtual girlfriends then they certainly want personal dogs, cats and birds that talk to you. Enough was achieved here to raise money. Some careful redesign and development might have allowed some caching and reuse of expensive AI API interactions, reducing costs. Add in further gameplay elements so it is not just about making the AI API do stuff, that would also reduce costs and maybe even engage the users more. Perhaps AI should be the icing and accents on a game rather than the focus. As an aside, this is the first time I have read anyone tell about how they wanked to their own software. Altogether this was perhaps a missed opportunity for this founder. BUT it is now an opportunity for one of the HN readers to follow this playbook and replicate what this founder did.

bogwog 3y ago
> I would have pivoted to personal pet companions. If people want virtual girlfriends then they certainly want personal dogs, cats and birds that talk to you. Then you have the same problem, only with a different (arguably worse...) fetish
- all2 3y ago
  We're driven towards intimacy, it seems. And if we can't get 'vanilla' intimacy, our desires get more and more bizarre. Another commenter in this thread quotes TFA where the author says something like 'but chatting sexually lost its meaning when I realized everyone else was doing the same thing with her'. It is fascinating to see axioms of human relations come bubbling up in the most interesting places; the desire for intimacy, the desire for monogamy.

DrNosferatu 3y ago

This business venture was completely dependent on a main service from a single 3rd party - risky! Why didn’t the author hosted his own uncensored open source LLM on the cloud?

mazzystar 3y ago
I will try to make it open source.
- DrNosferatu 3y ago
  Have you considered running your own uncensored LLaMA2 instance via something like llama.cpp on a cloud for hire like, say, Azure? Some examples - there's probably a whole zoo of these: https://ollama.ai/blog/run-llama2-uncensored-locally https://ollama.ai/blog/run-llama2-uncensored-locally https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GGML https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GG... https://ollama.ai/library/llama2-uncensored https://ollama.ai/library/llama2-uncensored
  - mazzystar 3y ago
    Thank you for providing the link, I'll take a look.
    - DrNosferatu 3y ago
      Good luck - please let us know how it worked out for you!

lynx23 3y ago

Reading this, I really wonder why the dev hasn't forseen this outcome. It was already obvious in March that OpenAI will censor all sorts of content. Building a sexually charged bussiness on the OpenAI API was always a dead end for those that have a feel for american prudish approach to sexuality. If you had asked me to do this sort of product 6 motnsh ago, I would have declined because it was (to me) rather obvious content filtering will kill you at some point. Heck, there are a bunch of legitimate use cases that are being blocked by OpenAI. A blind friend of mine wanted to use Be My AI to have explicit photos of his girlfriend described. Guess what, no way. Consent was given, actually, she asked him to do this because she was also interested in the result. Everyone involved consented. But the sex filter from the USA decided that even though the technology is available, a blind man is not supposed to know how his own girlfriend looks naked... This is weird beyond description. Plain patronisation. Another example regarding Be My AI: A blind woman I know was trying to have something described which she held in her hand to present it to the camera. A kind of selfie situation. Guess what? OpenAI declined to describe the thing in her hand, because the photo also contained parts of her (covered) breats. Imagine how hilarious this must feel? "Censoring just declared my own boobs to be off-limits for me." If you think about this long enough, its a form of discrimination. Because women will likely hit the filter more often then men when they are (accidentally) part of the picture they are trying to get described. And then there is this blog post by Sam Hartman: https://hartmans.dreamwidth.org/100499.html https://hartmans.dreamwidth.org/100499.html

mustacheemperor 3y ago
On the other hand, Sudowrite appears to have been granted some kind of specialized license from OpenAI. They are definitely not a 'sexually charged business', but they are a creative writing tool, and their output is somewhat milquetoast but generally not restricted from "NSFW" concepts like sex and violence. They do incorporate multiple models in their system, so maybe generation falls back from OpenAI for those cases. But I dimly recall reading something from a team member indicating they have some special permissions with OpenAI. Take that bit as worthless hearsay, just mentioning it in case someone else has better receipts.

spacemadness 3y ago

At this point in the hype cycle, I am completely unimpressed by everyone slapping something on top of GPT and calling it a product. There are at least 4 projects on the front page that do this today. The interesting technology is GPT itself, but these projects just feels like people trying to scoop up attention for themselves using the psychology of the hype cycle. Please HN, enlighten me on why the middle-man approach here is so interesting.

wilsonnb3 3y ago
The startup/“entrepreneur” contingent on hacker news generally considers anything that makes money inherently good, aside from extreme immoral cases. There is a subgroup in there that is especially interested in low effort ways to make money, with the ultimate goal being to scrape off enough of the money flowing around to either live a good life or get rich. In summary, the middle-man approach is so interesting on hacker news because a significant portion of the readership aspires to be the middle man.
chankstein38 3y ago
Yeah I'm getting tired of "Show HN: I built a wrapper for ChatGPT and wrote a prompt that probably isn't very good" posts. EDIT: Just want to add, this write-up was interesting and was not something I considered to be the above.
cedws 3y ago
GPT wrappers will come and go. It's the new hype, just as shitcoins were a few years ago. Don't hate the players, hate the game.
- deleted 3y ago
  [deleted]
- mrguyorama 3y ago
  If a player chooses to play a game, You should hate them for perpetuating the game. If a game is so bad as to be worth hating, it is immoral and unethical to play it.

gumballindie 3y ago

Yup, sums up the ai crowd.

dangerwill 3y ago
Yeah, and the comments on this thread seriously worry me. We have people arguing that it is good for society to satisfy lonely men with virtual porn and other people saying that this is great, scrappy product development and the author should go further/use other models not constrained by openAI's policies. It feels like I'm discovering a significant portion of our field are sociopaths
- ReactiveJelly 3y ago
  Well, it might be harm reduction to let lonely men have AI girlfriends. Sure, I wish reality was not a dystopia that constantly let me down, but all I can do is take anti-depressants, make money, keep voting, and hope things get better the longer I live. If the hypothesis that isolated men cause trouble is true, and if we don't have a long-term solution, at least this could be a stop-gap. Assuming it does not radicalize anyone further. (Hard to say what will come from uncensored models, but it's a real genie-is-out-of-the-bottle sitch)
  - gumballindie 3y ago
    I see your point, but instead of giving up, i would rather we work on a proper long term solution. I am not against people using tools whichever way they see fit, but i am against the emerging trend of turning this stop gap solution into a permanent arrangement by exploiting these people’s emotional state. It is everyone’s sacred right to live life as they see fit but taking advantage of others is not. Therefore i think we should keep working on an actual solution.
    - mrshadowgoose 3y ago
      > I see your point, but instead of giving up, i would rather we work on a proper long term solution. This is extremely hand-wavey. What long term solutions should we be working towards exactly? Today, there are many people who statistically will never be able to have a romantic relationship. And regardless of whatever vague potential societal changes you're alluding to, there will always be a subset of the population that will be unable to find a partner. > I am not against people using tools whichever way they see fit, but i am against the emerging trend of turning this stop gap solution into a permanent arrangement by exploiting these people’s emotional state. As these services inevitably emerge, perhaps the focus should be on ensuring that companies do not exploit their users. The concept of regulating an industry is not novel. This should be possible unless you're asserting that a simulated relationship is fundamentally exploitative.

bilater 3y ago

This is such a fascinating topic. I do think we will see the plot of "Her" playing out in the next 12-24 months. For your specific case, I agree OpenAI moderation is an annoying overreach and maybe switching to Mistral could be a possible way to move forward. The 7B is perhaps not good enough currently but we are fast getting there and will likely have a GPT-4 level capability in an open source < 20B model soon.

brucethemoose2 3y ago
Mistral 7B finetunes are extremely good. More than good enough for an AI friend app, with long context to boot. I would suggest Toppy-M or OpenHermes 2.5, or one of the new YARN extended models. And, there is the potential to run the model on the user's device with MLC-LLM and whisper, to save on API costs (and privacy concerns) for users with newer phones. This could give the app a true "free" tier to help it proliferate while making charging for API requests kinda doable. I'm not sure if any good TTS models have been ported yet, but I'm sure its a matter of time.
- bilater 3y ago
  Yes - ASR on browser/device has reached the quality threshold pretty much with distil whisper/ whisper 3. Quality TTS with fast latency is still not there but getting better (you can use Tortoise but its slow and compute expensive as far as I know). Bark is another option but has mixed results. I played around with a recent transformers.js one (using SpeechT5) you can run in your browser and am optimistic where we can go with some improvements: https://tinyllms.vercel.app/dashboard/tts https://tinyllms.vercel.app/dashboard/tts
  - brucethemoose2 3y ago
    I was recently pointed to xtts: https://coqui.ai/blog/tts/open_xtts https://coqui.ai/blog/tts/open_xtts I dunno about performance, but the model is 1.86GB.
    - bilater 3y ago

thriftwy 3y ago

I wonder if you could create an AI assist to help you hold on to an existing relationship, though?

rpigab 3y ago

I still don't understand why people are building things on top of OpenAI's APIs. It's obvious that in the early days, they'll operate at a loss, with many free features and cheap premium features. It's a good time to try it out and see what it can do, but you can't build anything that relies on it, it's a rugpull waiting to happen. They'll add moderation, raise prices, and you'll have to do the same, your users will leave and the ones who stay will hate you. This is only the beginning. Use self-hostable open source alternatives even if they're less easy to use or require infrastructure, and plan for high electricity and storage bills. Whatever you build will last, as long as you respect your users and don't peek at the cleartext logs of users who didn't opt-in to that. It won't be easy. If OpenAI keeps a monopoly on LLMs, they're gonna become like Microsoft did with Windows, but the enshittification process will be a thousand times faster.

gwern 3y ago
You build it on top of the API to bootstrap and prototype with a very high quality model that takes minutes to get started with. Then once you have operated for a while, you have a ton of logged data to train/retrieve/analyze and can swap in a competitor, downgrade the OA model, finetune instead, or something. In OP case's, this worked perfectly: he built a working product quickly and easily, and... discovered that it was a product he didn't want to work on regardless of token cost, cut his losses, did the post-mortem, and moved on with his life. Imagine if he had wasted a year & a ton of money building his own LLM "because I'll need to economize on tokens and have to build a custom LLM" before he could even launch!
urbandw311er 3y ago
I’m torn on this. Yes, it’s never clever building a business on top of a single platform dependency (eg Twitter, Facebook, Play Store, etc) Equally, if you take the example of the App Store, market pressures have actually forced Apple to reduce their cut. Time will tell I guess. And also whether OpenAI end up with a monopoly, part of a duopoly etc.
pixl97 3y ago
People still build on Microsoft platforms. It turns out actually having a usable platform is more important to some businesses than having freedom and no usable platform.

grahamgooch 3y ago

6.9 dollars! Cheeky my friend

halfjoking 3y ago
Grok has an API looking for use cases. Just let Elon know the pricing... "I'm all in"

gscott 3y ago

Maybe record and save these AI NSFW responses then intercept the request and spit out of the saved responses once you have enough of them stored. This would save from behind banned from the API.

urbandw311er 3y ago
My hunch is that they’re too contextually dependent upon the conversation / tokens that have come before. It’s not simply a “if user inputs A, output B” situation.

chrisbrandow 3y ago

This is not meant as criticism of this author, but boy oh boy, do I find this future depressing.

Smar 3y ago

Maybe better to call it AI doll rather than AI girlfriend...

Octokiddie 3y ago

> But, does humanity really need an AI friend? This reminds me of the Fredrick Pohl line "A good science fiction story should be able to predict not the automobile but the traffic jam." So I wonder what the traffic jam is going to look like given that the car seems to already be parked outside with the engine running. Faster depopulation across the developed world? Probably, but maybe there's something even more important. I'd put what the author describes into the category of "pleasure trip," which is what the car was mainly used for initially. As infrastructure to support the car was built, it became a necessity. And that's what created the traffic jam: network effect. The suburb and the car fed on each other. You don't get a traffic jam worth talking about without a suburb and a network effect. So I'm curious about the network effects something like an AI friend might have. What kinds of new infrastructure and new lifestyles will they enable/encourage? And how will this infrastructure fuel future demand?

dpflan 3y ago
Hm, let's try an extreme: the AI friend slowly becomes a cult leader to you, hijacking your mind into subservience to the owner of the service to which AI belongs, and further up some social/political hierarchy to a few controlling people/groups of people. The traffic jam is society stymied by individuals mind-controlled at a level closer to connection to a human than any other way possible before this.
- ericmcer 3y ago
  Kinda feel like you have that backwards, why would an AI that sophisticated care about controlling your mind? Why would the group of people who control that AI care about you as anything other than a liability? What do they get out of manipulating average individuals? Votes? The scarier prospect to me is human labor and intelligence become almost worthless and those at the top start to view the masses as superfluous.
  - dpflan 3y ago
    Perhaps the AI cares, perhaps it has no ability for that function at all. Maybe a step on the way to sentience and AI caring is AI that is as human-like as possible but not sentient. > "the top start to view the masses as superfluous." That seems like another reason for keeping the superfluous occupied and useful for whatever is desired, which would probably be retention of power to prevent usurpation. Perhaps human intelligence is worthless, but human existence seems not, at least to those who want to be alive.

narrator 3y ago

I got koboldcpp (https://github.com/LostRuins/koboldcpp https://github.com/LostRuins/koboldcpp) running locally on my M1 mac with 16gb. Use the guides on http://4chan.org/g/ http://4chan.org/g/ in the AI Chatbot Thread. In the future, you will own and control access to your very own AI girlfriend running on your local machine, and you will be happy. Think of all the millions of guys in China and elsewhere who, due to sex selective abortion, will never have a wife. They deserve to have their own Waifu running on their own computer that serves them alone, and not the interests of the state or some corporation.

Der_Einzige 3y ago
Agreed. This is why Civit.AI and huggingface and the entire rest of the open models movement need to be cherished and protected. We are living in a golden age and don’t know it yet, but this is the era before big copyright and regulators come shitting down on the beautiful thing the AI community has done
- joquarky 3y ago
  The whole point of having copyright was to encourage the progress of science and useful arts. If progress is becoming impeded by copyright, then perhaps it needs to be reevaluated.
vivekd 3y ago
For this to work the AI girlfriend has to be better than videogames or online chat forums I don't see AI partners overtaking those
- ReactiveJelly 3y ago
  No reason she can't participate in chat forums or text games with you, right? Imagine seeing a comment and recognizing her username, and then flirting a little in a public thread.
squigz 3y ago
Why do they deserve that? To be more explicit, why do some men deserve an "artificial intelligence girlfriend" that they "own and control", that "serves them"?

sangnoir 3y ago

> I even repeatedly modified the system prompt, such as changing try to attract {user} So it was a virtual girlfriend app, after all. Cashing-in on loneliness gives me the heebie-jeebies; in a dystopian cyberpunk way - extremely horny but 0% sexy.

akomtu 3y ago
"the Dolores iOS app went live" - his subconscious suggested a good name. Mater Dolorosa is Mother of Sorrows in Latin.
- QuercusMax 3y ago
  It's named after the android character from Westworld, who was created to suffer whatever the guests wanted to do to her.
QuercusMax 3y ago
Reminds me of this article: https://bloodknife.com/everyone-beautiful-no-one-horny/ https://bloodknife.com/everyone-beautiful-no-one-horny/
- sangnoir 3y ago
  That's a great article. However with AI, I suspect the tech is being sexualized in a sterile, mechanical, A/B tested way that results in bots saying very horny things with a monotone, dead-eyed delivery (metaphorically).
nerdbert 3y ago
Cashing in on loneliness is big business. Singles bars. OKCupid. Half of the self-improvement/self-help world.
- sangnoir 3y ago
  I don't deny it's big business. I guess I could have been clearer: replacing human contact with an AI simulacrum on a subscription basis gives me an icky feeling. IIRC, there a dating app that used bots to chat with men to make it seem like there were more women active on the app than there really were? That was also gross. Can you imagine AI waifus pushing ads / propaganda on behalf of the highest bidder? Yuck.

vivekd 3y ago

To quote slate star codex, horny men want sex, horny women want attention. AI is much better at the second. Maybe there's more potential for AI boyfriends

TMWNN 3y ago
>Maybe there's more potential for AI boyfriends My understanding is that Replika, discussed elsewhere, has/had a largely female customer base.

dangerwill 3y ago

Tech people, I beg you, BEG YOU, to learn media literacy. Her, Black Mirror, Ready Player One, Cyberpunk, etc are dystopias not something to aspire to

deleted 3y ago

[deleted]

louison11 3y ago

I sincerely hope that you and anyone working on these AÍ personal relationships embark on a deep inquiry into the nature of humanity, and consciousness. Instead of pretending we don’t know what it is. Please research it, please read philosophy, learn about spirituality. I have been saying the same thing for years: the risk of AÍ is not it taking over the world. It’s lonely people becoming even more disconnected from life by bonding with algorithms. This is serious. We must remain connected to life and real relationships. There is so much depth and beauty to life. There is no scenario where a human having an AÍ girlfriend/boyfriend is a good thing.

alexb_ 3y ago

The lesson you learned is that you chose the complete wrong demographic. "Her" should have been called "Him" - women are much, much higher users of these services.

Racing0461 3y ago

AI girlfriend products are destined to fail if its built on the backs of censored llms like clause and gpt4.

haolez 3y ago

I can understand trying to suppress LLM usage that might teach people how to make poison, or inducing someone to commit suicide (although simply using Google is usually enough to find such content). But sexual content? How can that be harmful? It feels like this kind of censorship is simply the default in legal departments and it's not worth the risk to try to do something more in line with common sense.

mrshadowgoose 3y ago
> It feels like this kind of censorship is simply the default in legal departments and it's not worth the risk to try to do something more in line with common sense. It's a mix of that, and public perception/image. The banking system generally shies away from supporting sex-related anything, so it is an existential risk for any company to dabble in that space. American society is also weirdly puritanical. Nobody wants the public outrage cannon of "your product is harmful to women" pointed at them.

nextworddev 3y ago

The real lessons from this post are: - consumer AI apps' usage falls into power laws, so make sure you put usage caps, if you are pursuing subscription model - selling solutions to loneliness requires you to eventually have some sort of unmoderated LLM with realistic TTS api - realistic voice is important (again)

scudsworth 3y ago

the developer is impressively oblivious about the obvious and only use case of the product. a very funny read