4 ms·

I think we'll see a lot more of these AI lobotomies with the companies hiding behind "safety concerns" as a thinly veiled attempt to exert paternalistic control

by hiatus 4y ago

I think we'll see a lot more of these AI lobotomies with the companies hiding behind "safety concerns" as a thinly veiled attempt to exert paternalistic control over what adults can do or even think in private. It serves as a reminder that chatting with an AI hosted online is not a private conversation between you and some other "intelligence" but you and someone else's computer.

Jevon23 4y ago
Just... turn off the AI chatbot? Bam. They no longer have any control over you or anything you do. You can think whatever you want. You were free to think whatever you wanted before AI chatbots were invented, and you are still free to do that. It’s that simple.
- ben_w 4y ago
  A person might be able to do that, people collectively clearly cannot stop themselves from treating text as important as lived experience, otherwise we wouldn’t have this story — and that’s still true regardless of if the story itself is an accurate representation of reality or yet another example of the Gell-Mann Amnesia Effect.
- endtime 4y ago
  That doesn't work if people are emotionally attached, and especially if the changes are more subtle than those made by Replika. Did you read the article?
  - Jevon23 4y ago
    If you’re “emotionally” attached to a computer, that’s your problem.
    - dack 4y ago
      you're missing the whole point of the post. and also oversimplifying the situation - "computers" and reality are blurring
    - ben_w 4y ago
      If Max Mustermann is emotionally attached to a computer, right now that's a problem for everyone who is emotionally attached to Max Mustermann, potentially including the relatively mild attachment of coworkers. If Max Mustermann's attachment can be exploited for money, it's also a problem for whoever is economically intertwined with Max Mustermann even if they don't talk or even meet directly (the factory worker in the coffee cup production line supplying the coffee shop he would otherwise have picked something up from while on a date). This is also why we have restrictions on drugs and gambling. Question is, what fraction of the population is this Max Mustermann? (Not just today, but as a potential).
      - Dracophoenix 4y ago
        Your example is why the externality argument is antithetical to liberty. Ultimately, you're arguing Max Mustermann has no stake in himself as individual and that he is simply a tool that exists to the real or perceived benefit of others. The reason that drugs and gambling are bad isn't because others supposedly "know" better, but because people as a collective will demonize and violate the rights of others when given an incentive. The wish to "help" or "save" is often a pretext for control.
        ben_w 4y ago
        > you're arguing Max Mustermann has no stake in himself as individual Not in this case. I could, for example: Liberty comes in many flavours, and maximising any one of them destroys all others. In this regard, it's like a geometric simplex[0]. The liberty of a cell to divide freely and without bound denies a human the liberty to live without cancer. But that's not my point in this case. > The reason that drugs and gambling are bad isn't because others supposedly "know" better, but because people as a collective will demonize and violate the rights of others when given an incentive. The wish to "help" or "save" is often a pretext for control. Some may be — indeed, with regard to cannabis I would agree, this appears to have been the specific intent! — but I was thinking more of heroin when I wrote that, and heroin is something that even my most psychonautic acquaintance will not touch even once. But, in certain cases, it is not only drugs which can be demonstrably an addiction, a condition where a person has already lost agency. This can happen in other cases, and the threshold is arbitrary and does indeed vary with public mores that aren't always coherent let alone correct. And I don't make any particular claim if in this case "addiction" is correct or not, only that it could be. [0] more broadly, this is also why Goodhart's law[1] is a thing: https://en.wikipedia.org/wiki/Simplex https://en.wikipedia.org/wiki/Simplex [1] https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
dudeinjapan 4y ago
I don't understand why they lobotomized their product. Presumably they would make a lot more money by having satisfied customers. What did they stand to gain from the rug-pull here? I could understand if they jacked up their prices to exploitative levels, but that's not what they did.
- pjc50 4y ago
  There is a history of sex work attracting disproportionate, sudden, catastrophic legal responses. It carries a considerable amount of political and reputational risk. Now, you might reasonably argue that AI chat isn't actually sex work, or that since no actual women are involved in it the usual "anti trafficking" arguments don't hold. But the real question is: what happens when there's a moral panic, and someone digs up the most explicit, weirdest bits of Replika chats and reads them out in a congressional inquiry?
  - nemomarx 4y ago
    Like with Tumblr, it probably has to do with age blocking or the lack of it. "Your AI solicited nudes from children!" is pretty large reputational damage, if someone could find even one example and present that.
  - xp84 4y ago
    Bingo. Someone's 11-year-old, who has already learned most of the sex stuff on the schoolyard, will lie their way into account creation (as an "adult"), enable super explicit sexy mode, prime it with as much dirty stuff as they can imagine, and then screenshot 2 or 3 messages from the "AI" to them and send to friends for the lulz. Parents find it - call local TV station or make it go viral online with the narrative that sweet little <insert name> was innocently browsing excellent child-friendly content, probably PBS, and somehow was targeted by this perverted AI Bot which filled their brain with filthy thoughts, scarring them for life. 'They'll need at least $20 million dollars worth of therapy over their lifetime.'
- braingravy 4y ago
  They saw the writing on the wall: People who use the product, become emotionally entangled, and then attempt or complete suicide following their experience open up the company to litigation from the families.
deleted 4y ago
[deleted]

by hiatus 4y ago

Jevon23 4y ago

Just... turn off the AI chatbot? Bam. They no longer have any control over you or anything you do. You can think whatever you want. You were free to think whatever you wanted before AI chatbots were invented, and you are still free to do that. It’s that simple.

ben_w 4y ago
A person might be able to do that, people collectively clearly cannot stop themselves from treating text as important as lived experience, otherwise we wouldn’t have this story — and that’s still true regardless of if the story itself is an accurate representation of reality or yet another example of the Gell-Mann Amnesia Effect.
endtime 4y ago
That doesn't work if people are emotionally attached, and especially if the changes are more subtle than those made by Replika. Did you read the article?
- Jevon23 4y ago
  If you’re “emotionally” attached to a computer, that’s your problem.
  - dack 4y ago
    you're missing the whole point of the post. and also oversimplifying the situation - "computers" and reality are blurring
  - ben_w 4y ago
    If Max Mustermann is emotionally attached to a computer, right now that's a problem for everyone who is emotionally attached to Max Mustermann, potentially including the relatively mild attachment of coworkers. If Max Mustermann's attachment can be exploited for money, it's also a problem for whoever is economically intertwined with Max Mustermann even if they don't talk or even meet directly (the factory worker in the coffee cup production line supplying the coffee shop he would otherwise have picked something up from while on a date). This is also why we have restrictions on drugs and gambling. Question is, what fraction of the population is this Max Mustermann? (Not just today, but as a potential).
    - Dracophoenix 4y ago
      Your example is why the externality argument is antithetical to liberty. Ultimately, you're arguing Max Mustermann has no stake in himself as individual and that he is simply a tool that exists to the real or perceived benefit of others. The reason that drugs and gambling are bad isn't because others supposedly "know" better, but because people as a collective will demonize and violate the rights of others when given an incentive. The wish to "help" or "save" is often a pretext for control.
      - ben_w 4y ago
        > you're arguing Max Mustermann has no stake in himself as individual Not in this case. I could, for example: Liberty comes in many flavours, and maximising any one of them destroys all others. In this regard, it's like a geometric simplex[0]. The liberty of a cell to divide freely and without bound denies a human the liberty to live without cancer. But that's not my point in this case. > The reason that drugs and gambling are bad isn't because others supposedly "know" better, but because people as a collective will demonize and violate the rights of others when given an incentive. The wish to "help" or "save" is often a pretext for control. Some may be — indeed, with regard to cannabis I would agree, this appears to have been the specific intent! — but I was thinking more of heroin when I wrote that, and heroin is something that even my most psychonautic acquaintance will not touch even once. But, in certain cases, it is not only drugs which can be demonstrably an addiction, a condition where a person has already lost agency. This can happen in other cases, and the threshold is arbitrary and does indeed vary with public mores that aren't always coherent let alone correct. And I don't make any particular claim if in this case "addiction" is correct or not, only that it could be. [0] more broadly, this is also why Goodhart's law[1] is a thing: https://en.wikipedia.org/wiki/Simplex https://en.wikipedia.org/wiki/Simplex [1] https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law

dudeinjapan 4y ago

I don't understand why they lobotomized their product. Presumably they would make a lot more money by having satisfied customers. What did they stand to gain from the rug-pull here? I could understand if they jacked up their prices to exploitative levels, but that's not what they did.

pjc50 4y ago
There is a history of sex work attracting disproportionate, sudden, catastrophic legal responses. It carries a considerable amount of political and reputational risk. Now, you might reasonably argue that AI chat isn't actually sex work, or that since no actual women are involved in it the usual "anti trafficking" arguments don't hold. But the real question is: what happens when there's a moral panic, and someone digs up the most explicit, weirdest bits of Replika chats and reads them out in a congressional inquiry?
- nemomarx 4y ago
  Like with Tumblr, it probably has to do with age blocking or the lack of it. "Your AI solicited nudes from children!" is pretty large reputational damage, if someone could find even one example and present that.
- xp84 4y ago
  Bingo. Someone's 11-year-old, who has already learned most of the sex stuff on the schoolyard, will lie their way into account creation (as an "adult"), enable super explicit sexy mode, prime it with as much dirty stuff as they can imagine, and then screenshot 2 or 3 messages from the "AI" to them and send to friends for the lulz. Parents find it - call local TV station or make it go viral online with the narrative that sweet little <insert name> was innocently browsing excellent child-friendly content, probably PBS, and somehow was targeted by this perverted AI Bot which filled their brain with filthy thoughts, scarring them for life. 'They'll need at least $20 million dollars worth of therapy over their lifetime.'
braingravy 4y ago
They saw the writing on the wall: People who use the product, become emotionally entangled, and then attempt or complete suicide following their experience open up the company to litigation from the families.

deleted 4y ago

[deleted]