7 ms·

Teens sue xAI over Grok's pornographic images of them

146 points by 1659447091 7mo ago

paxys 7mo ago
I bet shareholders of SpaceX are thrilled to be exposed to this for no reason
- manoDev 7mo ago
  You would lose this bet.
- bigyabai 7mo ago
  SpaceX owners do not care. If they were risk-averse, they would have dumped SpaceX like it was toxic waste. In a broader sense, this "I bet Oracle shareholders hate their bad PR" attitude is really zero-sum. It's pervasive on HN, and we rarely ever see bad PR snowball beyond niche discussions. I want $BIGCORP to collapse as much as the next guy, but the outrage-derived comments don't seem to reflect the market's response.
  - rsynnott 7mo ago
    I mean, there's "this is a risky investment in rockets and stuff" and then there is "this is a risky investment in rockets and stuff which has for some reason been coupled to a CSAM generator". One is going to be more off-putting to investors than the other.
    - bigyabai 7mo ago
      I'll actually steelman against this. For the majority of SpaceX holders, they really do not care. They did not sell their stock when SpaceX started taking NRO contracts. They didn't grow a conscience when Starlink usage was limited in occupied Ukrainian territory. They aren't protesting rocket launches that put Israeli satellites into orbit, aiding and abetting a likely genocide of civilian populations. You think fake child abuse imagery will finally burst the dam? After all of this? Like I said; some companies revel in their bad PR. Meta, Oracle, Microsoft, all of their misdeeds are untouchable. SpaceX isn't going anywhere, and everyone holding their shares knows it.
      - rsynnott 7mo ago
        I mean, even for the completely amoral investor, they would still have to think about whether the magic abomination generator will _make money_. I would imagine that the Venn diagram of people who think that SpaceX will make loadsa money and those who think that whatever-the-hell Twitter is now will make loadsa money is not a circle.
        bigyabai 7mo ago
        Making money off of the proverbial Torment Nexus is SpaceX' specialty. They do it pretty well; the US fed loves their rockets, and feeds them contracts with margins that could finance a dozen suburbs. If X can pull off a similar pivot to being a federal lapdog, it would cost SpaceX investors nothing and expand the scope of their service contracts. FWIW, I agree that "fake CSAM isn't real risk exposure" is a pretty crazy line to steelman. But we live in crazy times. SpaceX is an antifragile asset, and anything that enhances it's risk is seen as a value-add. This is exactly what America deserves for refusing to regulate our monopolies while they play the futures trade like Candy Land.
charcircuit 7mo ago
[flagged]
- catgirlinspace 7mo ago
  does photoshop have a “make porn of this person” button? does it have a “make me csam” button?
  - nofriend 7mo ago
    It's called the "brush" tool.
  - charcircuit 7mo ago
    Nor did Grok. You had to prompt what you wanted to have happen.
    - happytoexplain 7mo ago
      Typing a description of an image is comparable to finding a button in a menu regarding ease/enablement. The difference is actually that Grok decides explicitly to not add even simple (if imperfect) guards, which requires no real burden on their part, which is morally and spiritually hideous. I hope they experience consequences.
      - charcircuit 7mo ago
        It really is not the same at all. One is purely from the users mind and the other is an explicit part of the product. Photoshop could also add simple guards too, but this type of authoritarian is not appreciated by users. >I hope they experience consequences. I hope people who want a censored model can use one and people who want uncensored models can use uncensored ones.
    - jdiff 7mo ago
      And it had blatantly insufficient safety guarding that. I saw photos of 4th graders, labeled accordingly, that Grok happily modified. Given the model does have access to the text content of posts it's replying to, that's wildly negligent. This is not a simple brush tool we're talking about, and refusing requests to strip clothes off people is one of the most basic measures that could be taken here.
coolguysailer 7mo ago
I'm kind of curious what precedent this will set. It's pretty easy to create deepfake sexual content already and has been for years now. The grok thing is absurdly easy though on some level. To actually get full blown sexual content though I think is substantially more difficult and probably falls into the realm of hacking almost, that being said, it is of course possible. But it makes me wonder where the line lives for something like this. The people who did this are obviously scum of course and deserve to be punished, but by that argument facebook/instagram/whatsapp/whatever group-chats of leaked sex tapes have probably done far more damage.
- rawgabbit 7mo ago
  In the past, you could be fined and jailed for violating a community's standard for obscene material. With global companies, their multinational status doesn't stop a small country from issuing arrest warrants. Whether they will be carried out is another matter.
josefritzishere 7mo ago
This is a legit tech topic. I disagree with the flagging. Legal and responsible technology use is a material area of discussion.
- drcongo 7mo ago
  But poor, helpless little Elon must be protected from mean comments at all costs.
- ndsipa_pomu 7mo ago
  I don't understand why it was flagged unless it's just someone flagging anything negative about Musk related projects (which I don't think it is at all a valid reason).
  - thrance 7mo ago
    That's exactly it. Everything negative about Musk gets flagged and downvoted, and the moderation doesn't care about that.
    - Schmerika 7mo ago
      Honestly it's not even that moderation doesn't care. Garry Tan and PG were big DOGE fans, and I'm not silly enough to believe that the moderation are unaware of that fact. There's been far too many flagged Musk stories that never get unflagged to put it down to simple incompetence and neglect.
      - rsynnott 7mo ago
        I mean, honestly, "HN voting is poorly designed and is vulnerable to coordinated action" would explain it fairly neatly; I'd totally buy that this is incompetence vs malice by the operators.
        Schmerika 7mo ago
        With respect, if you believe that then you likely aren't aware of the true scale of the problem. Unless they follow /active, /new, and have [showdead] on, people have very little idea just how much is getting suppressed here... And it's a lot, especially this past year, and extremely especially if it's anything Musk-related. Another reason most people here don't know how bad it's got is because posts about the poor flagging system are explicitly disallowed. For that reason alone it's quite hard to say this is purely incompetence with no malice. You can ask mods for permission to make a post about the flagging system - you'll be denied. People have tried making posts about all the Musk stories being flagged - flagged by moderation. Every time. Quickly. So, no. Unfortunately, this isn't a case where I'm happy to assume incompetence. Look at where we're at - a post about the richest tech bro in the world enabling mass creep shit of the worst kind remains flagged for hours. That's absolutely vile - and far from a rare event here.
DocTomoe 7mo ago
Missing from such articles is a key information: who paid for the lawyers? I doubt teenage income would pay for a retainer. As so often, following the money will reveal who actually has an interest here.
- thrance 7mo ago
  > As so often, following the money will reveal who actually has an interest here. Do you mean to imply that the teens are a tool used by some secret cabal to attack Musk? Most likely their parents are paying for the lawyers in an attempt to get some justice from a billionaire that enabled a fucked up case of CSAM.
  - ndsipa_pomu 7mo ago
    More than just enabling a fucked up case of CSAM, it's more like an energy intensive, environment destroying factory that produces CSAM. Once again, I'm asking when we can start the Butlerian Jihad? Incidentally, is the USA now becoming a safe haven for pedophiles? It just seems odd how other countries are investigating high profile people with ties to Epstein, but the USA celebrates them and even votes the biggest offender into the highest office.
- ndsipa_pomu 7mo ago
  I think it is in almost everyone's interest to ensure that people cannot be abused in such a fashion. Maybe people who wish to destroy society would condone this kind of abuse. It's also possible for lawyers to take on cases on the understanding that they only get paid if they win.
- Anthony-G 7mo ago
  Occam’s razor would suggest that it’s the same group of active natalists who subsidise the teenagers’ housing, education and healthcare – none of which are any cheaper than lawyers.
- rsynnott 7mo ago
  In the US, lawyers may generally work on a contingency basis (https://en.wikipedia.org/wiki/Contingent_fee https://en.wikipedia.org/wiki/Contingent_fee), and many lawyers and law firms also do pro bono work. That is, almost certainly no-one has paid for the lawyers, though if they manage to recover anything the lawyers may be due a cut.
ndsipa_pomu 7mo ago
As there exist laws to protect against someone abusing someone's name/reputation (i.e. libel and slander), there should also be laws to protect against abusing someone's image. It does not seem reasonable for people/companies to be allowed to manipulate an image of someone to misrepresent a situation.

146 points by 1659447091 7mo ago

paxys 7mo ago

I bet shareholders of SpaceX are thrilled to be exposed to this for no reason

manoDev 7mo ago
You would lose this bet.
bigyabai 7mo ago
SpaceX owners do not care. If they were risk-averse, they would have dumped SpaceX like it was toxic waste. In a broader sense, this "I bet Oracle shareholders hate their bad PR" attitude is really zero-sum. It's pervasive on HN, and we rarely ever see bad PR snowball beyond niche discussions. I want $BIGCORP to collapse as much as the next guy, but the outrage-derived comments don't seem to reflect the market's response.
- rsynnott 7mo ago
  I mean, there's "this is a risky investment in rockets and stuff" and then there is "this is a risky investment in rockets and stuff which has for some reason been coupled to a CSAM generator". One is going to be more off-putting to investors than the other.
  - bigyabai 7mo ago
    I'll actually steelman against this. For the majority of SpaceX holders, they really do not care. They did not sell their stock when SpaceX started taking NRO contracts. They didn't grow a conscience when Starlink usage was limited in occupied Ukrainian territory. They aren't protesting rocket launches that put Israeli satellites into orbit, aiding and abetting a likely genocide of civilian populations. You think fake child abuse imagery will finally burst the dam? After all of this? Like I said; some companies revel in their bad PR. Meta, Oracle, Microsoft, all of their misdeeds are untouchable. SpaceX isn't going anywhere, and everyone holding their shares knows it.
    - rsynnott 7mo ago
      I mean, even for the completely amoral investor, they would still have to think about whether the magic abomination generator will _make money_. I would imagine that the Venn diagram of people who think that SpaceX will make loadsa money and those who think that whatever-the-hell Twitter is now will make loadsa money is not a circle.
      - bigyabai 7mo ago
        Making money off of the proverbial Torment Nexus is SpaceX' specialty. They do it pretty well; the US fed loves their rockets, and feeds them contracts with margins that could finance a dozen suburbs. If X can pull off a similar pivot to being a federal lapdog, it would cost SpaceX investors nothing and expand the scope of their service contracts. FWIW, I agree that "fake CSAM isn't real risk exposure" is a pretty crazy line to steelman. But we live in crazy times. SpaceX is an antifragile asset, and anything that enhances it's risk is seen as a value-add. This is exactly what America deserves for refusing to regulate our monopolies while they play the futures trade like Candy Land.

charcircuit 7mo ago

[flagged]

catgirlinspace 7mo ago
does photoshop have a “make porn of this person” button? does it have a “make me csam” button?
- nofriend 7mo ago
  It's called the "brush" tool.
- charcircuit 7mo ago
  Nor did Grok. You had to prompt what you wanted to have happen.
  - happytoexplain 7mo ago
    Typing a description of an image is comparable to finding a button in a menu regarding ease/enablement. The difference is actually that Grok decides explicitly to not add even simple (if imperfect) guards, which requires no real burden on their part, which is morally and spiritually hideous. I hope they experience consequences.
    - charcircuit 7mo ago
      It really is not the same at all. One is purely from the users mind and the other is an explicit part of the product. Photoshop could also add simple guards too, but this type of authoritarian is not appreciated by users. >I hope they experience consequences. I hope people who want a censored model can use one and people who want uncensored models can use uncensored ones.
  - jdiff 7mo ago
    And it had blatantly insufficient safety guarding that. I saw photos of 4th graders, labeled accordingly, that Grok happily modified. Given the model does have access to the text content of posts it's replying to, that's wildly negligent. This is not a simple brush tool we're talking about, and refusing requests to strip clothes off people is one of the most basic measures that could be taken here.

coolguysailer 7mo ago

I'm kind of curious what precedent this will set. It's pretty easy to create deepfake sexual content already and has been for years now. The grok thing is absurdly easy though on some level. To actually get full blown sexual content though I think is substantially more difficult and probably falls into the realm of hacking almost, that being said, it is of course possible. But it makes me wonder where the line lives for something like this. The people who did this are obviously scum of course and deserve to be punished, but by that argument facebook/instagram/whatsapp/whatever group-chats of leaked sex tapes have probably done far more damage.

rawgabbit 7mo ago
In the past, you could be fined and jailed for violating a community's standard for obscene material. With global companies, their multinational status doesn't stop a small country from issuing arrest warrants. Whether they will be carried out is another matter.

josefritzishere 7mo ago

This is a legit tech topic. I disagree with the flagging. Legal and responsible technology use is a material area of discussion.

drcongo 7mo ago
But poor, helpless little Elon must be protected from mean comments at all costs.
ndsipa_pomu 7mo ago
I don't understand why it was flagged unless it's just someone flagging anything negative about Musk related projects (which I don't think it is at all a valid reason).
- thrance 7mo ago
  That's exactly it. Everything negative about Musk gets flagged and downvoted, and the moderation doesn't care about that.
  - Schmerika 7mo ago
    Honestly it's not even that moderation doesn't care. Garry Tan and PG were big DOGE fans, and I'm not silly enough to believe that the moderation are unaware of that fact. There's been far too many flagged Musk stories that never get unflagged to put it down to simple incompetence and neglect.
    - rsynnott 7mo ago
      I mean, honestly, "HN voting is poorly designed and is vulnerable to coordinated action" would explain it fairly neatly; I'd totally buy that this is incompetence vs malice by the operators.
      - Schmerika 7mo ago
        With respect, if you believe that then you likely aren't aware of the true scale of the problem. Unless they follow /active, /new, and have [showdead] on, people have very little idea just how much is getting suppressed here... And it's a lot, especially this past year, and extremely especially if it's anything Musk-related. Another reason most people here don't know how bad it's got is because posts about the poor flagging system are explicitly disallowed. For that reason alone it's quite hard to say this is purely incompetence with no malice. You can ask mods for permission to make a post about the flagging system - you'll be denied. People have tried making posts about all the Musk stories being flagged - flagged by moderation. Every time. Quickly. So, no. Unfortunately, this isn't a case where I'm happy to assume incompetence. Look at where we're at - a post about the richest tech bro in the world enabling mass creep shit of the worst kind remains flagged for hours. That's absolutely vile - and far from a rare event here.

DocTomoe 7mo ago

Missing from such articles is a key information: who paid for the lawyers? I doubt teenage income would pay for a retainer. As so often, following the money will reveal who actually has an interest here.

thrance 7mo ago
> As so often, following the money will reveal who actually has an interest here. Do you mean to imply that the teens are a tool used by some secret cabal to attack Musk? Most likely their parents are paying for the lawyers in an attempt to get some justice from a billionaire that enabled a fucked up case of CSAM.
- ndsipa_pomu 7mo ago
  More than just enabling a fucked up case of CSAM, it's more like an energy intensive, environment destroying factory that produces CSAM. Once again, I'm asking when we can start the Butlerian Jihad? Incidentally, is the USA now becoming a safe haven for pedophiles? It just seems odd how other countries are investigating high profile people with ties to Epstein, but the USA celebrates them and even votes the biggest offender into the highest office.
ndsipa_pomu 7mo ago
I think it is in almost everyone's interest to ensure that people cannot be abused in such a fashion. Maybe people who wish to destroy society would condone this kind of abuse. It's also possible for lawyers to take on cases on the understanding that they only get paid if they win.
Anthony-G 7mo ago
Occam’s razor would suggest that it’s the same group of active natalists who subsidise the teenagers’ housing, education and healthcare – none of which are any cheaper than lawyers.
rsynnott 7mo ago
In the US, lawyers may generally work on a contingency basis (https://en.wikipedia.org/wiki/Contingent_fee https://en.wikipedia.org/wiki/Contingent_fee), and many lawyers and law firms also do pro bono work. That is, almost certainly no-one has paid for the lawyers, though if they manage to recover anything the lawyers may be due a cut.

ndsipa_pomu 7mo ago

As there exist laws to protect against someone abusing someone's name/reputation (i.e. libel and slander), there should also be laws to protect against abusing someone's image. It does not seem reasonable for people/companies to be allowed to manipulate an image of someone to misrepresent a situation.