5 ms·

Wasn't this entirely predictable and inevitable? The genie is out of the bottle. Where can we realistically draw the line? Preventing distribution of this sort

by elric 9mo ago

Wasn't this entirely predictable and inevitable? The genie is out of the bottle. Where can we realistically draw the line? Preventing distribution of this sort of shit is impossible, anyone can run their own generator. CSAM is already banned pretty much everywhere, and making money off it certainly is, but somehow Musk is getting away with distributing it at a massive scale. Is it because it's fake? And can we even tell whether it's still fake? Do we ban profiting from fake porn? Do we ban computing? Do we ban unregulated access to generative AI? X/Grok is an attractive obvious target because it's so heinous and widespread, but putting the axe on them won't make much of a difference.

Jordan-117 9mo ago
How about we start at "not enabling users to directly generate nonconsensual porn of other users using your platform and then posting it as a reply to their content"?
- nailer 9mo ago
  Does any platform prevent that? HN doesn’t.
  - roryirvine 9mo ago
    HN will ban accounts that do that. Grok hasn't been banned from Twitter.
    - nailer 9mo ago
      Would you (and others) be happier if when someone asked Grok to post something, it came from the person's own account? If so we agree.
  - Jordan-117 9mo ago
    I said generating it using the platform. These creeps aren't just freelancing -- the images are being created by Twitter's servers via their native UI and published directly by their official @Grok account. They are utterly complicit in a way they wouldn't be if they were "just" permitting people to post AI revenge porn they created independently.
johnnyanmac 9mo ago
>CSAM is already banned pretty much everywhere, and making money off it certainly is, but somehow Musk is getting away with distributing it at a massive scale. Is it because it's fake? It's because law is slow and right now the US government is completely stalled out in terms of performing its job (thanks in part to Musk himself). Things will eventually catch up but it's simply the wild west for the next few years.
- elric 9mo ago
  Is government intervention even necessary? IANAL, and I don't know shit about US law, but if this this crap on X is illegal, then surely the courts can handle this and ban/fine/jail the responsible parties? If this isn't illegal, then sure, government intervention will be required, laws will have to be amended, etc. Until that happens, what are realistic options? Shaming the perps? A bit of hacktivism?
  - johnnyanmac 9mo ago
    The federal courts in the US are theoretically non-partisian. But 2025 has shown that the Department of Justice has functioned as Trump's personal legal counsel. That's even happening as we speak with the kerfuffle behind the Central Bank (another non-partisan organization that Trump is desperately trying to make partisan). As is, Musk probably isn't going to get confronted by this current DoJ. The state courts may try to take this up, but it has less reach than the federal courts. Other country's courts may take action and even ban X. >what are realistic options? Shaming the perps? A bit of hacktivism? Those can happen. I don't know how much it moves the needle, but those will be inevitable reactions. The only way out for the American people would be to mass boycott X over this, but our political activism has been fairly weak. Especially for software.
  - array_key_first 9mo ago
    The argument is X themselves are responsible, but they make it extremely easy and do the service for you. It's different. Like, if I sell a gun and you go and shoot someone I'm not necessarily responsible. Okay, makes sense. But if I run a shooting range and I give you zero training and don't even bother to put up walls, and someone gets shot, then I probably am responsible. That might mean something like Grok cannot realistically run at scale. I say good riddance and who cares.
asplake 9mo ago
What AI can generate, AI can detect. It is well within the power of the social media companies to deal with this stuff. It’s not crazy to hope that hitting X has a meaningful effect not only on them, but also the others.

by elric 9mo ago

Jordan-117 9mo ago

How about we start at "not enabling users to directly generate nonconsensual porn of other users using your platform and then posting it as a reply to their content"?

nailer 9mo ago
Does any platform prevent that? HN doesn’t.
- roryirvine 9mo ago
  HN will ban accounts that do that. Grok hasn't been banned from Twitter.
  - nailer 9mo ago
    Would you (and others) be happier if when someone asked Grok to post something, it came from the person's own account? If so we agree.
- Jordan-117 9mo ago
  I said generating it using the platform. These creeps aren't just freelancing -- the images are being created by Twitter's servers via their native UI and published directly by their official @Grok account. They are utterly complicit in a way they wouldn't be if they were "just" permitting people to post AI revenge porn they created independently.

johnnyanmac 9mo ago

>CSAM is already banned pretty much everywhere, and making money off it certainly is, but somehow Musk is getting away with distributing it at a massive scale. Is it because it's fake? It's because law is slow and right now the US government is completely stalled out in terms of performing its job (thanks in part to Musk himself). Things will eventually catch up but it's simply the wild west for the next few years.

elric 9mo ago
Is government intervention even necessary? IANAL, and I don't know shit about US law, but if this this crap on X is illegal, then surely the courts can handle this and ban/fine/jail the responsible parties? If this isn't illegal, then sure, government intervention will be required, laws will have to be amended, etc. Until that happens, what are realistic options? Shaming the perps? A bit of hacktivism?
- johnnyanmac 9mo ago
  The federal courts in the US are theoretically non-partisian. But 2025 has shown that the Department of Justice has functioned as Trump's personal legal counsel. That's even happening as we speak with the kerfuffle behind the Central Bank (another non-partisan organization that Trump is desperately trying to make partisan). As is, Musk probably isn't going to get confronted by this current DoJ. The state courts may try to take this up, but it has less reach than the federal courts. Other country's courts may take action and even ban X. >what are realistic options? Shaming the perps? A bit of hacktivism? Those can happen. I don't know how much it moves the needle, but those will be inevitable reactions. The only way out for the American people would be to mass boycott X over this, but our political activism has been fairly weak. Especially for software.
- array_key_first 9mo ago
  The argument is X themselves are responsible, but they make it extremely easy and do the service for you. It's different. Like, if I sell a gun and you go and shoot someone I'm not necessarily responsible. Okay, makes sense. But if I run a shooting range and I give you zero training and don't even bother to put up walls, and someone gets shot, then I probably am responsible. That might mean something like Grok cannot realistically run at scale. I say good riddance and who cares.

asplake 9mo ago

What AI can generate, AI can detect. It is well within the power of the social media companies to deal with this stuff. It’s not crazy to hope that hitting X has a meaningful effect not only on them, but also the others.