4 ms·

> service generated illegal imagery For my own curiosity, could you please elaborate which law was broken? And if you do find one, elaborate how is it not unco

by dmitrygr 10mo ago

> service generated illegal imagery For my own curiosity, could you please elaborate which law was broken? And if you do find one, elaborate how is it not unconstitutional, given that “put X into a bikini” is constitutionally protected speech, and the output fails the Roth test for obscenity and is thus protected just the same. Would not every other law claiming to be relevant be null and void w.r.t. stoping this in this particular case? SCOTUS was quite clear in Roth v. United States. The later Miller case is also of no help, since nowadays appearing publicly in a bikini is considered quite normal and not at all obscene or sexual. I get it. If this happened to a member of my family, I'd be pissed too. But, as logical people is it not our job to NOT simply leap from "I'm very angry" to "It must be illegal", but to be calm and rational.

labrador 10mo ago
It's illegal in California.
- dmitrygr 10mo ago
  > It's illegal in California. 1. What specifically is? Posting twitter comments? Prompting an ML model? Generating images based on existing images? 2. Even if so, states are not allowed to make unconstitutional laws. 1stA covers this, as far as I can tell. Do you read Roth or Miller differently?
  - labrador 10mo ago
    "is posting nudes of women without their consent legal in california" https://www.google.com/search?q=is+posting+nudes+of+women+without+their+consent+legal+in+california https://www.google.com/search?q=is+posting+nudes+of+women+wi...
    - xigoi 10mo ago
      Posting such images is illegal, i.e. it’s the user who has violated the law.
      - Timon3 10mo ago
        But the user isn't posting the image, Grok is. The user can't even delete the image if Grok generated something the user doesn't want.
        xigoi 10mo ago
        Ah, that’s a terrible implementation then.
  - kccoder 10mo ago
    It’s seems as though you’re being deliberately obtuse. It’s obvious why this is illegal.
- kemayo 10mo ago
  Not just California. Non-consensual deepfake porn is explicitly federally illegal now. Ted Cruz's "Take It Down" Act passed last year: https://www.congress.gov/bill/119th-congress/senate-bill/146 https://www.congress.gov/bill/119th-congress/senate-bill/146 (This may or may not be constitutional in the end, but it's certainly current law on the books.)
text0404 10mo ago
> “put X into a bikini” is constitutionally protected speech Unfortunately if "put X into a bikini" is a command to an AI to generate CSAM of an underage child, that's no longer "constitutionally protected" territory. Also, these are global investigations - most countries don't have the same approach to freedom of speech as the US.
- ndsipa_pomu 10mo ago
  > Also, these are global investigations - most countries don't have the same approach to freedom of speech as the US It's peculiar to me how people in the U.S. seem to prioritise "freedom of speech" and de-prioritise freedom from being harassed online. In general, I support free speech, but I don't take an absolutist position as there can be a lot of harm created by allowing lies to propagate unrestricted (c.f. Brexit in the UK). Meanwhile, there's direct emotional harm being caused by these AI images being posted and also a speech chilling effect by making people, especially women, afraid of posting on contentious topics.
soraminazuki 10mo ago
Grok is generating content of real woman and children being undressed, raped, humiliated, hurt, or killed. I wonder what's driving some people to downplay and defend these monstrous actions. "Calm down! Not only is this quite normal, it's a constitutional act." The reality distortion field is very strong this past year or two when it comes to AI-related or wrongfully flagged threads. - https://futurism.com/future-society/grok-violence-women https://futurism.com/future-society/grok-violence-women - https://www.reuters.com/legal/litigation/grok-says-safeguard-lapses-led-images-minors-minimal-clothing-x-2026-01-02/ https://www.reuters.com/legal/litigation/grok-says-safeguard... - https://www.rollingstone.com/culture/culture-news/elon-musk-grok-rape-fantasies-1235381746/ https://www.rollingstone.com/culture/culture-news/elon-musk-...
techblueberry 10mo ago
There’s and established set and growing number of deepfake and revenge porn laws. Targeted Harassment is not protected speech. Also, I imagine intent would come into play. “All I did was post some text, is it my fault grok generated kiddie porn” sounds similar to “all I did was undo the ebrake on my car, how was I know it would roll over onto that person”. If a reasonable person could predict a crime would occur, usually that’s a crime.

labrador 10mo ago

It's illegal in California.

dmitrygr 10mo ago
> It's illegal in California. 1. What specifically is? Posting twitter comments? Prompting an ML model? Generating images based on existing images? 2. Even if so, states are not allowed to make unconstitutional laws. 1stA covers this, as far as I can tell. Do you read Roth or Miller differently?
- labrador 10mo ago
  "is posting nudes of women without their consent legal in california" https://www.google.com/search?q=is+posting+nudes+of+women+without+their+consent+legal+in+california https://www.google.com/search?q=is+posting+nudes+of+women+wi...
  - xigoi 10mo ago
    Posting such images is illegal, i.e. it’s the user who has violated the law.
    - Timon3 10mo ago
      But the user isn't posting the image, Grok is. The user can't even delete the image if Grok generated something the user doesn't want.
      - xigoi 10mo ago
        Ah, that’s a terrible implementation then.
- kccoder 10mo ago
  It’s seems as though you’re being deliberately obtuse. It’s obvious why this is illegal.
kemayo 10mo ago
Not just California. Non-consensual deepfake porn is explicitly federally illegal now. Ted Cruz's "Take It Down" Act passed last year: https://www.congress.gov/bill/119th-congress/senate-bill/146 https://www.congress.gov/bill/119th-congress/senate-bill/146 (This may or may not be constitutional in the end, but it's certainly current law on the books.)

text0404 10mo ago

> “put X into a bikini” is constitutionally protected speech Unfortunately if "put X into a bikini" is a command to an AI to generate CSAM of an underage child, that's no longer "constitutionally protected" territory. Also, these are global investigations - most countries don't have the same approach to freedom of speech as the US.

ndsipa_pomu 10mo ago
> Also, these are global investigations - most countries don't have the same approach to freedom of speech as the US It's peculiar to me how people in the U.S. seem to prioritise "freedom of speech" and de-prioritise freedom from being harassed online. In general, I support free speech, but I don't take an absolutist position as there can be a lot of harm created by allowing lies to propagate unrestricted (c.f. Brexit in the UK). Meanwhile, there's direct emotional harm being caused by these AI images being posted and also a speech chilling effect by making people, especially women, afraid of posting on contentious topics.

soraminazuki 10mo ago

Grok is generating content of real woman and children being undressed, raped, humiliated, hurt, or killed. I wonder what's driving some people to downplay and defend these monstrous actions. "Calm down! Not only is this quite normal, it's a constitutional act." The reality distortion field is very strong this past year or two when it comes to AI-related or wrongfully flagged threads. - https://futurism.com/future-society/grok-violence-women https://futurism.com/future-society/grok-violence-women - https://www.reuters.com/legal/litigation/grok-says-safeguard-lapses-led-images-minors-minimal-clothing-x-2026-01-02/ https://www.reuters.com/legal/litigation/grok-says-safeguard... - https://www.rollingstone.com/culture/culture-news/elon-musk-grok-rape-fantasies-1235381746/ https://www.rollingstone.com/culture/culture-news/elon-musk-...

techblueberry 10mo ago

There’s and established set and growing number of deepfake and revenge porn laws. Targeted Harassment is not protected speech. Also, I imagine intent would come into play. “All I did was post some text, is it my fault grok generated kiddie porn” sounds similar to “all I did was undo the ebrake on my car, how was I know it would roll over onto that person”. If a reasonable person could predict a crime would occur, usually that’s a crime.