7 ms·

FBI Arrests Man for Generating AI Child Sexual Abuse Imagery

30 points by g1a55er 2y ago

jjgreen 2y ago
The title and story are rather misleading, he's been charged with “exposing a child to harmful material” and “sexual contact with a child under age 13" rather than for generating the images per-se.
- meowface 2y ago
  It's confusing because that was also my interpretation until I saw the quote from the DoJ prosecutor in the article: >“Today’s announcement sends a clear message: using AI to produce sexually explicit depictions of children is illegal, and the Justice Department will not hesitate to hold accountable those who possess, produce, or distribute AI-generated child sexual abuse material"
  - squarefoot 2y ago
    I completely understand forbidding the distribution to minors, but how can they hold accountable someone for producing CSAM and keeping all data in their hard drive? This sets a dangerous precedent; how is it different from simulating a murder, or simply speeding in a videogame? That "or distribute" should really be "and distribute"; the whole point should be to protect kids, not to punish people that are indeed fucked up but found a way to not harm the society.
    - Sporktacular 2y ago
      "Not harm society" will depend on what material the models are trained with.
      - roenxi 2y ago
        Running some order of magnitudes, we've had people abusing children and taking photos of it since around 1850. By this point there is a large enough corpus of whatever horrors they want to look at that a model can be trained without doing any further harm. It is a total waste of taxpayer funds to be going after someone if all they've done is generate however many images with stable diffusion. I'm glad in this case he's at least done something objectionable although an arrest seems a bit of an overreach, this seems to be the sort of situation that could be resolved with a fine rather than jail time based on skimming the indictment. If that.
        Sporktacular 2y ago
        How would policing that work exactly? Downloading only old CSAM is acceptable? Technically any child abuse material available was produced in the past, so you could argue any pedophile downloading any of it isn't 'doing any further harm'. But the harm in it comes from generating further demand and normalising the sexualization of children. And than can happen with any material from before or after 1850. If a model is trained on it, it becomes a derivative work that should be treated no differently than a b&w version of a colour abuse image. Not sure what tax dollars have to do with this but IMO it's a small price to pay for an important service.
        roenxi 2y ago
        > Technically any child abuse material available was produced in the past, so you could argue any pedophile downloading any of it isn't 'doing any further harm'. Sounds like I'd be happy to go with that. You're taking it to the logical conclusion and I generally agree. The problem to me is (1) people abusing children and (2) people paying other people to abuse children. > Not sure what tax dollars have to do with this but IMO it's a small price to pay for an important service. It isn't an important service. With the advent of generative AI models, going after people for possessing dodgy images is now more than likely persecuting people who haven't done any harm to anyone and aren't going to. We really should have learned this lesson after Turing. If people aren't harming anyone then it is OK for them to have bizarre sexual likes and dislikes.
      - s1artibartfast 2y ago
        how is that? the AI companies aren't commissioning new sexual abuse to train their models on.
- ghostpepper 2y ago
  That’s what the state police charged him with in March. The article doesn’t include the specific federal charges for some reason, but the FBI statement suggest they believe the generation itself is also illegal. > “Today’s announcement sends a clear message: using AI to produce sexually explicit depictions of children is illegal, and the Justice Department will not hesitate to hold accountable those who possess, produce, or distribute AI-generated child sexual abuse material,” Principal Deputy Assistant Attorney General Nicole M. Argentieri said in an FBI press release.
  - jjgreen 2y ago
    Sure, but what would the charge be? (I'm in the UK, here such is definitely illegal.)
    - pseudalopex 2y ago
      producing, distributing, and possessing obscene visual depictions of minors engaged in sexually explicit conduct[1][2] [1] https://www.justice.gov/opa/pr/man-arrested-producing-distributing-and-possessing-ai-generated-images-minors-engaged https://www.justice.gov/opa/pr/man-arrested-producing-distri... [2] https://www.law.cornell.edu/uscode/text/18/1466A https://www.law.cornell.edu/uscode/text/18/1466A
      - jjgreen 2y ago
        Not a lawyer, but [2] indicate circumstance in section (d), that involves distribution; production without distribution would not satisfy that (?)
        g1a55er 2y ago
        (Also not a lawyer) The point of (d) is to create a jurisdictional hook for the US federal government. The federal government only has power over a limited number of offenses, including "interstate commerce" related offenses, but not most normal criminal offenses, which can only be criminalized by the states. So, for the federal government to regulate it, the easiest way is to only regulate things that involve interstate commerce somehow. In this case, downloading the Stable Diffusion model over the Internet probably creates enough of a hook, and this defendant is alleged to have done much more than that, so it's probably enough.
- deleted 2y ago
  [deleted]
- g1a55er 2y ago
  The first, second, and fourth charges in the indictment[1] is for violating 18 USC 1466A(a)(1)/(b)(1)/(d)(1)/(d)(4), which are the statutes against producing/distributing/possessing an "obscene" "drawing, cartoon, sculpture, or painting" "depict[ing] a minor engaging in sexually explicit conduct". [1] https://www.justice.gov/opa/media/1352606/dl?inline https://www.justice.gov/opa/media/1352606/dl?inline
- deleted 2y ago
  [deleted]
bitsinthesky 2y ago
How is pedophilia considered psychologically? Is it a treatable condition, an untreatable permanent condition, or simply a criminal choice?
- pseudalopex 2y ago
  Pedophilia is a paraphilia. Most psychologists believe paraphilias are permanent. Therapy and medication can help people control their urges if needed. Child sex abuse is a criminal choice.
  - devindotcom 2y ago
    Yeah, I had to work with SOs in my previous job and this sounds accurate. It's very hard to characterize this kind of thing though so if anyone else is curious it would be best to read up a bit on modern approaches.
- dudul 2y ago
  As of now there is no proof that it can be cured. Most treatments and therapies are to help pedophiles control their urges and not act. At the end of the day, it's a paraphilia. I don't think you can cure someone who's aroused by feet or uniforms. I don't believe there is any evidence today that you can alter people's sexual interests.
deleted 2y ago
[deleted]
Mathnerd314 2y ago
I guess there are two cases, a Wisconsin state case (March, out on bail, still pending) and this federal case. Apparently it is not double jeopardy to re-arrest for the same thing (or different statutes, in this case) if one is state and one federal.
delichon 2y ago
With an image of a real scene, I suppose it would be a defense against a child porn charge to prove that the participants were not minors. But there could be no such defense if the porn is generated. It becomes a matter for judges and juries to decide if a given picture of a person is over or under a particular age. I wonder if it would be a defense to prove that the prompt did not designate an illegal age. It may be a useful liability hedge to provably include the prompt in generated content, like as a watermark. But since existing generation isn't repeatable, and the prompt can't reliably generate the same image, is that even possible?
- ivanjermakov 2y ago
  If innocent prompt would generate that, then it's responsibility of the LLM model authors.
0xDEADFED5 2y ago
it seems to me like we're missing an opportunity to give these people a victim-free outlet for their urges. i'm not really educated on what treatment looks like for these people, and harms to society from AI CSAM should obviously be considered too. but some kind of "prescription" or otherwise regulated generation of these images should at least be considered, i think.
- Sporktacular 2y ago
  It's not victimless if they use real images to train their AI models.
  - qurl 2y ago
    Controversial images like this, such as Taylor Swift deepfakes, can be generated without training on any related explicit material.
Sporktacular 2y ago
"Today’s announcement sends a clear message: using AI to produce sexually explicit depictions of children is illegal" Not with the charges given. Prosecutors seem keen to push the normalisation angle, but would then have to explain why FPS games aren't also illegal. Unless they can prove real CSAM was used to train the model so they can draw a line from the material to the defendant's production. Let's see if there are more charges to come.
underseacables 2y ago
So....what does this mean for Hentai and Yaoi??

30 points by g1a55er 2y ago

The title and story are rather misleading, he's been charged with “exposing a child to harmful material” and “sexual contact with a child under age 13" rather than for generating the images per-se.

meowface 2y ago
It's confusing because that was also my interpretation until I saw the quote from the DoJ prosecutor in the article: >“Today’s announcement sends a clear message: using AI to produce sexually explicit depictions of children is illegal, and the Justice Department will not hesitate to hold accountable those who possess, produce, or distribute AI-generated child sexual abuse material"
- squarefoot 2y ago
  I completely understand forbidding the distribution to minors, but how can they hold accountable someone for producing CSAM and keeping all data in their hard drive? This sets a dangerous precedent; how is it different from simulating a murder, or simply speeding in a videogame? That "or distribute" should really be "and distribute"; the whole point should be to protect kids, not to punish people that are indeed fucked up but found a way to not harm the society.
  - Sporktacular 2y ago
    "Not harm society" will depend on what material the models are trained with.
    - roenxi 2y ago
      Running some order of magnitudes, we've had people abusing children and taking photos of it since around 1850. By this point there is a large enough corpus of whatever horrors they want to look at that a model can be trained without doing any further harm. It is a total waste of taxpayer funds to be going after someone if all they've done is generate however many images with stable diffusion. I'm glad in this case he's at least done something objectionable although an arrest seems a bit of an overreach, this seems to be the sort of situation that could be resolved with a fine rather than jail time based on skimming the indictment. If that.
      - Sporktacular 2y ago
        How would policing that work exactly? Downloading only old CSAM is acceptable? Technically any child abuse material available was produced in the past, so you could argue any pedophile downloading any of it isn't 'doing any further harm'. But the harm in it comes from generating further demand and normalising the sexualization of children. And than can happen with any material from before or after 1850. If a model is trained on it, it becomes a derivative work that should be treated no differently than a b&w version of a colour abuse image. Not sure what tax dollars have to do with this but IMO it's a small price to pay for an important service.
        roenxi 2y ago
        > Technically any child abuse material available was produced in the past, so you could argue any pedophile downloading any of it isn't 'doing any further harm'. Sounds like I'd be happy to go with that. You're taking it to the logical conclusion and I generally agree. The problem to me is (1) people abusing children and (2) people paying other people to abuse children. > Not sure what tax dollars have to do with this but IMO it's a small price to pay for an important service. It isn't an important service. With the advent of generative AI models, going after people for possessing dodgy images is now more than likely persecuting people who haven't done any harm to anyone and aren't going to. We really should have learned this lesson after Turing. If people aren't harming anyone then it is OK for them to have bizarre sexual likes and dislikes.
    - s1artibartfast 2y ago
      how is that? the AI companies aren't commissioning new sexual abuse to train their models on.
ghostpepper 2y ago
That’s what the state police charged him with in March. The article doesn’t include the specific federal charges for some reason, but the FBI statement suggest they believe the generation itself is also illegal. > “Today’s announcement sends a clear message: using AI to produce sexually explicit depictions of children is illegal, and the Justice Department will not hesitate to hold accountable those who possess, produce, or distribute AI-generated child sexual abuse material,” Principal Deputy Assistant Attorney General Nicole M. Argentieri said in an FBI press release.
- jjgreen 2y ago
  Sure, but what would the charge be? (I'm in the UK, here such is definitely illegal.)
  - pseudalopex 2y ago
    producing, distributing, and possessing obscene visual depictions of minors engaged in sexually explicit conduct[1][2] [1] https://www.justice.gov/opa/pr/man-arrested-producing-distributing-and-possessing-ai-generated-images-minors-engaged https://www.justice.gov/opa/pr/man-arrested-producing-distri... [2] https://www.law.cornell.edu/uscode/text/18/1466A https://www.law.cornell.edu/uscode/text/18/1466A
    - jjgreen 2y ago
      Not a lawyer, but [2] indicate circumstance in section (d), that involves distribution; production without distribution would not satisfy that (?)
      - g1a55er 2y ago
        (Also not a lawyer) The point of (d) is to create a jurisdictional hook for the US federal government. The federal government only has power over a limited number of offenses, including "interstate commerce" related offenses, but not most normal criminal offenses, which can only be criminalized by the states. So, for the federal government to regulate it, the easiest way is to only regulate things that involve interstate commerce somehow. In this case, downloading the Stable Diffusion model over the Internet probably creates enough of a hook, and this defendant is alleged to have done much more than that, so it's probably enough.
deleted 2y ago
[deleted]
g1a55er 2y ago
The first, second, and fourth charges in the indictment[1] is for violating 18 USC 1466A(a)(1)/(b)(1)/(d)(1)/(d)(4), which are the statutes against producing/distributing/possessing an "obscene" "drawing, cartoon, sculpture, or painting" "depict[ing] a minor engaging in sexually explicit conduct". [1] https://www.justice.gov/opa/media/1352606/dl?inline https://www.justice.gov/opa/media/1352606/dl?inline
deleted 2y ago
[deleted]

bitsinthesky 2y ago

How is pedophilia considered psychologically? Is it a treatable condition, an untreatable permanent condition, or simply a criminal choice?

pseudalopex 2y ago
Pedophilia is a paraphilia. Most psychologists believe paraphilias are permanent. Therapy and medication can help people control their urges if needed. Child sex abuse is a criminal choice.
- devindotcom 2y ago
  Yeah, I had to work with SOs in my previous job and this sounds accurate. It's very hard to characterize this kind of thing though so if anyone else is curious it would be best to read up a bit on modern approaches.
dudul 2y ago
As of now there is no proof that it can be cured. Most treatments and therapies are to help pedophiles control their urges and not act. At the end of the day, it's a paraphilia. I don't think you can cure someone who's aroused by feet or uniforms. I don't believe there is any evidence today that you can alter people's sexual interests.

deleted 2y ago

[deleted]

Mathnerd314 2y ago

I guess there are two cases, a Wisconsin state case (March, out on bail, still pending) and this federal case. Apparently it is not double jeopardy to re-arrest for the same thing (or different statutes, in this case) if one is state and one federal.

delichon 2y ago

With an image of a real scene, I suppose it would be a defense against a child porn charge to prove that the participants were not minors. But there could be no such defense if the porn is generated. It becomes a matter for judges and juries to decide if a given picture of a person is over or under a particular age. I wonder if it would be a defense to prove that the prompt did not designate an illegal age. It may be a useful liability hedge to provably include the prompt in generated content, like as a watermark. But since existing generation isn't repeatable, and the prompt can't reliably generate the same image, is that even possible?

ivanjermakov 2y ago
If innocent prompt would generate that, then it's responsibility of the LLM model authors.

0xDEADFED5 2y ago

it seems to me like we're missing an opportunity to give these people a victim-free outlet for their urges. i'm not really educated on what treatment looks like for these people, and harms to society from AI CSAM should obviously be considered too. but some kind of "prescription" or otherwise regulated generation of these images should at least be considered, i think.

Sporktacular 2y ago
It's not victimless if they use real images to train their AI models.
- qurl 2y ago
  Controversial images like this, such as Taylor Swift deepfakes, can be generated without training on any related explicit material.

Sporktacular 2y ago

"Today’s announcement sends a clear message: using AI to produce sexually explicit depictions of children is illegal" Not with the charges given. Prosecutors seem keen to push the normalisation angle, but would then have to explain why FPS games aren't also illegal. Unless they can prove real CSAM was used to train the model so they can draw a line from the material to the defendant's production. Let's see if there are more charges to come.

underseacables 2y ago

So....what does this mean for Hentai and Yaoi??