4 ms·

This is absolutely disturbing. While I fully advocate allocating resources to stop child sexual abuse and the pornographic material created during such crimes,

This is absolutely disturbing. While I fully advocate allocating resources to stop child sexual abuse and the pornographic material created during such crimes, no one was hurt here. this was a written story fabricated from the author's mind. Now we're on the very of though crime. > Amanda was 10 years old. she went into the bathroom and had sex with a 30 year old man. I think it would be ridiculous to say that the above sentence is on the same level as creating or distributing CSAM. Yet the predication of the argument is that the story conjured csam in the user's mind. Basically thought crime.

0x3f 8mo ago
I'm curious how you feel about images, because it seems we have the same problem: I draw a stick figure with genitals. All good. I put a little line and write '10 year old child', then... illegal? In some places, anyway. The difference with text I suppose is that text is _never_ real. The provenance of an image can be hard to determine.
- cultofmetatron 8mo ago
  I think the ethics here get complicated. for me the line would be if the AI itself was trained on actual CSAM. as long as no one was sexually violated in the course of creating the final image, I see no problem with it from an ethical perspective; all the better if it keeps potential predators from acting on real children. Wether it does or not is a complex topic that I won't claim to have any kind of qualifications to address.
  - croes 8mo ago
    > all the better if it keeps potential predators from acting on real children. The big question is if, those pictures could have the opposite effect.
    - pdpi 8mo ago
      And the followup big question is — how do you measure which effect, if any, occurs in practice?
    - chii 8mo ago
      So do you believe violent video games induce more violent crimes then?
      - pdpi 8mo ago
        The issue is a fair bit subtler than that. The analogous question here isn't "do violent video games induce violent behaviour in the general population?" but rather "do violent video games induce violent behaviour in people who already have a propensity for violence?" Or, even more specifically, "does incredibly realistic-looking violence in video games induce violent behaviour in people who already have a propensity for violence?". I'm not talking about the graphics being photorealistic enough or anything, I mean that, in games, the actual actions, the violence itself is extremely over the top. At least to me, it rarely registers as real violence at all, because it's so stylised. Real-world aggression looks nothing like that, it's much more contained.
        tosti 8mo ago
        Yep. It can definately go both ways. A game like Doom can be a nice way to put off some steam.
    - delecti 8mo ago
      That's a valid and interesting question to ask and study, but I don't think it's relevant to the decision of whether it should be illegal.
      - bmicraft 8mo ago
        I think that's the most, if not only relevant part to base your decision on
      - Insanity 8mo ago
        It is incredibly relevant. If murder is prevented by having people play violent games and live out their fantasy there, isn’t that a good thing? I’m not convinced that it would be, but it’s an interesting hypothesis.
        delecti 8mo ago
        The comment I replied to was proposing the opposite equivalent, that fake CSAM (written fiction, AI generated images not trained on real CSAM) could increase risk of action. I don't think violent video games should be banned, whether they increase or decrease IRL violence (I personally suspect they don't have a significant effect either way). And I don't think "simulated CSAM" (where no actual minors were involved in any part of the creation) should be banned on that basis either (though I don't know enough to guess whether it would tend to increase or decrease actual violations).
    - mrighele 8mo ago
      If there is no proof there should be no ban. What if parent is right (more widespread porn caused people to have less sex after all) ? This means that a ban caused more harm on real children.
  - hansvm 8mo ago
    IIRC, violent crime is increased in people pre-disposed to it when they use outlets and substitutes (consuming violent media, etc). That might not translate to pedophilia, but my prior would be that such content existing does cause more CSA to happen.
    - alexgieg 8mo ago
      That's incorrect. There have been studies on this. In a few cases seeing depictions of violence causes an urge to act violently, but in the majority of people predisposed to violence it causes a reduction in that impulse, so on average there's a reduction. The same has been shown to be the case with depictions of sexual abuse. For some it leads the person to go out and do it. For the majority of those predisposed to be sexual predators it "satisfies" them, and they end up causing less harm. Presumably the same applies to pedophiles. I remember reading a study on this that suggested this to be the case, but the sample size was small so the statistical significance was weak.
      - hansvm 8mo ago
        This review [0] is a bit reductionist and overconfident with some of its adjacent claims, but it includes a decent overview of the studies we've done on the topic and references those for further reading. The effect is weak enough at a societal level that it mostly doesn't make sense to consider (and those effect directions are not supportive of your claim of overall reduction if you want to interpret them as strong enough to matter), but when restricted to groups pre-disposed to violence you do see a meaningful increase in violent behaviors. [0] https://www.mdpi.com/2075-4698/3/4/491 https://www.mdpi.com/2075-4698/3/4/491
- amiga386 8mo ago
  Like this sketch where Chris Morris tries to get a (former) police officer to say what is and what isn't an indecent photograph? https://www.youtube.com/watch?v=eC7gH91Aaoo&t=1014s https://www.youtube.com/watch?v=eC7gH91Aaoo&t=1014s
glimshe 8mo ago
I gotta say that I'm leaning towards your argument but the quote you provided made me think... Would a prompt able to generate CSAM on an AI be considered itself CSAM?
- Tade0 8mo ago
  IANAL, but: If drawings overall are anything to go by it varies greatly by legal system, but most would lean on "yes". A generated image would most likely be not made locally, so there the added question of the image being understood as "distributed".
  - benchloftbrunch 8mo ago
    GP is asking about the text prompt itself, not the generated image. If pure text can qualify as CSAM in Australia then it's a logical question.
    - Tade0 8mo ago
      Really LLMed this one, thank you for pointing that out.
- 827a 8mo ago
  No, because AI makes the economy a lot of money, whereas authors do not.
alwayseasy 8mo ago
When I read your quote, I was agreeing with you. However, according to the article this very far from the very graphic content of the book in question! It feels like a strawman quote.
qntmfred 8mo ago
[flagged]
OskarS 8mo ago
> Basically thought crime I 100% agree with your central point, and I do think this is a very disturbing ruling. But it's not "thought crime", it's speech regulation. There's a very big difference between thought crime as in 1984 and speech regulation. There are many ways societies regulate speech, even liberal democratic ones: we don't allow defamation, and there are "time, place and manner" regulations (e.g. "yelling 'Fire!' in a crowded theater is not free speech"), and many countries have varieties of hate speech regulation. In Germany, speech denying the Holocaust is illegal. No society on earth has unlimited free speech. "Thought crime", as described in 1984, is something different: "thought crime" is when certain patterns of thought are illegal, even when unexpressed. This was, most certainly, expressed, which places it in a different category. Again, I totally agree with your central point that this is a censorious moral panic to a disturbing degree (are they banning "Lolita" next?), but it's not thought crime.
croes 8mo ago
They will argue that it could motivate perpetrators who read such stories to act when reading isn’t enough anymore. Some logic as for AI generated abuse material. You could also argue in the other way that it could prevent real abuse. Maybe a study would be useful if such a study doesn’t exist already
- KumaBear 8mo ago
  Slippery slope. What about a novel about the main character being a serial killer. Is that where we start saying that's illegal as well?
  - RajT88 8mo ago
    Jeff Lindsey's Dexter novels come to mind.
- RajT88 8mo ago
  From what I recall on the debates about manga ~20 years ago when people were getting in trouble for sexual mangas with young characters, consumers do not escalate their behavior to abuse. There may also be more recent studies. This is definitely a rehash of the same debate though - there should be lots of materials out there.
  - croes 8mo ago
    It’s not about consumers per se but abuser who consume. The Manga doesn’t turn people into abusers but what is the effect on already abusive personalities.
    - RajT88 8mo ago
      I can appreciate the argument, but it also lends itself to (as Jello Biafra famously said), "Ban Everything" thinking. I guess an example for your proposal is the gun laws for not allowing convicted domestic abusers firearms. But a comic is nowhere near equivalent to a firearm. I think the argument is fraught.
mothballed 8mo ago
[flagged]
- alexgieg 8mo ago
  That isn't in the Quran though.
  - stvltvs 8mo ago
    Banning some of the the Hadiths then?
- IAmBroom 8mo ago
  Will HN have the balls to ban you for misrepresenting the Quran? That isn't in the Quran.
  - mothballed 8mo ago
    You got me! The Hadiths that explain the Quran, then.
rented_mule 8mo ago
> Basically thought crime Let's go in the opposite direction... >> Amanda was 10 years old. she went into the bathroom and had sex with a 30 year old man. If the story was real, should Amanda be banned from publishing her own account of her experience later in life? Should she be able to write about the impact it had on her? I think she should have that freedom. What if she was 17 years 364 days old and the adult was 18 years 1 day old, assuming the age of consent is 18, and she writes about it being a good experience for her? 16 years old and 20? 4 and 40? Those are increasingly grotesque to me, but I don't know where to draw the line. Wait, have I crossed the line in what I've written in this reply? Have we all?
- mothballed 8mo ago
  I have no idea about Australia, but in USA it's pretty well established it is a crime to publish CSAM of yourself. Children are prosecuted for sending their own provocative images to others. I can only imagine the punishment would be worse if they distributed them after they were an adult. So I would think hypothetically if the words were CSAM, the fact they are the victim publishing their own account would be immaterial to their defense.
  - stvltvs 8mo ago
    IANAL, but written materials about sexual abuse don't seem to be illegal in the US. For recent-ish publications, see My Dark Vanessa by Russell and Tampa by Nutting. (I liked the former which took a thoughtful approach whereas I didn't finish the latter because it just felt like erotica for pedophiles which isn't what I was looking for.)
b2ccb2 8mo ago
Absolutely agree, wait until the judge finds out about Bukowski's "The Fiend".

0x3f 8mo ago

I'm curious how you feel about images, because it seems we have the same problem: I draw a stick figure with genitals. All good. I put a little line and write '10 year old child', then... illegal? In some places, anyway. The difference with text I suppose is that text is _never_ real. The provenance of an image can be hard to determine.

cultofmetatron 8mo ago
I think the ethics here get complicated. for me the line would be if the AI itself was trained on actual CSAM. as long as no one was sexually violated in the course of creating the final image, I see no problem with it from an ethical perspective; all the better if it keeps potential predators from acting on real children. Wether it does or not is a complex topic that I won't claim to have any kind of qualifications to address.
- croes 8mo ago
  > all the better if it keeps potential predators from acting on real children. The big question is if, those pictures could have the opposite effect.
  - pdpi 8mo ago
    And the followup big question is — how do you measure which effect, if any, occurs in practice?
  - chii 8mo ago
    So do you believe violent video games induce more violent crimes then?
    - pdpi 8mo ago
      The issue is a fair bit subtler than that. The analogous question here isn't "do violent video games induce violent behaviour in the general population?" but rather "do violent video games induce violent behaviour in people who already have a propensity for violence?" Or, even more specifically, "does incredibly realistic-looking violence in video games induce violent behaviour in people who already have a propensity for violence?". I'm not talking about the graphics being photorealistic enough or anything, I mean that, in games, the actual actions, the violence itself is extremely over the top. At least to me, it rarely registers as real violence at all, because it's so stylised. Real-world aggression looks nothing like that, it's much more contained.
      - tosti 8mo ago
        Yep. It can definately go both ways. A game like Doom can be a nice way to put off some steam.
  - delecti 8mo ago
    That's a valid and interesting question to ask and study, but I don't think it's relevant to the decision of whether it should be illegal.
    - bmicraft 8mo ago
      I think that's the most, if not only relevant part to base your decision on
    - Insanity 8mo ago
      It is incredibly relevant. If murder is prevented by having people play violent games and live out their fantasy there, isn’t that a good thing? I’m not convinced that it would be, but it’s an interesting hypothesis.
      - delecti 8mo ago
        The comment I replied to was proposing the opposite equivalent, that fake CSAM (written fiction, AI generated images not trained on real CSAM) could increase risk of action. I don't think violent video games should be banned, whether they increase or decrease IRL violence (I personally suspect they don't have a significant effect either way). And I don't think "simulated CSAM" (where no actual minors were involved in any part of the creation) should be banned on that basis either (though I don't know enough to guess whether it would tend to increase or decrease actual violations).
  - mrighele 8mo ago
    If there is no proof there should be no ban. What if parent is right (more widespread porn caused people to have less sex after all) ? This means that a ban caused more harm on real children.
- hansvm 8mo ago
  IIRC, violent crime is increased in people pre-disposed to it when they use outlets and substitutes (consuming violent media, etc). That might not translate to pedophilia, but my prior would be that such content existing does cause more CSA to happen.
  - alexgieg 8mo ago
    That's incorrect. There have been studies on this. In a few cases seeing depictions of violence causes an urge to act violently, but in the majority of people predisposed to violence it causes a reduction in that impulse, so on average there's a reduction. The same has been shown to be the case with depictions of sexual abuse. For some it leads the person to go out and do it. For the majority of those predisposed to be sexual predators it "satisfies" them, and they end up causing less harm. Presumably the same applies to pedophiles. I remember reading a study on this that suggested this to be the case, but the sample size was small so the statistical significance was weak.
    - hansvm 8mo ago
      This review [0] is a bit reductionist and overconfident with some of its adjacent claims, but it includes a decent overview of the studies we've done on the topic and references those for further reading. The effect is weak enough at a societal level that it mostly doesn't make sense to consider (and those effect directions are not supportive of your claim of overall reduction if you want to interpret them as strong enough to matter), but when restricted to groups pre-disposed to violence you do see a meaningful increase in violent behaviors. [0] https://www.mdpi.com/2075-4698/3/4/491 https://www.mdpi.com/2075-4698/3/4/491
amiga386 8mo ago
Like this sketch where Chris Morris tries to get a (former) police officer to say what is and what isn't an indecent photograph? https://www.youtube.com/watch?v=eC7gH91Aaoo&t=1014s https://www.youtube.com/watch?v=eC7gH91Aaoo&t=1014s

glimshe 8mo ago

I gotta say that I'm leaning towards your argument but the quote you provided made me think... Would a prompt able to generate CSAM on an AI be considered itself CSAM?

Tade0 8mo ago
IANAL, but: If drawings overall are anything to go by it varies greatly by legal system, but most would lean on "yes". A generated image would most likely be not made locally, so there the added question of the image being understood as "distributed".
- benchloftbrunch 8mo ago
  GP is asking about the text prompt itself, not the generated image. If pure text can qualify as CSAM in Australia then it's a logical question.
  - Tade0 8mo ago
    Really LLMed this one, thank you for pointing that out.
827a 8mo ago
No, because AI makes the economy a lot of money, whereas authors do not.

alwayseasy 8mo ago

When I read your quote, I was agreeing with you. However, according to the article this very far from the very graphic content of the book in question! It feels like a strawman quote.

qntmfred 8mo ago

[flagged]

OskarS 8mo ago

> Basically thought crime I 100% agree with your central point, and I do think this is a very disturbing ruling. But it's not "thought crime", it's speech regulation. There's a very big difference between thought crime as in 1984 and speech regulation. There are many ways societies regulate speech, even liberal democratic ones: we don't allow defamation, and there are "time, place and manner" regulations (e.g. "yelling 'Fire!' in a crowded theater is not free speech"), and many countries have varieties of hate speech regulation. In Germany, speech denying the Holocaust is illegal. No society on earth has unlimited free speech. "Thought crime", as described in 1984, is something different: "thought crime" is when certain patterns of thought are illegal, even when unexpressed. This was, most certainly, expressed, which places it in a different category. Again, I totally agree with your central point that this is a censorious moral panic to a disturbing degree (are they banning "Lolita" next?), but it's not thought crime.

croes 8mo ago

They will argue that it could motivate perpetrators who read such stories to act when reading isn’t enough anymore. Some logic as for AI generated abuse material. You could also argue in the other way that it could prevent real abuse. Maybe a study would be useful if such a study doesn’t exist already

KumaBear 8mo ago
Slippery slope. What about a novel about the main character being a serial killer. Is that where we start saying that's illegal as well?
- RajT88 8mo ago
  Jeff Lindsey's Dexter novels come to mind.
RajT88 8mo ago
From what I recall on the debates about manga ~20 years ago when people were getting in trouble for sexual mangas with young characters, consumers do not escalate their behavior to abuse. There may also be more recent studies. This is definitely a rehash of the same debate though - there should be lots of materials out there.
- croes 8mo ago
  It’s not about consumers per se but abuser who consume. The Manga doesn’t turn people into abusers but what is the effect on already abusive personalities.
  - RajT88 8mo ago
    I can appreciate the argument, but it also lends itself to (as Jello Biafra famously said), "Ban Everything" thinking. I guess an example for your proposal is the gun laws for not allowing convicted domestic abusers firearms. But a comic is nowhere near equivalent to a firearm. I think the argument is fraught.

mothballed 8mo ago

[flagged]

alexgieg 8mo ago
That isn't in the Quran though.
- stvltvs 8mo ago
  Banning some of the the Hadiths then?
IAmBroom 8mo ago
Will HN have the balls to ban you for misrepresenting the Quran? That isn't in the Quran.
- mothballed 8mo ago
  You got me! The Hadiths that explain the Quran, then.

rented_mule 8mo ago

> Basically thought crime Let's go in the opposite direction... >> Amanda was 10 years old. she went into the bathroom and had sex with a 30 year old man. If the story was real, should Amanda be banned from publishing her own account of her experience later in life? Should she be able to write about the impact it had on her? I think she should have that freedom. What if she was 17 years 364 days old and the adult was 18 years 1 day old, assuming the age of consent is 18, and she writes about it being a good experience for her? 16 years old and 20? 4 and 40? Those are increasingly grotesque to me, but I don't know where to draw the line. Wait, have I crossed the line in what I've written in this reply? Have we all?

mothballed 8mo ago
I have no idea about Australia, but in USA it's pretty well established it is a crime to publish CSAM of yourself. Children are prosecuted for sending their own provocative images to others. I can only imagine the punishment would be worse if they distributed them after they were an adult. So I would think hypothetically if the words were CSAM, the fact they are the victim publishing their own account would be immaterial to their defense.
- stvltvs 8mo ago
  IANAL, but written materials about sexual abuse don't seem to be illegal in the US. For recent-ish publications, see My Dark Vanessa by Russell and Tampa by Nutting. (I liked the former which took a thoughtful approach whereas I didn't finish the latter because it just felt like erotica for pedophiles which isn't what I was looking for.)

b2ccb2 8mo ago

Absolutely agree, wait until the judge finds out about Bukowski's "The Fiend".