5 ms·

What’s the right number of lives to destroy from CSAM? There’s a middle ground between doing nothing and totalitarianism. “Destroyed lives” from false positive

by cmckn 5y ago

What’s the right number of lives to destroy from CSAM? There’s a middle ground between doing nothing and totalitarianism. “Destroyed lives” from false positives are at this point hypothetical. Child abuse is not. It’s fair to be concerned about false positives and ensure the system handles such failures appropriately. It’s also fair to directly intervene in the widespread circulation of CSAM.

logicchains 5y ago
>What’s the right number of lives to destroy from CSAM? Is there any evidence such things actually reduce the production of CSAM? Or is it like the war on drugs where drug production is as high as it's ever been.
vetinari 5y ago
There's no really a middle ground; the right number is 0. It is better that ten guilty persons escape than that one innocent suffer. -- Blackstone's ratio
- account-5 5y ago
  Or in this situation more like: It is better that ten children get sexually abused than one innocent person comes under suspicion /s
  - omginternets 5y ago
    This isn't combatting abuse because producing CSAM != consuming CSAM. It's far better that 10 people beat off to children than have one innocent person come under suspicion.
    - account-5 5y ago
      I disagree. Suspicion is not the same as prosecuted, and every time someone "beats off" to an image of child sexual abuse, that child is re-victimised; every time. You'd rather 10 children be victimised than 1 person falls under suspicion? Ok...
      - omginternets 5y ago
        >you’d rather 10 children be victimized Now you’re just arguing in bad faith.
        account-5 5y ago
        > Now you’re just arguing in bad faith. In what way? I'm just characterising you argument from the point of view of the victims of the crime, who you appear quite happy to throw under the bus.
      - creata 5y ago
        > You'd rather 10 children be [re-]victimised than 1 person falls under suspicion? Ok... Well yeah, mostly because I don't understand what you mean be "revictimization", but I can easily imagine what it's like to have such suspicions leaked to the angry mob.
        account-5 5y ago
        I'm not sure what's difficult to understand about it. How about we pivot to intimate image abuse (revenge porn). The argument is the same. The victims are re-victimised every time their intimate images are shared, for the victims it's always the same stress, shame, embarrassment, etc. However apparently the damage is already done so there's actually no issues according to the other user so what's all the fuss about...
        creata 5y ago
        > I'm not sure what's difficult to understand about it. I guess the difficult bit for me is that you talk about throwing victims under the bus, and letting them be "victimized" all over again, but I just don't understand how this solution helps victims at all. Here's the example in my head: imagine you're a victim of revenge porn. The police, using technology like the stuff Apple has developed, catch one random guy (out of hundreds or thousands more) in possession of this porn. The police probably won't tell you that they caught this guy. How does this event help you? > there's actually no issues It's not that there are no issues: the distribution of CSAM (and of revenge porn) are terrible crimes, and iirc they're increasing in prevalence. I just think that Apple's solution doesn't help victims, and seriously discomforts millions of people. As you've seen, many people do see this as privacy invasion. The feeling that folks somewhere far away might be able to see your photos (even low resolution thumbnails) without your knowledge isn't a good feeling to inflict on the world. And the feeling that your phone could report you to the police at any time isn't a good feeling either, even if you know you've done nothing wrong.
- School-Cotton 5y ago
  There’s a reason he said ten and not a million. This argument is absurd when taken to its maximalist conclusion. Whether a rational person would accept chance X of wrongfully being convicted of a crime to decrease the chance of being a victim of crime by Y obviously depends on the values of X and Y.
kelnos 5y ago
This is certainly a matter of opinion, but mine is that I would rather let an arbitrary number of criminals go free than jail (or ruin the life of) even one innocent person.
devwastaken 5y ago
The middle ground is 0. The idea of equal jailed innocents to jailed criminals is mind numbing leaps of logic that is against the very ethos the united states was founded on.
- omginternets 5y ago
  I understand the sentiment and even share it to some extent, but we should really be arguing against the strongest possible version of claim, which is as follows: signal-detection theory tells us that there is a direct relationship between the number of correct-detections and false-alarms, and the only way to achieve 0 false-alarms is not to label anything a hit. In the present case, this means not prosecuting anyone, ever. That's probably not a solution you're happy with. Therefore, the argument is one of degree. I agree with you that Apple's CSAM-detection is going too far, and this is what we should be articulating. Chanting "not even one" is not particularly convincing, nor sensible.
jchw 5y ago
From this mechanism? Zero. And if even if it was zero anyways, these measures are not justified. There are plenty of other ways to catch out criminals that doesn’t involve dubious phoning home. Devices that treat their owners like potential criminals are nothing more than government informants. Also… The goal of law enforcement is not to vengefully destroy the lives of people who commit crime, but that’s a whole different can of tuna. Still worth noting, because it hints at a larger problem about how we approach heinous crimes.
avazhi 5y ago
CSAM is, by definition, photography or videography of something that has already happened. Therefore, quite literally, doing absolutely nothing about CSAM itself would result in no harm to any child, as the harm has already occurred. Now you're probably going to then cry about incentivising or normalising CSAM - but that's a different argument. And if you then try to argue that the normalisation of CSAM would somehow encourage people to abuse children, well then you're really off into the zero-evidence weeds. Go look at porn research (the actual Google Scholar/JSTOR/Elsevier kind), and you'll see that almost everybody who looks at porn neither wants to nor would actually do what they see in porn, if they were given the opportunity. Surprise, surprise, most people wouldn't actually get gangbanged/bukkake'd/fucked by their sibling, mom, dad, grandpa/pooped on by their next door bespectacled red-headed neighbour, etc. Nor is there any evidence that inadvertently coming across CSAM turns people into pedophiles (news flash: pedos were turned on by kids long before they were ever exposed to CSAM on the internet), and porn itself is almost invariably used for fantasy or as something wholly unrealistic that people get off to precisely because it's unrealistic. Even though it might be unsavoury to do so, we could follow this reasoning to its extreme but undoubtedly true conclusion and state that there are individuals who get off to CSAM notwithstanding that they would never themselves abuse children. So to recap, 0 children would be saved from harm because the CSAM itself is ex post facto; it wouldn't de-incentivise CSAM because the demand and markets for CSAM and pedophilia existed long before the internet was a thing, and pedophiles will find avenues around dumbass implementations like Apple's scanning (TOR, anyone? not using an iPhone/Mac?); and, finally, just because somebody looks at CSAM doesn't mean they're an actual pedo or would ever harm children themselves. The fact that possession of such material is illegal is not to the point - Apple is not the police, and the police and other executive agencies need warrants for this kind of thing (in common law countries this notion is more than 400 years old). Meanwhile, we know the false positive risks are not insignificant - look at the white papers yourself, or just look at the numbers that smart people have crunched. The best part is that even though Apple says the false positive rate is 1 in a trillion accounts, people's photo libraries are exactly the sort of thing you can't extrapolate statistically. Maybe your Aunt June really likes photos of her nephews in the pool, and she's got a library with 300 photos of her nephews half naked and swimming when they visited her last year. Apple has no fucking clue whether they would or would not trigger its scanner, because it currently does not have access to Aunt June's unique photos to test vs its database. Apple quite literally doesn't know what the fuck will happen. I and many others find that abhorrent when you consider the effect that even the mere accusation of pedophilia has on a person's life. And that isn't even to start the discussion of what kind of precedent on-device scanning would set for other subjects (political and religious dissent, for example - if not in America then in places like China). Apple and everybody else can fuck right off with this Orwellian shit.
deleted 5y ago
[deleted]
creata 5y ago
> “Destroyed lives” from false positives are at this point hypothetical. Child abuse is not. The idea that all of this effort (and all of the direct discomfort inflicted on Apple users) will do anything to stop child abuse is just as hypothetical.
gray_charger 5y ago
You're right, there is a middle ground. Law enforcement can request a warrant from a judge and demonstrate probable cause to search the machine they believe is harboring CSAM. Any attempt to bypass this process is totalitarianism.

by cmckn 5y ago

logicchains 5y ago

>What’s the right number of lives to destroy from CSAM? Is there any evidence such things actually reduce the production of CSAM? Or is it like the war on drugs where drug production is as high as it's ever been.

vetinari 5y ago

There's no really a middle ground; the right number is 0. It is better that ten guilty persons escape than that one innocent suffer. -- Blackstone's ratio

account-5 5y ago
Or in this situation more like: It is better that ten children get sexually abused than one innocent person comes under suspicion /s
- omginternets 5y ago
  This isn't combatting abuse because producing CSAM != consuming CSAM. It's far better that 10 people beat off to children than have one innocent person come under suspicion.
  - account-5 5y ago
    I disagree. Suspicion is not the same as prosecuted, and every time someone "beats off" to an image of child sexual abuse, that child is re-victimised; every time. You'd rather 10 children be victimised than 1 person falls under suspicion? Ok...
    - omginternets 5y ago
      >you’d rather 10 children be victimized Now you’re just arguing in bad faith.
      - account-5 5y ago
        > Now you’re just arguing in bad faith. In what way? I'm just characterising you argument from the point of view of the victims of the crime, who you appear quite happy to throw under the bus.
    - creata 5y ago
      > You'd rather 10 children be [re-]victimised than 1 person falls under suspicion? Ok... Well yeah, mostly because I don't understand what you mean be "revictimization", but I can easily imagine what it's like to have such suspicions leaked to the angry mob.
      - account-5 5y ago
        I'm not sure what's difficult to understand about it. How about we pivot to intimate image abuse (revenge porn). The argument is the same. The victims are re-victimised every time their intimate images are shared, for the victims it's always the same stress, shame, embarrassment, etc. However apparently the damage is already done so there's actually no issues according to the other user so what's all the fuss about...
        creata 5y ago
        > I'm not sure what's difficult to understand about it. I guess the difficult bit for me is that you talk about throwing victims under the bus, and letting them be "victimized" all over again, but I just don't understand how this solution helps victims at all. Here's the example in my head: imagine you're a victim of revenge porn. The police, using technology like the stuff Apple has developed, catch one random guy (out of hundreds or thousands more) in possession of this porn. The police probably won't tell you that they caught this guy. How does this event help you? > there's actually no issues It's not that there are no issues: the distribution of CSAM (and of revenge porn) are terrible crimes, and iirc they're increasing in prevalence. I just think that Apple's solution doesn't help victims, and seriously discomforts millions of people. As you've seen, many people do see this as privacy invasion. The feeling that folks somewhere far away might be able to see your photos (even low resolution thumbnails) without your knowledge isn't a good feeling to inflict on the world. And the feeling that your phone could report you to the police at any time isn't a good feeling either, even if you know you've done nothing wrong.
School-Cotton 5y ago
There’s a reason he said ten and not a million. This argument is absurd when taken to its maximalist conclusion. Whether a rational person would accept chance X of wrongfully being convicted of a crime to decrease the chance of being a victim of crime by Y obviously depends on the values of X and Y.

kelnos 5y ago

This is certainly a matter of opinion, but mine is that I would rather let an arbitrary number of criminals go free than jail (or ruin the life of) even one innocent person.

devwastaken 5y ago

The middle ground is 0. The idea of equal jailed innocents to jailed criminals is mind numbing leaps of logic that is against the very ethos the united states was founded on.

omginternets 5y ago
I understand the sentiment and even share it to some extent, but we should really be arguing against the strongest possible version of claim, which is as follows: signal-detection theory tells us that there is a direct relationship between the number of correct-detections and false-alarms, and the only way to achieve 0 false-alarms is not to label anything a hit. In the present case, this means not prosecuting anyone, ever. That's probably not a solution you're happy with. Therefore, the argument is one of degree. I agree with you that Apple's CSAM-detection is going too far, and this is what we should be articulating. Chanting "not even one" is not particularly convincing, nor sensible.

jchw 5y ago

From this mechanism? Zero. And if even if it was zero anyways, these measures are not justified. There are plenty of other ways to catch out criminals that doesn’t involve dubious phoning home. Devices that treat their owners like potential criminals are nothing more than government informants. Also… The goal of law enforcement is not to vengefully destroy the lives of people who commit crime, but that’s a whole different can of tuna. Still worth noting, because it hints at a larger problem about how we approach heinous crimes.

avazhi 5y ago

CSAM is, by definition, photography or videography of something that has already happened. Therefore, quite literally, doing absolutely nothing about CSAM itself would result in no harm to any child, as the harm has already occurred. Now you're probably going to then cry about incentivising or normalising CSAM - but that's a different argument. And if you then try to argue that the normalisation of CSAM would somehow encourage people to abuse children, well then you're really off into the zero-evidence weeds. Go look at porn research (the actual Google Scholar/JSTOR/Elsevier kind), and you'll see that almost everybody who looks at porn neither wants to nor would actually do what they see in porn, if they were given the opportunity. Surprise, surprise, most people wouldn't actually get gangbanged/bukkake'd/fucked by their sibling, mom, dad, grandpa/pooped on by their next door bespectacled red-headed neighbour, etc. Nor is there any evidence that inadvertently coming across CSAM turns people into pedophiles (news flash: pedos were turned on by kids long before they were ever exposed to CSAM on the internet), and porn itself is almost invariably used for fantasy or as something wholly unrealistic that people get off to precisely because it's unrealistic. Even though it might be unsavoury to do so, we could follow this reasoning to its extreme but undoubtedly true conclusion and state that there are individuals who get off to CSAM notwithstanding that they would never themselves abuse children. So to recap, 0 children would be saved from harm because the CSAM itself is ex post facto; it wouldn't de-incentivise CSAM because the demand and markets for CSAM and pedophilia existed long before the internet was a thing, and pedophiles will find avenues around dumbass implementations like Apple's scanning (TOR, anyone? not using an iPhone/Mac?); and, finally, just because somebody looks at CSAM doesn't mean they're an actual pedo or would ever harm children themselves. The fact that possession of such material is illegal is not to the point - Apple is not the police, and the police and other executive agencies need warrants for this kind of thing (in common law countries this notion is more than 400 years old). Meanwhile, we know the false positive risks are not insignificant - look at the white papers yourself, or just look at the numbers that smart people have crunched. The best part is that even though Apple says the false positive rate is 1 in a trillion accounts, people's photo libraries are exactly the sort of thing you can't extrapolate statistically. Maybe your Aunt June really likes photos of her nephews in the pool, and she's got a library with 300 photos of her nephews half naked and swimming when they visited her last year. Apple has no fucking clue whether they would or would not trigger its scanner, because it currently does not have access to Aunt June's unique photos to test vs its database. Apple quite literally doesn't know what the fuck will happen. I and many others find that abhorrent when you consider the effect that even the mere accusation of pedophilia has on a person's life. And that isn't even to start the discussion of what kind of precedent on-device scanning would set for other subjects (political and religious dissent, for example - if not in America then in places like China). Apple and everybody else can fuck right off with this Orwellian shit.

deleted 5y ago

[deleted]

creata 5y ago

> “Destroyed lives” from false positives are at this point hypothetical. Child abuse is not. The idea that all of this effort (and all of the direct discomfort inflicted on Apple users) will do anything to stop child abuse is just as hypothetical.

gray_charger 5y ago

You're right, there is a middle ground. Law enforcement can request a warrant from a judge and demonstrate probable cause to search the machine they believe is harboring CSAM. Any attempt to bypass this process is totalitarianism.