8 ms·

Most everyone would love to see more work on stopping child sexual abuse. But this is the ultimate "grant me dictatorial powers so I can do good" play. Rather

Most everyone would love to see more work on stopping child sexual abuse. But this is the ultimate "grant me dictatorial powers so I can do good" play. Rather than narrow and specific - it's a broad based law that suddenly touches everyone even though offenders are a small percentage and should be able to be targeted more efficiently.

cortesoft 3mo ago
Yep, and this is a perfect example of a base rate fallacy situation... even if the scanner is 99.99% accurate, because an even higher percentage of photos are innocent, most matches the scanner will find will be false positives.
- dtj1123 3mo ago
  I thought this was known as Bonferonni'a principle? Or am I getting mixed up?
  - vaylian 3mo ago
    Bonferonni correction is relevant when you calculate multiple p-values. Most statistical tests are used with a p-value threshold of 5% to reject the null-hypothesis. But because you are repeatedly testing, the probability for false positives increases and that is why you need to decrease the threshold and make it harder, to obtain a p-value below that threshold to declare a significant result. You typically use the Bonferroni correction when making general statements about a statistical relationship. You wouldn't use it for checking if a particular image shows illegal content. If you kept testing with your image classifier, your significance threshold would need to be continuously lowered and you would asymptotically reach zero. Relevant XKCD: 882
    - dtj1123 3mo ago
      No, I'm not referring to Bonferroni's correction, which obviously has nothing to do with what we're discussing. I'm sure I've heard this called Bonferroni's principle. Edit: Yes, in the section 'Statistical Limits of Data Mining' in the book Mining of mMssive Data Sets it's called Bonferroni's principle.
- wesammikhail 3mo ago
  Funny you bring this up. Back in the day when I was like 15 and DC++ was still a thing, I used to browse people's shared folders. One day I came across a file called "the paradox of false positive". It was a 1 pager that described how a machine which is 99.9% accurate at identifying terrorists would be completely useless due to this false positive base rate fallacy you're describing. It really stuck with me throughout the years. It's kind o remarkable how even a 99.9% accurate heuristic is insufficient at scale. Which begs the question: lets assume the intentions are pure (which we know they're not but lets be generous), what other options are there when 99.9% heuristic is not good enough? how do you design systems when they're guaranteed to fail as they scale up? edit: and what do you know, I just saw this as I scrolled down on HN https://news.ycombinator.com/item?id=48816959 https://news.ycombinator.com/item?id=48816959
  - m12k 3mo ago
    The intuition I've built is that you can't talk about a false positive rate being high or low on its own - it's always relative to the actual occurrence rate of positives in the tested population. E.g. if there's a 1 in 10000 risk of a false positive, but real positives also are only 1 out of 10000 tested cases, then a positive case will have a 50/50 chance of being a false positive (because for every 10000 tests, you'll have on average one false positive and one real positive). So a false positive rate can only be said to be low if it's significantly lower than the real occurrence rate of positives.
    - ablob 3mo ago
      The mentioned accuracy in the comment you are replying to already encapsulates the relation of true positives to false positives.
      - fc417fc802 3mo ago
        No I don't believe it does. I interpret 99.9% accurate to mean 1 in 1000 false positives. If 0.1% of your population are terrorists that means each alert has a 50% chance of being correct. That's nowhere near good enough to fully automate things but it is quite reasonable assuming this is merely information provided to a human agent. Whereas if only 0.001% of your population are terrorists then 99 out of 100 alerts are false positives at which point the system is well on its way to being useless. There is an important difference between scenarios where we care about the relative versus absolute frequency of errors.
- kleiba2 3mo ago
  > even if the scanner is 99.99% accurate, because an even higher percentage of photos are innocent, most matches the scanner will find will be false positives. If the scanner is 99.99% accurate, then most classifications will be correct.
  - cenamus 3mo ago
    If you scan 1,000,000 pictures (with let's say 10 CSAM), you'll have 100 false positives and 10 true positives, giving you like only 10% correct results
    - kleiba2 3mo ago
      Ah, sorry, I misread what the OP meant by "matches" - thought they were referring to all classifier outputs, while they specifically meant the positives. I changed my original comment to better reflect what I meant, even though that makes it a bit of a non-sequitur now.
  - usrnm 3mo ago
    How many child abusers do you think there are out there?
    - myrmidon 3mo ago
      Even if 10% of population were actively criminal pedos (which is waaaay too high), its pretty safe to assume that the majority of even their online footprint would be ordinary images/messages. So a quota of 0.1% or even less material being detectably criminal sounds realistic (probably not much less, though).
  - acksmack 3mo ago
    https://en.wikipedia.org/wiki/Base_rate_fallacy https://en.wikipedia.org/wiki/Base_rate_fallacy
- tjpnz 3mo ago
  Google have already caused significant hardship to a father for such kinds of photos. What's particularly galling is how they've continued to maintain they were in the right, despite the police saying no crime had been committed. https://www.koffellaw.com/blog/google-ai-technology-flags-dad-who-took-photos-o/ https://www.koffellaw.com/blog/google-ai-technology-flags-da...
  - joe_mamba 3mo ago
    Of course google and every other big-tech platform is gonna insta-wipe every account containing detected nudes of children, regardless if you're the parent. The corporate liability of such content being found on their cloud is so insanely nuclear, that they're not gonna wait and ask you "hey are those nudes your own kids or are you a pedo?" before they wipe the account with all pics off their servers.
    - zmmmmm 3mo ago
      And yet the will badger you endlessly to the point their photos app is near unusable to turn on auto sync which slurps up every photo and makes it very awkward to then delete them after. To me, this makes Google a liable party even if real CSAM is stored.
ggthrowaway 3mo ago
CSA makes ppl lose all logic, so is used to justify illogical things. Reminder that none of this has any evidence that it helps CSA, but nobody cares about the actual children.
- teaearlgraycold 3mo ago
  I feel like the world cares more about stopping the spread of CSAM than it does the actual abusive actions against children.
  - mrtesthah 3mo ago
    We can look to certain world "leaders" for confirmation of that.
  - pydry 3mo ago
    It's not the world that's the problem it's the small group of individuals trying to create stasi 2.0, hiding behind the children.
englishspot 3mo ago
so much for the principle of least privilege..
bonoboTP 3mo ago
The bad consequences are diffuse, abstract and distant (conspiracy-looking, tinfoil-like), while it's very easy to viscerally understand that "even if they just save one child, it's already worth it". They should give precise numbers of how many such crimes are detected via such means or are expected to be detected per year, and how many of those are not possible to catch through regular investigative work. It just seems ridiculously out of proportion especially that with all this flurry around the topic, the criminals surely aren't using WhatsApp for this any more, but especially won't be once the law is adopted. Sure, many are likely stupid but if they are so stupid, won't they fall into other honeypots? Why are chat apps the best leverage for uncovering this? They'd have to justify this with some sort of data and numbers. Because later they can just come back and say, well unfortunately they are now all using other means, so now we need to break https,we need to ban e2e, we need to ban vpns, tor and foss operating systems etc etc.
- iamnothere 3mo ago
  They should add to those metrics: hours and funds wasted investigating false positives, reputations ruined from false accusations and investigations, decline in public trust, etc.
- u8080 3mo ago
  Yeah, and also how many such crimes are actually prosecuted because you know, there is certain island with certain high-ranked people. Anyways, once that implemented noone will report to you and there will be no means of pushing against it because all your online efforts to coordinate will be compromised.
attila-lendvai 3mo ago
especially that the guard applying to protect the henhouse seems to have a suspiciously furry tail...
brikym 3mo ago
[flagged]
- wredcoll 3mo ago
  That's not a hot take, it's just boring old racism.
- bwhitty 3mo ago
  Hot take !== unsubstantiated, xenophobic take
greenleafone7 3mo ago
In the list of people that are worried about children.... the government is at the very end.
EarlKing 3mo ago
> stopping child sexual abuse > suddenly touches everyone ..............I see what you did there.
baxtr 3mo ago
I’ve shared this before, I really like this quote: "The urge to save humanity is almost always a false front for the urge to rule." H.L. Mencken
- myrmidon 3mo ago
  Mencken just has the best quotes. Here's a few of my favorites: > The trouble with fighting for human freedom is that one spends most of one's time defending scoundrels. For it is against scoundrels that oppressive laws are first aimed, and oppression must be stopped at the beginning if it is to be stopped at all. > For every problem, there is a solution that is simple, neat, and wrong. > Freedom of press is limited to those who own one.
latentsea 3mo ago
At some point we just have to accept the kids as collateral.
- taneq 3mo ago
  What, like with guns? Never!
  - latentsea 3mo ago
    If you want the guns, then yes. I come from somewhere without guns, and as a result without school shootings. Nobody here wants guns as a part of everyday life. America seems to want them.
sneak 3mo ago
Technology is, furthermore, the wrong place to address child abuse of any kind, sexual or otherwise. This is like trying to prevent burglary by working with the factory that manufactures pry bars.
f6v 3mo ago
> Most everyone would love to see more work on stopping child sexual abuse. By the parents. Install parental controls that only allow to message you and closest relatives. Problem solved.
eunos 3mo ago
> Rather than narrow and specific - it's a broad based law Because narrow law is easier to avoid or find the loophole and a single case is enough to induce panic and anger.
AlexanderHanff 3mo ago
I have put up a list of all the MEPs who voted for the urgency procedure yesterday (in breach of EU rules) as well as their voting history on fundamental rights issues and who has been lobbying them: https://www.thatprivacyguy.com/blog/chat-control-the-415-who-failed-you/ https://www.thatprivacyguy.com/blog/chat-control-the-415-who...
- vaylian 3mo ago
  Thanks. Please note that your link doesn't work with the tor browser.
  - AlexanderHanff 3mo ago
    Not sure why, it is working fine everywhere else - I see in Tor it gives an invalid certificate error, but the Lets Encrypt certificate is working fine in other browsers, so seems to be a Tor thing specifically. I will investigate.
  - AlexanderHanff 3mo ago
    I cannot see what is causing the issue, the certificate's full chain is sent, the clock is synced, the cert is showing zero errors in OpenSSL - so this is very confusing. The irony is, you don't actually need Tor on my site because there is no logging, no third parties, no adtech etc. it is just static HTML files - so whereas I would normally recommend Tor I designed the site specifically to be privacy first. I will try to figure out what is going on though because obviously I am fully supportive of people protecting their privacy with Tor.
  - AlexanderHanff 3mo ago
    OK it should be fixed now - there was a rate limit which doesn't normally land but due to multiple Tor users coming through the same exit node, it was triggering the limit. Nothing to do with the certs just told me a cert error because it never finished the handshake. I tested through Tor on multiple machines now and multiple circuits and is working clean - thanks for the heads up.
- almaya 3mo ago
order-matters 3mo ago
its also just disastrous for signal to noise ratios. scanning everything means any sort of error rate is going to cause massive amounts of incorrect labelling. this means innocent things getting flagged and put into a system where people are treated like offenders when they arent until they can get an actual human with authority to review their circumstances (not guaranteed to happen at all btw), or some actual offenders get away with more bc they passed a scan outlier cases aside, there is also just a large amount of processing power that will go into this, the service can only be worse off for it. Privacy is not just about being able to hide things, it is also about being in control of how you present to the world. not because that control is maniupulative but because we all exist within our own microcosms of uniqueness, using words slightly differently than each other, and having certain balances of intention and meaning with those we send messages to that cannot be fully presumed from a 3rd party. even in images. Are they really saying "if you want to send private messages then go make your own network" ?
- dpoloncsak 3mo ago
  >"if you want to send private messages then go make your own network" Unironically, we should all move to using TOR. Anyone setup a .onion mirror for HN yet? I'd assume usual HN-mirror-rules, no login or posting but free to view...
  - order-matters 3mo ago
    Tor is full of honeypots, its good for getting around 3rd party snooping but useless for government level privacy
    - deleted 3mo ago
      [deleted]
- pixl97 3mo ago
  >Are they really saying "if you want to send private messages then go make your own network No, because with the way things work, they'll make that illegal next.

cortesoft 3mo ago

Yep, and this is a perfect example of a base rate fallacy situation... even if the scanner is 99.99% accurate, because an even higher percentage of photos are innocent, most matches the scanner will find will be false positives.

dtj1123 3mo ago
I thought this was known as Bonferonni'a principle? Or am I getting mixed up?
- vaylian 3mo ago
  Bonferonni correction is relevant when you calculate multiple p-values. Most statistical tests are used with a p-value threshold of 5% to reject the null-hypothesis. But because you are repeatedly testing, the probability for false positives increases and that is why you need to decrease the threshold and make it harder, to obtain a p-value below that threshold to declare a significant result. You typically use the Bonferroni correction when making general statements about a statistical relationship. You wouldn't use it for checking if a particular image shows illegal content. If you kept testing with your image classifier, your significance threshold would need to be continuously lowered and you would asymptotically reach zero. Relevant XKCD: 882
  - dtj1123 3mo ago
    No, I'm not referring to Bonferroni's correction, which obviously has nothing to do with what we're discussing. I'm sure I've heard this called Bonferroni's principle. Edit: Yes, in the section 'Statistical Limits of Data Mining' in the book Mining of mMssive Data Sets it's called Bonferroni's principle.
wesammikhail 3mo ago
Funny you bring this up. Back in the day when I was like 15 and DC++ was still a thing, I used to browse people's shared folders. One day I came across a file called "the paradox of false positive". It was a 1 pager that described how a machine which is 99.9% accurate at identifying terrorists would be completely useless due to this false positive base rate fallacy you're describing. It really stuck with me throughout the years. It's kind o remarkable how even a 99.9% accurate heuristic is insufficient at scale. Which begs the question: lets assume the intentions are pure (which we know they're not but lets be generous), what other options are there when 99.9% heuristic is not good enough? how do you design systems when they're guaranteed to fail as they scale up? edit: and what do you know, I just saw this as I scrolled down on HN https://news.ycombinator.com/item?id=48816959 https://news.ycombinator.com/item?id=48816959
- m12k 3mo ago
  The intuition I've built is that you can't talk about a false positive rate being high or low on its own - it's always relative to the actual occurrence rate of positives in the tested population. E.g. if there's a 1 in 10000 risk of a false positive, but real positives also are only 1 out of 10000 tested cases, then a positive case will have a 50/50 chance of being a false positive (because for every 10000 tests, you'll have on average one false positive and one real positive). So a false positive rate can only be said to be low if it's significantly lower than the real occurrence rate of positives.
  - ablob 3mo ago
    The mentioned accuracy in the comment you are replying to already encapsulates the relation of true positives to false positives.
    - fc417fc802 3mo ago
      No I don't believe it does. I interpret 99.9% accurate to mean 1 in 1000 false positives. If 0.1% of your population are terrorists that means each alert has a 50% chance of being correct. That's nowhere near good enough to fully automate things but it is quite reasonable assuming this is merely information provided to a human agent. Whereas if only 0.001% of your population are terrorists then 99 out of 100 alerts are false positives at which point the system is well on its way to being useless. There is an important difference between scenarios where we care about the relative versus absolute frequency of errors.
kleiba2 3mo ago
> even if the scanner is 99.99% accurate, because an even higher percentage of photos are innocent, most matches the scanner will find will be false positives. If the scanner is 99.99% accurate, then most classifications will be correct.
- cenamus 3mo ago
  If you scan 1,000,000 pictures (with let's say 10 CSAM), you'll have 100 false positives and 10 true positives, giving you like only 10% correct results
  - kleiba2 3mo ago
    Ah, sorry, I misread what the OP meant by "matches" - thought they were referring to all classifier outputs, while they specifically meant the positives. I changed my original comment to better reflect what I meant, even though that makes it a bit of a non-sequitur now.
- usrnm 3mo ago
  How many child abusers do you think there are out there?
  - myrmidon 3mo ago
    Even if 10% of population were actively criminal pedos (which is waaaay too high), its pretty safe to assume that the majority of even their online footprint would be ordinary images/messages. So a quota of 0.1% or even less material being detectably criminal sounds realistic (probably not much less, though).
- acksmack 3mo ago
  https://en.wikipedia.org/wiki/Base_rate_fallacy https://en.wikipedia.org/wiki/Base_rate_fallacy
tjpnz 3mo ago
Google have already caused significant hardship to a father for such kinds of photos. What's particularly galling is how they've continued to maintain they were in the right, despite the police saying no crime had been committed. https://www.koffellaw.com/blog/google-ai-technology-flags-dad-who-took-photos-o/ https://www.koffellaw.com/blog/google-ai-technology-flags-da...
- joe_mamba 3mo ago
  Of course google and every other big-tech platform is gonna insta-wipe every account containing detected nudes of children, regardless if you're the parent. The corporate liability of such content being found on their cloud is so insanely nuclear, that they're not gonna wait and ask you "hey are those nudes your own kids or are you a pedo?" before they wipe the account with all pics off their servers.
  - zmmmmm 3mo ago
    And yet the will badger you endlessly to the point their photos app is near unusable to turn on auto sync which slurps up every photo and makes it very awkward to then delete them after. To me, this makes Google a liable party even if real CSAM is stored.

ggthrowaway 3mo ago

CSA makes ppl lose all logic, so is used to justify illogical things. Reminder that none of this has any evidence that it helps CSA, but nobody cares about the actual children.

teaearlgraycold 3mo ago
I feel like the world cares more about stopping the spread of CSAM than it does the actual abusive actions against children.
- mrtesthah 3mo ago
  We can look to certain world "leaders" for confirmation of that.
- pydry 3mo ago
  It's not the world that's the problem it's the small group of individuals trying to create stasi 2.0, hiding behind the children.

englishspot 3mo ago

so much for the principle of least privilege..

bonoboTP 3mo ago

The bad consequences are diffuse, abstract and distant (conspiracy-looking, tinfoil-like), while it's very easy to viscerally understand that "even if they just save one child, it's already worth it". They should give precise numbers of how many such crimes are detected via such means or are expected to be detected per year, and how many of those are not possible to catch through regular investigative work. It just seems ridiculously out of proportion especially that with all this flurry around the topic, the criminals surely aren't using WhatsApp for this any more, but especially won't be once the law is adopted. Sure, many are likely stupid but if they are so stupid, won't they fall into other honeypots? Why are chat apps the best leverage for uncovering this? They'd have to justify this with some sort of data and numbers. Because later they can just come back and say, well unfortunately they are now all using other means, so now we need to break https,we need to ban e2e, we need to ban vpns, tor and foss operating systems etc etc.

iamnothere 3mo ago
They should add to those metrics: hours and funds wasted investigating false positives, reputations ruined from false accusations and investigations, decline in public trust, etc.
u8080 3mo ago
Yeah, and also how many such crimes are actually prosecuted because you know, there is certain island with certain high-ranked people. Anyways, once that implemented noone will report to you and there will be no means of pushing against it because all your online efforts to coordinate will be compromised.

attila-lendvai 3mo ago

especially that the guard applying to protect the henhouse seems to have a suspiciously furry tail...

brikym 3mo ago

[flagged]

wredcoll 3mo ago
That's not a hot take, it's just boring old racism.
bwhitty 3mo ago
Hot take !== unsubstantiated, xenophobic take

greenleafone7 3mo ago

In the list of people that are worried about children.... the government is at the very end.

EarlKing 3mo ago

> stopping child sexual abuse > suddenly touches everyone ..............I see what you did there.

baxtr 3mo ago

I’ve shared this before, I really like this quote: "The urge to save humanity is almost always a false front for the urge to rule." H.L. Mencken

myrmidon 3mo ago
Mencken just has the best quotes. Here's a few of my favorites: > The trouble with fighting for human freedom is that one spends most of one's time defending scoundrels. For it is against scoundrels that oppressive laws are first aimed, and oppression must be stopped at the beginning if it is to be stopped at all. > For every problem, there is a solution that is simple, neat, and wrong. > Freedom of press is limited to those who own one.

latentsea 3mo ago

At some point we just have to accept the kids as collateral.

taneq 3mo ago
What, like with guns? Never!
- latentsea 3mo ago
  If you want the guns, then yes. I come from somewhere without guns, and as a result without school shootings. Nobody here wants guns as a part of everyday life. America seems to want them.

sneak 3mo ago

Technology is, furthermore, the wrong place to address child abuse of any kind, sexual or otherwise. This is like trying to prevent burglary by working with the factory that manufactures pry bars.

f6v 3mo ago

> Most everyone would love to see more work on stopping child sexual abuse. By the parents. Install parental controls that only allow to message you and closest relatives. Problem solved.

eunos 3mo ago

> Rather than narrow and specific - it's a broad based law Because narrow law is easier to avoid or find the loophole and a single case is enough to induce panic and anger.

AlexanderHanff 3mo ago

I have put up a list of all the MEPs who voted for the urgency procedure yesterday (in breach of EU rules) as well as their voting history on fundamental rights issues and who has been lobbying them: https://www.thatprivacyguy.com/blog/chat-control-the-415-who-failed-you/ https://www.thatprivacyguy.com/blog/chat-control-the-415-who...

vaylian 3mo ago
Thanks. Please note that your link doesn't work with the tor browser.
- AlexanderHanff 3mo ago
  Not sure why, it is working fine everywhere else - I see in Tor it gives an invalid certificate error, but the Lets Encrypt certificate is working fine in other browsers, so seems to be a Tor thing specifically. I will investigate.
- AlexanderHanff 3mo ago
  I cannot see what is causing the issue, the certificate's full chain is sent, the clock is synced, the cert is showing zero errors in OpenSSL - so this is very confusing. The irony is, you don't actually need Tor on my site because there is no logging, no third parties, no adtech etc. it is just static HTML files - so whereas I would normally recommend Tor I designed the site specifically to be privacy first. I will try to figure out what is going on though because obviously I am fully supportive of people protecting their privacy with Tor.
- AlexanderHanff 3mo ago
  OK it should be fixed now - there was a rate limit which doesn't normally land but due to multiple Tor users coming through the same exit node, it was triggering the limit. Nothing to do with the certs just told me a cert error because it never finished the handshake. I tested through Tor on multiple machines now and multiple circuits and is working clean - thanks for the heads up.
almaya 3mo ago

order-matters 3mo ago

its also just disastrous for signal to noise ratios. scanning everything means any sort of error rate is going to cause massive amounts of incorrect labelling. this means innocent things getting flagged and put into a system where people are treated like offenders when they arent until they can get an actual human with authority to review their circumstances (not guaranteed to happen at all btw), or some actual offenders get away with more bc they passed a scan outlier cases aside, there is also just a large amount of processing power that will go into this, the service can only be worse off for it. Privacy is not just about being able to hide things, it is also about being in control of how you present to the world. not because that control is maniupulative but because we all exist within our own microcosms of uniqueness, using words slightly differently than each other, and having certain balances of intention and meaning with those we send messages to that cannot be fully presumed from a 3rd party. even in images. Are they really saying "if you want to send private messages then go make your own network" ?

dpoloncsak 3mo ago
>"if you want to send private messages then go make your own network" Unironically, we should all move to using TOR. Anyone setup a .onion mirror for HN yet? I'd assume usual HN-mirror-rules, no login or posting but free to view...
- order-matters 3mo ago
  Tor is full of honeypots, its good for getting around 3rd party snooping but useless for government level privacy
  - deleted 3mo ago
    [deleted]
pixl97 3mo ago
>Are they really saying "if you want to send private messages then go make your own network No, because with the way things work, they'll make that illegal next.