8 ms·

Scanning phones to detect child abuse evidence is harmful, 'magical' thinking

141 points by arkadiyt 4y ago

4oo4 4y ago
Never expected the GCHQ to come out against this. In a paper titled Chat Control or Child Protection?, to be distributed via ArXiv, Anderson offers a rebuttal to arguments advanced in July by UK government cyber and intelligence experts Ian Levy, technical director of the UK National Cyber Security Centre, and Crispin Robinson, technical director of cryptanalysis at Government Communications Headquarters (GCHQ), the UK's equivalent to the NSA.
- soulofmischief 4y ago
  Single-line block quoting is impossible to parse on mobile screens.
  - lolinder 4y ago
    I remember that used to be the case, but for the past few months it's been wrapping for me (as it is here, for me.) I wonder if you have a cached version of an old stylesheet or something?
    - soulofmischief 4y ago
      I'm using Glider, a mobile HN reader.
      - dicknuckle 4y ago
        Same. I'm pretty sure it's just the client.
      - MerelyMortal 4y ago
        It used to be an issue for me too, but now it looks fine on Firefox for Android. In this case, I think it's your third party app that has the issue, not HN.
      - lolinder 4y ago
        Sounds like a bug report is due to Glider, then. Now that code blocks are fixed on mobile in the default stylesheet, I think they're fair game.
        soulofmischief 4y ago
        Thanks, yep seems like it!
  - 4oo4 4y ago
    Yeah it looks OK for me, using Firefox on Android.
- cvwright 4y ago
  GCHQ is for it. Ross Anderson is arguing against them. He’s right BTW.
  - 4oo4 4y ago
    Yeah he is. Wow, totally misread that, that makes a lot more sense.
- nl 4y ago
  > Never expected the GCHQ to come out against this. They haven't. > Anderson offers a rebuttal to arguments advanced .. by .. Crispin Robinson, technical director of cryptanalysis at Government Communications Headquarters (GCHQ) So GCHQ is putting out the argument that is being rebutted.
remote_phone 4y ago
Because they aren’t scanning phone to detect child abuse. That’s the excuse and gateway into your phone. If you say “no” then you are supporting pedophilia. However, it’s not the ultimate objective.
- junofan 4y ago
  Scanning phones would be great for detecting illicit gun ownership.
  - onionisafruit 4y ago
    Abortions too
    - reitanqild 4y ago
      ... and with that both sides of American politics should be covered. My favourite wrench to throw in such gears has been that whatever affordances the current (maybe reasonable) government affords itself will be inherited by the next. Whatever Joe Biden can do, the next Republican president can do, too and vice versa. Or, to take a historic perspective: whatever rules and exist and whatever data (e.g. list of Jews) existed before the Nazis came to power were also available to the Nazis even if the original reason for these rules and data collections were well meaning and maybe even good in context.
nl 4y ago
Note that this is an argument about the use of client side language models to detect grooming in chat(!), not Apple's CSAM program.
- stephen_g 4y ago
  Many of the arguments in the paper apply to any AI-based system for scanning images client-side too (and some even to hash based systems like the one Apple was talking about, but yes, there are other, specific, valid arguments against hash-based client side scanning too).
Manu40 4y ago
Edit: 1 hour later: I am signing off for the night folks. If I don't respond to new replies to my comment, that's why. You all have a good night, or day. -- OG comment -- Forgive me for being potentially controversial here, but considering the topic I think it should be fine for me to suggest this, especially since this measure is (Edit: In my opinion after reading it) about 'grooming', which requires interaction. We don't allow children/minors into bars, and many other places where adults congregate, right? Why are we allowing them on the internet? Seems rather simple to me that if we just remove their access to the internet, that nearly almost all the potential issues involving children cease to exist. Companies can't exploit them. (as easily) Creeps can't harm them. (as easily) Etc and so forth. Sure, the internet is a wonderful tool for learning, and having fun, and communicating. But so is going outside, playing with your friends, and paying attention in school. That's all I have to say on this. --- Edit:30 minutes after post: I guess there is one more solution. Make a kids only internet, if we really must allow them to have such a thing. All adults are restricted from it. *except perhaps their parents, but even then that opens new loopholes. So even that's not perfect.
- mixmastamyk 4y ago
  Generally agreed but that’s not a solution to what this article is about.
  - Manu40 4y ago
    > "Meanwhile the UK is considering its own Online Safety Bill, which also imagines bypassing encryption via device-side scanning. Similar proposals, such as the EARN IT bill, keep surfacing in the US. The paper by Levy and Robinson – itself a response to a paper opposing device-side scanning that Anderson co-authored with 13 other security experts in 2021 – outlines the various types of harms children may encounter online: consensual peer-to-peer indecent image sharing; viral image sharing; offender to offender indecent image/video sharing; offender to victim grooming; offender to offender communication; offender to offender group communication; and streaming of on-demand contact abuse." ---- Maybe not specifically only grooming, but considering that most of the potential harm caused is going to start with interaction between the creep and the child, that means grooming is going to be one of the starting points they will be on the look out for, which is why this is all about client side infiltration. Seems to me that by nipping that problem in the bud first and foremost, by removing the potential for interaction at all in the first place; we remove the potential for harm to occur as well (edit: Which means there is no need to snoop on the client side.) But like you said, you already agree for the most part.
erenyeager 4y ago
While I don’t support privacy invasions, I think, as a teen who was exploited online by adults, that there needs to be more protection for young people online. A client side scanning solution sounds ideal in this regard.
- reitanqild 4y ago
  I am saddened to hear. But I don't think we can use the possibility of bad things happening to restrict everyones liberty. What if someone writes: ... as a kid who was exploited at home by adults ? Can that be used as an excuse to check peoples homes? FTR: I think this is a hard problem. But I think the correct way to address such problems must be to teach digital self defense, not to mandate surveillance at home and in everyone's pockets.
totetsu 4y ago
> "The idea that complex social problems are amenable to cheap technical solutions is the siren song of the software salesman and has lured many a gullible government department on to the rocks," This is the bigger issues. The charitable reading is that politicians are gullible and pursing magical thinking led on by greedy tech bros. Another way to look at it is a political ruling in-group that do not support democratic ideas of people choosing how to rule themselves, and pursue any and everything to make sure they stay the big boss. *edit* On the Non-Theoretical note, I recently turned Stable diffusion to generating images from Mnemonic stories used for remembering Chinese characters. Because we remember things that .. appeal to our atavistic natures .. better than dull everyday things, some of the public stories I fed into SD to make images involved nudity, violence, some children. There are no child abuse stories in there, nor where the images close to child abuse images, but it did generate the occasional naked baby or conjoined twins example for 世. When I had to back these images up on a apple device I defiantly felt worried I'd have a knock on my door some day. Will I have to self censor what happens on my own devices now? I'm sure mnemonics is not the only example of some harmless private realm of thinking that needs to be disconnected from social norms. Will these things have to stay off of our normative enforcing technology?
- kweingar 4y ago
  > On the Non-Theoretical note, I recently turned Stable diffusion to generating images from Mnemonic stories used for remembering Chinese characters. What a great idea! I’m learning Chinese characters through mnemonic, and I’m also an SD enthusiast, but I never thought to combine the two
LatteLazy 4y ago
Increasingly I worry that people are not engaged enough to actually think about these points. Instead they hear "child abuse" and their brain immediately jumps to enforcement mode. Based on that, we should not point out that this is bs (which it is). We should point out that it will give police access to kids personal pics, or point out it will be used for tax enforcement. I hope I am being too cynical...
ETH_start 4y ago
It seems the purveyors of bad ideas never stop introducing new ones and organizing their imposition on the rest of us.
thewest039 4y ago

141 points by arkadiyt 4y ago

4oo4 4y ago

Never expected the GCHQ to come out against this. In a paper titled Chat Control or Child Protection?, to be distributed via ArXiv, Anderson offers a rebuttal to arguments advanced in July by UK government cyber and intelligence experts Ian Levy, technical director of the UK National Cyber Security Centre, and Crispin Robinson, technical director of cryptanalysis at Government Communications Headquarters (GCHQ), the UK's equivalent to the NSA.

soulofmischief 4y ago
Single-line block quoting is impossible to parse on mobile screens.
- lolinder 4y ago
  I remember that used to be the case, but for the past few months it's been wrapping for me (as it is here, for me.) I wonder if you have a cached version of an old stylesheet or something?
  - soulofmischief 4y ago
    I'm using Glider, a mobile HN reader.
    - dicknuckle 4y ago
      Same. I'm pretty sure it's just the client.
    - MerelyMortal 4y ago
      It used to be an issue for me too, but now it looks fine on Firefox for Android. In this case, I think it's your third party app that has the issue, not HN.
    - lolinder 4y ago
      Sounds like a bug report is due to Glider, then. Now that code blocks are fixed on mobile in the default stylesheet, I think they're fair game.
      - soulofmischief 4y ago
        Thanks, yep seems like it!
- 4oo4 4y ago
  Yeah it looks OK for me, using Firefox on Android.
cvwright 4y ago
GCHQ is for it. Ross Anderson is arguing against them. He’s right BTW.
- 4oo4 4y ago
  Yeah he is. Wow, totally misread that, that makes a lot more sense.
nl 4y ago
> Never expected the GCHQ to come out against this. They haven't. > Anderson offers a rebuttal to arguments advanced .. by .. Crispin Robinson, technical director of cryptanalysis at Government Communications Headquarters (GCHQ) So GCHQ is putting out the argument that is being rebutted.

remote_phone 4y ago

Because they aren’t scanning phone to detect child abuse. That’s the excuse and gateway into your phone. If you say “no” then you are supporting pedophilia. However, it’s not the ultimate objective.

junofan 4y ago
Scanning phones would be great for detecting illicit gun ownership.
- onionisafruit 4y ago
  Abortions too
  - reitanqild 4y ago
    ... and with that both sides of American politics should be covered. My favourite wrench to throw in such gears has been that whatever affordances the current (maybe reasonable) government affords itself will be inherited by the next. Whatever Joe Biden can do, the next Republican president can do, too and vice versa. Or, to take a historic perspective: whatever rules and exist and whatever data (e.g. list of Jews) existed before the Nazis came to power were also available to the Nazis even if the original reason for these rules and data collections were well meaning and maybe even good in context.

nl 4y ago

Note that this is an argument about the use of client side language models to detect grooming in chat(!), not Apple's CSAM program.

stephen_g 4y ago
Many of the arguments in the paper apply to any AI-based system for scanning images client-side too (and some even to hash based systems like the one Apple was talking about, but yes, there are other, specific, valid arguments against hash-based client side scanning too).

Manu40 4y ago

Edit: 1 hour later: I am signing off for the night folks. If I don't respond to new replies to my comment, that's why. You all have a good night, or day. -- OG comment -- Forgive me for being potentially controversial here, but considering the topic I think it should be fine for me to suggest this, especially since this measure is (Edit: In my opinion after reading it) about 'grooming', which requires interaction. We don't allow children/minors into bars, and many other places where adults congregate, right? Why are we allowing them on the internet? Seems rather simple to me that if we just remove their access to the internet, that nearly almost all the potential issues involving children cease to exist. Companies can't exploit them. (as easily) Creeps can't harm them. (as easily) Etc and so forth. Sure, the internet is a wonderful tool for learning, and having fun, and communicating. But so is going outside, playing with your friends, and paying attention in school. That's all I have to say on this. --- Edit:30 minutes after post: I guess there is one more solution. Make a kids only internet, if we really must allow them to have such a thing. All adults are restricted from it. *except perhaps their parents, but even then that opens new loopholes. So even that's not perfect.

mixmastamyk 4y ago
Generally agreed but that’s not a solution to what this article is about.
- Manu40 4y ago
  > "Meanwhile the UK is considering its own Online Safety Bill, which also imagines bypassing encryption via device-side scanning. Similar proposals, such as the EARN IT bill, keep surfacing in the US. The paper by Levy and Robinson – itself a response to a paper opposing device-side scanning that Anderson co-authored with 13 other security experts in 2021 – outlines the various types of harms children may encounter online: consensual peer-to-peer indecent image sharing; viral image sharing; offender to offender indecent image/video sharing; offender to victim grooming; offender to offender communication; offender to offender group communication; and streaming of on-demand contact abuse." ---- Maybe not specifically only grooming, but considering that most of the potential harm caused is going to start with interaction between the creep and the child, that means grooming is going to be one of the starting points they will be on the look out for, which is why this is all about client side infiltration. Seems to me that by nipping that problem in the bud first and foremost, by removing the potential for interaction at all in the first place; we remove the potential for harm to occur as well (edit: Which means there is no need to snoop on the client side.) But like you said, you already agree for the most part.

erenyeager 4y ago

While I don’t support privacy invasions, I think, as a teen who was exploited online by adults, that there needs to be more protection for young people online. A client side scanning solution sounds ideal in this regard.

reitanqild 4y ago
I am saddened to hear. But I don't think we can use the possibility of bad things happening to restrict everyones liberty. What if someone writes: ... as a kid who was exploited at home by adults ? Can that be used as an excuse to check peoples homes? FTR: I think this is a hard problem. But I think the correct way to address such problems must be to teach digital self defense, not to mandate surveillance at home and in everyone's pockets.

totetsu 4y ago

> "The idea that complex social problems are amenable to cheap technical solutions is the siren song of the software salesman and has lured many a gullible government department on to the rocks," This is the bigger issues. The charitable reading is that politicians are gullible and pursing magical thinking led on by greedy tech bros. Another way to look at it is a political ruling in-group that do not support democratic ideas of people choosing how to rule themselves, and pursue any and everything to make sure they stay the big boss. *edit* On the Non-Theoretical note, I recently turned Stable diffusion to generating images from Mnemonic stories used for remembering Chinese characters. Because we remember things that .. appeal to our atavistic natures .. better than dull everyday things, some of the public stories I fed into SD to make images involved nudity, violence, some children. There are no child abuse stories in there, nor where the images close to child abuse images, but it did generate the occasional naked baby or conjoined twins example for 世. When I had to back these images up on a apple device I defiantly felt worried I'd have a knock on my door some day. Will I have to self censor what happens on my own devices now? I'm sure mnemonics is not the only example of some harmless private realm of thinking that needs to be disconnected from social norms. Will these things have to stay off of our normative enforcing technology?

kweingar 4y ago
> On the Non-Theoretical note, I recently turned Stable diffusion to generating images from Mnemonic stories used for remembering Chinese characters. What a great idea! I’m learning Chinese characters through mnemonic, and I’m also an SD enthusiast, but I never thought to combine the two

LatteLazy 4y ago

Increasingly I worry that people are not engaged enough to actually think about these points. Instead they hear "child abuse" and their brain immediately jumps to enforcement mode. Based on that, we should not point out that this is bs (which it is). We should point out that it will give police access to kids personal pics, or point out it will be used for tax enforcement. I hope I am being too cynical...

ETH_start 4y ago

It seems the purveyors of bad ideas never stop introducing new ones and organizing their imposition on the rest of us.

thewest039 4y ago