4 ms·

Ok what if someone did sneak some non-CSAM? Some random person would get flagged, the photos would be manually reviewed, and then nothing bad would happen. Does

Ok what if someone did sneak some non-CSAM? Some random person would get flagged, the photos would be manually reviewed, and then nothing bad would happen. Doesn’t seem like a very big deal.

l33t2328 4y ago
I disagree. The manual review process uses a downscaled/greyscaled version of the image, and there are known attacks on this type of preporcessing.
- xadhominemx 4y ago
  Ok what if someone attacks the preprocessing. What bad thing happens next?
  - falcolas 4y ago
    CASM can be distributed unimpeded, negating the positive aspects of such technology.
  - l33t2328 4y ago
    Imagine Xi puts a hash of Winnie on the list and the preprocessing attack is done to conceal CSAM in pictures of Winnie. Such images could be distributed to launch an attack.
falcolas 4y ago
Here's a hypothetical: The FBI is responsible in the US for investigating interstate sex trafficking and CSAM (Note, the 'M' stands for "material" - not just photos). The FBI subsequently provides the CSAM hash list, and volunteers manpower for the manual reviews. Where does this go wrong? When the FBI expands the CSAM hash list to include "The Anarchists' Cookbook".
- xadhominemx 4y ago
  They expand the hashlist to include images of the cover of “The Anarchists Cookbook?”
  - whatshisface 4y ago
    The EU law in the works doesn't only cover images.
  - ImPostingOnHN 4y ago
    how would you know if they did?
  - falcolas 4y ago
    Only Apple's implementation was limited to images. There's nothing in the current or proposed laws that prevents searching other media. Otherwise, since an ebook is just a zipped HTML page with its resources, it leaves a huge loophole in the identification of CSAM.
lotsofpulp 4y ago
> and then nothing bad would happen. Source? If you have ever been a target of the police/legal/judicial system, there is rarely a case where the individual has nothing bad happen. The government actors’ impetus is to make sure they are not found at fault, not to ensure that the individual is actually guilty or innocent. At a minimum, the individual will lose many hours and go through a lot of stress. See any and all police misconduct cases in the US, as well as border patrol or really any other dealings with a government agent. The only protection is to be able to afford highly connected lawyers or be connected yourself.
- xadhominemx 4y ago
  People are investigated or are considered within the scope of an investigation all the time. We don’t learn/ care about it if there are no arrests. What specifically do you think could go wrong here?
  - ImPostingOnHN 4y ago
    What could go wrong was explained above
bakuninsbart 4y ago
- AI is generally shit at not being biased, this is more of a meta point, but it is important to remember that all abuses and incosistencies will hit some people harder than other. - Some random government guy will look through my fotos. I do not want that. - There is no proper oversight for what is actually scanned, and on which paramaters. The gov could just "accidentally" also scan for other crimes or things it doesn't like. Democracy isn't set in stone, and this would be extremely useful infrastructure for a tyrant. - Banning encryption or requiring backdoors makes all of us less safe from third parties.
- xadhominemx 4y ago
  > that all abuses and incosistencies will hit some people harder than other How specifically could that play out in this instance? > The gov could just "accidentally" also scan for other crimes or things it doesn't like. Yes, if a government decides the rule of law does not matter bad things happen. That was true in the a analog days as well. > Banning encryption or requiring backdoors makes all of us less safe from third parties. This is not really a backdoor. Do you understand how the technology works?
  - shakna 4y ago
    > How specifically could that play out in this instance? As AI is famously bad at matching against dark skinned individuals [0], all photos of black kids now flag as CSAM, whether or not any part of them is even dubious at all. [0] https://www.theverge.com/2020/6/24/21301759/facial-recognition-detroit-police-wrongful-arrest-robert-williams-artificial-intelligence https://www.theverge.com/2020/6/24/21301759/facial-recogniti...
    - ImPostingOnHN 4y ago
      Or, for the same reason, law enforcement switches methodologies from proper, normal investigation to just scanning, since "its so easy!", but the software fails to recognize dark skinned children, and as a result all law enforcement efforts end up helping only lighter skinned victims

Ok what if someone did sneak some non-CSAM? Some random person would get flagged, the photos would be manually reviewed, and then nothing bad would happen. Doesn’t seem like a very big deal.

l33t2328 4y ago

I disagree. The manual review process uses a downscaled/greyscaled version of the image, and there are known attacks on this type of preporcessing.

xadhominemx 4y ago
Ok what if someone attacks the preprocessing. What bad thing happens next?
- falcolas 4y ago
  CASM can be distributed unimpeded, negating the positive aspects of such technology.
- l33t2328 4y ago
  Imagine Xi puts a hash of Winnie on the list and the preprocessing attack is done to conceal CSAM in pictures of Winnie. Such images could be distributed to launch an attack.

falcolas 4y ago

Here's a hypothetical: The FBI is responsible in the US for investigating interstate sex trafficking and CSAM (Note, the 'M' stands for "material" - not just photos). The FBI subsequently provides the CSAM hash list, and volunteers manpower for the manual reviews. Where does this go wrong? When the FBI expands the CSAM hash list to include "The Anarchists' Cookbook".

xadhominemx 4y ago
They expand the hashlist to include images of the cover of “The Anarchists Cookbook?”
- whatshisface 4y ago
  The EU law in the works doesn't only cover images.
- ImPostingOnHN 4y ago
  how would you know if they did?
- falcolas 4y ago
  Only Apple's implementation was limited to images. There's nothing in the current or proposed laws that prevents searching other media. Otherwise, since an ebook is just a zipped HTML page with its resources, it leaves a huge loophole in the identification of CSAM.

lotsofpulp 4y ago

> and then nothing bad would happen. Source? If you have ever been a target of the police/legal/judicial system, there is rarely a case where the individual has nothing bad happen. The government actors’ impetus is to make sure they are not found at fault, not to ensure that the individual is actually guilty or innocent. At a minimum, the individual will lose many hours and go through a lot of stress. See any and all police misconduct cases in the US, as well as border patrol or really any other dealings with a government agent. The only protection is to be able to afford highly connected lawyers or be connected yourself.

xadhominemx 4y ago
People are investigated or are considered within the scope of an investigation all the time. We don’t learn/ care about it if there are no arrests. What specifically do you think could go wrong here?
- ImPostingOnHN 4y ago
  What could go wrong was explained above

bakuninsbart 4y ago

- AI is generally shit at not being biased, this is more of a meta point, but it is important to remember that all abuses and incosistencies will hit some people harder than other. - Some random government guy will look through my fotos. I do not want that. - There is no proper oversight for what is actually scanned, and on which paramaters. The gov could just "accidentally" also scan for other crimes or things it doesn't like. Democracy isn't set in stone, and this would be extremely useful infrastructure for a tyrant. - Banning encryption or requiring backdoors makes all of us less safe from third parties.

xadhominemx 4y ago
> that all abuses and incosistencies will hit some people harder than other How specifically could that play out in this instance? > The gov could just "accidentally" also scan for other crimes or things it doesn't like. Yes, if a government decides the rule of law does not matter bad things happen. That was true in the a analog days as well. > Banning encryption or requiring backdoors makes all of us less safe from third parties. This is not really a backdoor. Do you understand how the technology works?
- shakna 4y ago
  > How specifically could that play out in this instance? As AI is famously bad at matching against dark skinned individuals [0], all photos of black kids now flag as CSAM, whether or not any part of them is even dubious at all. [0] https://www.theverge.com/2020/6/24/21301759/facial-recognition-detroit-police-wrongful-arrest-robert-williams-artificial-intelligence https://www.theverge.com/2020/6/24/21301759/facial-recogniti...
  - ImPostingOnHN 4y ago
    Or, for the same reason, law enforcement switches methodologies from proper, normal investigation to just scanning, since "its so easy!", but the software fails to recognize dark skinned children, and as a result all law enforcement efforts end up helping only lighter skinned victims