11 ms·

Apple’s ill thought out on-device CSAM detection and thankfully, the removal of it from its plans after heavy pushback comes to mind. Any time a specific abili

by AnonC 3y ago

Apple’s ill thought out on-device CSAM detection and thankfully, the removal of it from its plans after heavy pushback comes to mind. Any time a specific ability is granted or is available to law enforcement, it will expand to broaden surveillance on everything. CSAM is horrific, despicable and devastating. But the way law enforcement treats it as an easy “in” to push for more surveillance and more powers for itself is shameful.

dcow 3y ago
It’s not shameful that people try to grab power to stop evil, it’s natural. What’s shameful is that we don’t have more political checks against these type of power grabs. And that citizens who could stand up and fight against abusive surveillance are increasingly apathetic.
- errantmind 3y ago
  The statement is that it is shameful to grab for powers that invade the privacy of law abiding citizens.
  - ethbr1 3y ago
    The quandary is that powers that "invade the privacy of law abiding citizens" are also powers "to stop evil." Pretending they aren't is part of the problem, as it empowers those who would push them to publicly advertise the latter good in a vacuum of silence from the tech side. Something like 'Personal privacy is more important than maximizing law enforcement efficiency, including of CSAM' is a more honest, complete position.
    - qwytw 3y ago
      > to stop evil It's not at all obvious whether stopping CSAM is really the primary goal of some of the people who are pushing these regulations. It seems just like a justification to invade personal privacy.
      - throwaway290 3y ago
        You are speculating about intents some other people might have. Let's take the issue in vacuum first. Either you hold your privacy as more important than X, or you are willing to compromise some of your privacy in the name of X-- there's no third option. Eg. in case of airport security or CCTV in some public space suddenly everyone is OK compromising personal privacy in the name of personal life and safety. Now finally let's get back to those other people whom you suspect of having an agenda to surveil everybody. If they honestly tried to combat child abuse, how do you see them going about it?
        notpushkin 3y ago
        > in case of airport security Not everyone is OK – lots of people argue it's a security theatre.
        pasc1878 3y ago
        Sopmer of it is theatre but screening people to make sure they don't have a bomb ios not theatre - people will do this as has been shown.
        throwaway290 3y ago
        It is a bit weird in that screening started only after bombing... a useful metric would be to see how many times bookings/hijackings were thwarted. (which I would guess many)
        pasc1878 3y ago
        No screening started after hijacking in the 1970s.
        throwaway290 3y ago
        Let's see if you think it's a security theatre next time you fly from Jordan to Israel. Or actually anywhere within the US, where gun carry is allowed...
- colordrops 3y ago
  It's absolutely shameful. These people are not naive children. They know exactly the ramifications of the tradeoff, which indicates that they are not doing so in good faith, but rather at the behest of vested interests.
  - justinclift 3y ago
    Or they're mentally unsound of some variety.
- ekianjo 3y ago
  > try to grab power to stop evil, it’s natural Those who seek this kind of power over others are themselves EVIL by definition
- noobermin 3y ago
  Pedantic but I would still say it's shameful although expected. Like that it is shameful to have your car stolen if you leave it unlocked with the keys in clear view on the driver's seat, although it is expected.
- Xelbair 3y ago
  Mostly because they can go for those power grabs again, and again - ad nauseam. After a while populace is just bored of fighting, and only few people care.
- ajsnigrutin 3y ago
  The problem is, that we have to complain loudly every goddamn time, get the media, protest, write articles, etc., to get them to pull back. They only have to succeed once, and the law is written and stamped.
  - dcow 3y ago
    I know, sigh. Which is exactly why we need more structural defense against this behavior. I think it’s about time for a modern day constitutional convention.
    - lost_tourist 3y ago
      People who prefer liberty over tyranny have to win almost every time, it's only takes one failure to lose liberty for a hundred--two hundred?-- years.
nucleardog 3y ago
> Apple’s ill thought out on-device CSAM detection and thankfully, the removal of it from its plans after heavy pushback comes to mind. Calling Apple's CSAM detection "ill thought out" seems to me to be letting the perfect be the enemy of the good. Remember when everyone was up in arms about W3C standardizing DRM? Yes, the world would be better without DRM. But the DRM exists and will continue to exist. Arguing for not standardizing DRM isn't arguing for DRM to not exist, it's arguing for it to not be standardized. The encrypted media extensions let me watch DRM-protected content on Linux instead of being locked out of it all because none of the proprietary DRM everyone's using works on anything but Windows/OSX. Linux and Firefox and a DRM blob so I can watch Netflix is better than being forced to use Windows 10 with all its telemetry, Microsoft proprietary DRM, and a browser from Google or Microsoft to watch things. Yes. It would be better if the government stayed the hell away from my files. Apple not implementing their CSAM scanning didn't get rid of the underlying issue or argument. It just means that it's no longer the tech industry setting the standard for how this will work--it's going to be the government. Apple's implementation[0] was about as privacy preserving as we can hope for. It only scanned media that was about to be uploaded to their cloud service. It did scanning on device and used crypto to ensure, mathematically, that they couldn't even access the _hash_ of matching images until such a point that enough images matched to cross a threshold. They had client devices feed in fake matches to obscure even the number of potential matches that occur before reaching the threshold. At any no point in any of this is it possible for them to retrieve even the hash of an image that does not match, even after you've passed the threshold. Would it be better if none of this happened at all? Sure. Is this a _fuck_ of a lot better than what we're seeing Europol pushing for? Absolutely. Apple cancelling their scanning was a short term win. This isn't a new problem, and this isn't one that's going away. We can throw all the technical solutions we want at it, but it's not a technical problem. ("Sorry, can't scan user's content it's all end-to-end encrypted!". "Don't care. You wrote the encryption. Work around it.". "It's impossible!". "Okay, enjoy your new regulation that all encryption has to have a backdoor HTH HAND.") I'd rather lose the battle and win the war. Let's put the most privacy-preserving CSAM scanning we can in place and take that card out of play. Let the regulators come out and try and explain how "Well yeah, you're scanning every image for CSAM but, uh, it's not enough. We do really need to see _all_ the images people have on their phones!". We're not making an argument, we're drawing a hard line and standing in place. The Europols of the world are not going to stop pushing. Apple's big, but not "override the EU" big. When push comes to shove, we will lose. The EU _will_ respond with regulations. Maybe not now, but it should be obvious the way the winds are shifting. I'd bet my left testicle their vision for this is much more onerous than what Apple was proposing. But hey, at least we can tell our children (away from our phones or any other electronics, and probably standing somewhere deep in the woods) that we were proudly defiant to the end. [0]: https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
jacooper 3y ago
Apple announcing that stupid Csam scanning started all of this and they proved it's possible. An absolutely stupid and disastrous move.
- zimpenfish 3y ago
  > Apple announcing that stupid Csam scanning started all of this Except all of the cloud providers are already scanning uploaded photos for CSAM and have been for years. Trying to blame Apple for this is insane.
  - jacooper 3y ago
    This isnt comparable. Apple was going to scan local photos, not cloud stored ones. No matter the source, no matter the app. And the phone will report you to the police if the algorithm marks any local photos as ones reported by the police using a "neural hash", which has a non-zero amount of hash collisions. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issues/1 https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...
    - zimpenfish 3y ago
      > Apple was going to scan local photos, not cloud stored ones. At the point of upload to the cloud service -where they would be scanned anyway-. > if the algorithm marks any local photos No, it required a threshold of N photos to match before they were submitted for human verification. > which has a non-zero amount of hash collisions Hence the threshold and human verification step.
      - ajsnigrutin 3y ago
        So, imagine your new house having cameras and microphones all over the place, that you cannot turn off, recording 24/7, but "only locally". If there's screaming, could be TV, could be just an argument, could be rough sex, drama practice, or maybe even violence and murder, it will mark those recordings and after a few repeats it'll send them to a person to look at your private recordings to see if it's just some bdsm play or if you're murdering your wife. Oh, the police officer looked at the video and it was just bdsm? Ok, continue until the next threshold.
lost_tourist 3y ago
Also that would have set the precedence for the police to use apple as a proxy to put software on your phone/computer to monitor everything you do and scan for them, while they lean back in office chairs and wait for a hit. Same would have been extended to any and all government bureaucracy. No warrant, no innocent until proven guilty; guilty until proven innocent is their credo.

by AnonC 3y ago

dcow 3y ago

It’s not shameful that people try to grab power to stop evil, it’s natural. What’s shameful is that we don’t have more political checks against these type of power grabs. And that citizens who could stand up and fight against abusive surveillance are increasingly apathetic.

errantmind 3y ago
The statement is that it is shameful to grab for powers that invade the privacy of law abiding citizens.
- ethbr1 3y ago
  The quandary is that powers that "invade the privacy of law abiding citizens" are also powers "to stop evil." Pretending they aren't is part of the problem, as it empowers those who would push them to publicly advertise the latter good in a vacuum of silence from the tech side. Something like 'Personal privacy is more important than maximizing law enforcement efficiency, including of CSAM' is a more honest, complete position.
  - qwytw 3y ago
    > to stop evil It's not at all obvious whether stopping CSAM is really the primary goal of some of the people who are pushing these regulations. It seems just like a justification to invade personal privacy.
    - throwaway290 3y ago
      You are speculating about intents some other people might have. Let's take the issue in vacuum first. Either you hold your privacy as more important than X, or you are willing to compromise some of your privacy in the name of X-- there's no third option. Eg. in case of airport security or CCTV in some public space suddenly everyone is OK compromising personal privacy in the name of personal life and safety. Now finally let's get back to those other people whom you suspect of having an agenda to surveil everybody. If they honestly tried to combat child abuse, how do you see them going about it?
      - notpushkin 3y ago
        > in case of airport security Not everyone is OK – lots of people argue it's a security theatre.
        pasc1878 3y ago
        Sopmer of it is theatre but screening people to make sure they don't have a bomb ios not theatre - people will do this as has been shown.
        throwaway290 3y ago
        It is a bit weird in that screening started only after bombing... a useful metric would be to see how many times bookings/hijackings were thwarted. (which I would guess many)
        pasc1878 3y ago
        No screening started after hijacking in the 1970s.
        throwaway290 3y ago
        Let's see if you think it's a security theatre next time you fly from Jordan to Israel. Or actually anywhere within the US, where gun carry is allowed...
colordrops 3y ago
It's absolutely shameful. These people are not naive children. They know exactly the ramifications of the tradeoff, which indicates that they are not doing so in good faith, but rather at the behest of vested interests.
- justinclift 3y ago
  Or they're mentally unsound of some variety.
ekianjo 3y ago
> try to grab power to stop evil, it’s natural Those who seek this kind of power over others are themselves EVIL by definition
noobermin 3y ago
Pedantic but I would still say it's shameful although expected. Like that it is shameful to have your car stolen if you leave it unlocked with the keys in clear view on the driver's seat, although it is expected.
Xelbair 3y ago
Mostly because they can go for those power grabs again, and again - ad nauseam. After a while populace is just bored of fighting, and only few people care.
ajsnigrutin 3y ago
The problem is, that we have to complain loudly every goddamn time, get the media, protest, write articles, etc., to get them to pull back. They only have to succeed once, and the law is written and stamped.
- dcow 3y ago
  I know, sigh. Which is exactly why we need more structural defense against this behavior. I think it’s about time for a modern day constitutional convention.
  - lost_tourist 3y ago
    People who prefer liberty over tyranny have to win almost every time, it's only takes one failure to lose liberty for a hundred--two hundred?-- years.

nucleardog 3y ago

> Apple’s ill thought out on-device CSAM detection and thankfully, the removal of it from its plans after heavy pushback comes to mind. Calling Apple's CSAM detection "ill thought out" seems to me to be letting the perfect be the enemy of the good. Remember when everyone was up in arms about W3C standardizing DRM? Yes, the world would be better without DRM. But the DRM exists and will continue to exist. Arguing for not standardizing DRM isn't arguing for DRM to not exist, it's arguing for it to not be standardized. The encrypted media extensions let me watch DRM-protected content on Linux instead of being locked out of it all because none of the proprietary DRM everyone's using works on anything but Windows/OSX. Linux and Firefox and a DRM blob so I can watch Netflix is better than being forced to use Windows 10 with all its telemetry, Microsoft proprietary DRM, and a browser from Google or Microsoft to watch things. Yes. It would be better if the government stayed the hell away from my files. Apple not implementing their CSAM scanning didn't get rid of the underlying issue or argument. It just means that it's no longer the tech industry setting the standard for how this will work--it's going to be the government. Apple's implementation[0] was about as privacy preserving as we can hope for. It only scanned media that was about to be uploaded to their cloud service. It did scanning on device and used crypto to ensure, mathematically, that they couldn't even access the _hash_ of matching images until such a point that enough images matched to cross a threshold. They had client devices feed in fake matches to obscure even the number of potential matches that occur before reaching the threshold. At any no point in any of this is it possible for them to retrieve even the hash of an image that does not match, even after you've passed the threshold. Would it be better if none of this happened at all? Sure. Is this a _fuck_ of a lot better than what we're seeing Europol pushing for? Absolutely. Apple cancelling their scanning was a short term win. This isn't a new problem, and this isn't one that's going away. We can throw all the technical solutions we want at it, but it's not a technical problem. ("Sorry, can't scan user's content it's all end-to-end encrypted!". "Don't care. You wrote the encryption. Work around it.". "It's impossible!". "Okay, enjoy your new regulation that all encryption has to have a backdoor HTH HAND.") I'd rather lose the battle and win the war. Let's put the most privacy-preserving CSAM scanning we can in place and take that card out of play. Let the regulators come out and try and explain how "Well yeah, you're scanning every image for CSAM but, uh, it's not enough. We do really need to see _all_ the images people have on their phones!". We're not making an argument, we're drawing a hard line and standing in place. The Europols of the world are not going to stop pushing. Apple's big, but not "override the EU" big. When push comes to shove, we will lose. The EU _will_ respond with regulations. Maybe not now, but it should be obvious the way the winds are shifting. I'd bet my left testicle their vision for this is much more onerous than what Apple was proposing. But hey, at least we can tell our children (away from our phones or any other electronics, and probably standing somewhere deep in the woods) that we were proudly defiant to the end. [0]: https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

jacooper 3y ago

Apple announcing that stupid Csam scanning started all of this and they proved it's possible. An absolutely stupid and disastrous move.

zimpenfish 3y ago
> Apple announcing that stupid Csam scanning started all of this Except all of the cloud providers are already scanning uploaded photos for CSAM and have been for years. Trying to blame Apple for this is insane.
- jacooper 3y ago
  This isnt comparable. Apple was going to scan local photos, not cloud stored ones. No matter the source, no matter the app. And the phone will report you to the police if the algorithm marks any local photos as ones reported by the police using a "neural hash", which has a non-zero amount of hash collisions. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issues/1 https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...
  - zimpenfish 3y ago
    > Apple was going to scan local photos, not cloud stored ones. At the point of upload to the cloud service -where they would be scanned anyway-. > if the algorithm marks any local photos No, it required a threshold of N photos to match before they were submitted for human verification. > which has a non-zero amount of hash collisions Hence the threshold and human verification step.
    - ajsnigrutin 3y ago
      So, imagine your new house having cameras and microphones all over the place, that you cannot turn off, recording 24/7, but "only locally". If there's screaming, could be TV, could be just an argument, could be rough sex, drama practice, or maybe even violence and murder, it will mark those recordings and after a few repeats it'll send them to a person to look at your private recordings to see if it's just some bdsm play or if you're murdering your wife. Oh, the police officer looked at the video and it was just bdsm? Ok, continue until the next threshold.

lost_tourist 3y ago

Also that would have set the precedence for the police to use apple as a proxy to put software on your phone/computer to monitor everything you do and scan for them, while they lean back in office chairs and wait for a hit. Same would have been extended to any and all government bureaucracy. No warrant, no innocent until proven guilty; guilty until proven innocent is their credo.