3 ms·

Every step you've described is unfalsifyable: You just have to blindly trust that Apple is doing these things, and that e.g. authoritarian regemes haven't compr

by nullc 5y ago

Every step you've described is unfalsifyable: You just have to blindly trust that Apple is doing these things, and that e.g. authoritarian regemes haven't compromised Apple staff with access to the data. > They have to come from the intersection of two databases from two jurisdictions. My message directly answered that. A state actor can modify an apparent childporn image to match an arbitrarily hash and hand that image to other agencies who will dutifully include it in their database. > Then you’d have to match _nearly exact photos_ It's unclear what you mean here. It's easy to construct completely different images that share a neuralhash. Apple also has no access to the original "child porn" (in quotes because it may not be), as it would be unlawful to provide it to them. > but let’s be honest about the real risks Yes. Lets be honest: Apple has made a decision to reprogram devices owned by their customers to act against their users best interest. They assure us that they will be taking steps to mitigate harm but have used powerful cryptography to conceal their actions and most of their supposed protections are unfalsifable. You're just supposed to explicitly take the word of a party that is already admittedly acting against your best interest. Finally, at best their protections are only moderate. Almost every computer security vulnerability could be dismissed as requiring an impossible series of coincidences, at yet attacks exist.

simondotau 5y ago
> A state actor can modify an apparent childporn image to match an arbitrarily hash and hand that image to other agencies who will dutifully include it in their database. Even if a state actor constructs an image that is an NeuralHash collision for the material they wish to find, that only gets them through one of the three barriers Apple has erected between your device and the images being reported to a third party. They also need to cause 30 image matches in order to pass threshold secret sharing, and they need to pass human review of these 30 images. Arguably investigation by NCMEC represents a fourth barrier, but I'll ignore that because it's beyond Apple's control. > You just have to blindly trust that Apple This has been true of all closed source operating systems since forever. Functionally, nothing has changed. And whatever you think of the decision Apple has made, you can't argue that they tried to do it in secret.
- nullc 5y ago
  The invocation of 30 images like it's a barrier confuses me. I created a bunch of preimages posted on github, I could easily create 30 or 3000 but at this point all I'd be doing is helping apple cover up their bad hash algorithm[1]. I pointed out above that the attacker could use legal pornography images selected to make it look like child porn. This isn't hard. Doing it 30 times is no particular challenge. I didn't use pornographic images in the examples I created out of good taste, not because it would be any harder. [1] I've made a statement that I won't be posting any more preimages for now for that reason: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issues/1#issuecomment-903178370 https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...
  - simondotau 5y ago
    If someone is trying to frame a known individual, the 30 image threshold may not be a significant barrier, I'll grant you that. But if you're enlisting Apple's algorithm to perform a dragnet search of the citizenry (e.g. leaked state secrets) then this mechanism cannot be effective unless the material in question is comprised of at least 30 photographs.
    - nullc 5y ago
      I'll grant you that! I have some residual nitpicks, on that point: many leaked data troves are much larger than that, though it is a material restriction. The 30 threshold isn't leakless. Say you only have one hit, it still gets reported to Apple. The software also emits a small rate of "chaff", fake hits to help obscure the sub-threshold real hits. But it could still be used to produce a list of possible matches, including anyone with targeted material plus people who emitted fake matches, producing a list of potential targets much smaller than the whole population, for enhanced surveillance.
      - simondotau 5y ago
        This still relies upon some degree of compliance by Apple in order to acquire the stream of vouchers. Or alternatively a working security breach of Apple's systems. Either way this represents an additional, non-trivial barrier to overcome. It would be interesting to know what the "chaff" rate is and whether any intelligence agency could stomach that amount of surveillance, particularly since it's by no means certain that any of them are real. In fact it seems to me that it's very unlikely indeed, especially if the material is in any way radioactive. After all, finding a match this way requires quite a few assumptions: 1. The target owns an iPhone; 2. The target has enabled iCloud Photo Library; 3. The target has a photo library small enough, or is paying for sufficient iCloud storage space, that the flagged images are included in the (sub)set stored in the cloud; 4. The target has imported the flagged images into their photo library rather than to iCloud Drive or any third party app like SpiderOak, Mega or Tresorit.
dwaite 5y ago
> Every step you've described is unfalsifyable: You just have to blindly trust that Apple is doing these things, and that e.g. authoritarian regemes haven't compromised Apple staff with access to the data. So third party auditors as well?
- nullc 5y ago
  Better than not, but how much can you trust the word of a third party auditor in a world where government intelligence agencies conspire to spy on each other's citizens because their laws expressly prohibit on spying on their own... where the (one time) worlds largest supplier of cryptographic hardware was covertly owned by the CIA and shipped backdoored units for decades, where our national standards bodies make standard backdoored random number generators, and where the US government used a sham vaccination campaign to collect genetic samples from a whole community in order to locate and assassinate a single terrorist. When it comes to keeping data private we face adversaries whos only rules seem to be what they can get away with. Against that, full transparency when it comes to the construction of our security infrastructure really needs to be the starting position.
  - brokenmachine 5y ago
    >and where the US government used a sham vaccination campaign to collect genetic samples from a whole community in order to locate and assassinate a single terrorist. What story are you referring to here?

by nullc 5y ago

simondotau 5y ago

> A state actor can modify an apparent childporn image to match an arbitrarily hash and hand that image to other agencies who will dutifully include it in their database. Even if a state actor constructs an image that is an NeuralHash collision for the material they wish to find, that only gets them through one of the three barriers Apple has erected between your device and the images being reported to a third party. They also need to cause 30 image matches in order to pass threshold secret sharing, and they need to pass human review of these 30 images. Arguably investigation by NCMEC represents a fourth barrier, but I'll ignore that because it's beyond Apple's control. > You just have to blindly trust that Apple This has been true of all closed source operating systems since forever. Functionally, nothing has changed. And whatever you think of the decision Apple has made, you can't argue that they tried to do it in secret.

nullc 5y ago
The invocation of 30 images like it's a barrier confuses me. I created a bunch of preimages posted on github, I could easily create 30 or 3000 but at this point all I'd be doing is helping apple cover up their bad hash algorithm[1]. I pointed out above that the attacker could use legal pornography images selected to make it look like child porn. This isn't hard. Doing it 30 times is no particular challenge. I didn't use pornographic images in the examples I created out of good taste, not because it would be any harder. [1] I've made a statement that I won't be posting any more preimages for now for that reason: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issues/1#issuecomment-903178370 https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...
- simondotau 5y ago
  If someone is trying to frame a known individual, the 30 image threshold may not be a significant barrier, I'll grant you that. But if you're enlisting Apple's algorithm to perform a dragnet search of the citizenry (e.g. leaked state secrets) then this mechanism cannot be effective unless the material in question is comprised of at least 30 photographs.
  - nullc 5y ago
    I'll grant you that! I have some residual nitpicks, on that point: many leaked data troves are much larger than that, though it is a material restriction. The 30 threshold isn't leakless. Say you only have one hit, it still gets reported to Apple. The software also emits a small rate of "chaff", fake hits to help obscure the sub-threshold real hits. But it could still be used to produce a list of possible matches, including anyone with targeted material plus people who emitted fake matches, producing a list of potential targets much smaller than the whole population, for enhanced surveillance.
    - simondotau 5y ago
      This still relies upon some degree of compliance by Apple in order to acquire the stream of vouchers. Or alternatively a working security breach of Apple's systems. Either way this represents an additional, non-trivial barrier to overcome. It would be interesting to know what the "chaff" rate is and whether any intelligence agency could stomach that amount of surveillance, particularly since it's by no means certain that any of them are real. In fact it seems to me that it's very unlikely indeed, especially if the material is in any way radioactive. After all, finding a match this way requires quite a few assumptions: 1. The target owns an iPhone; 2. The target has enabled iCloud Photo Library; 3. The target has a photo library small enough, or is paying for sufficient iCloud storage space, that the flagged images are included in the (sub)set stored in the cloud; 4. The target has imported the flagged images into their photo library rather than to iCloud Drive or any third party app like SpiderOak, Mega or Tresorit.

dwaite 5y ago

> Every step you've described is unfalsifyable: You just have to blindly trust that Apple is doing these things, and that e.g. authoritarian regemes haven't compromised Apple staff with access to the data. So third party auditors as well?

nullc 5y ago
Better than not, but how much can you trust the word of a third party auditor in a world where government intelligence agencies conspire to spy on each other's citizens because their laws expressly prohibit on spying on their own... where the (one time) worlds largest supplier of cryptographic hardware was covertly owned by the CIA and shipped backdoored units for decades, where our national standards bodies make standard backdoored random number generators, and where the US government used a sham vaccination campaign to collect genetic samples from a whole community in order to locate and assassinate a single terrorist. When it comes to keeping data private we face adversaries whos only rules seem to be what they can get away with. Against that, full transparency when it comes to the construction of our security infrastructure really needs to be the starting position.
- brokenmachine 5y ago
  >and where the US government used a sham vaccination campaign to collect genetic samples from a whole community in order to locate and assassinate a single terrorist. What story are you referring to here?