4 ms·

I think most people don't upload to facebook pictures of their kids taking a bath? But they more than likely store such pictures on their phones/laptops.

• simondotau 5y ago
  Sure, but none of those images will be a hash match to any material in NCMEC databases.
  • hsn915 5y ago
    What if they decide next year to use AI to automatically detect potential violating images? I hope you won't throw the "slippery slope is a fallacy" fallacy at me.
    • simondotau 5y ago
      Slippery slope isn't always a fallacy, but it is the way most people use it. Most people just define the slope however they want to suit their argument. Let me Does this change to the software code represent a slippery slope of motive or opportunity? Many here have said yes, but in my opinion, no. When software can update itself, every single update is an opportunity for the software to betray you. That risk is already high; the risk profile doesn't increase because a particular change feels slippery slopey to you. As soon as any closed-source software implements automatic software updates, you've always one malicious update away from the system betraying you. Interim steps are unnecessary. Whether it's Chrome, or Firefox, or Windows, or Android. Heck, even Ubuntu. Any of them could betray you at any time. Potentially trash their reputation in the process, but that's a mere technicality. Therefore the slippery slope is the wrong metaphor. The correct metaphor is trust. Does this change lower my trust in Apple? Me personally, no. If anything, Apple's transparency has increased my trust in them. It gives me confidence that Apple won't use the fear of bad PR as an excuse to conceal serious things like this.
      • hsn915 5y ago
        The reason that "slippery slope" is almost never a fallacy is that people with intentions to perform actions that are outside the overton window[0] of acceptable behavior will first do a thing that lies just on the edge of the overton window. It is currently considered completely unacceptable for companies to scan the data on the user's own disk. If someone wanted to start doing that, they would have to create a plausible and convincing excuse. Once people accept that excuse and time passes, they can slowly "expand" the territory of this excuse (push the overton window further). "Protecting the children" is exactly this kind of plausible and convincing excuse. It needs not be the case that Apple specifically wants to scan user data so they came up with this excuse. It's simply that, once scanning users data "for the good of humanity" becomes acceptable, _some_ malicious actor will push the overton window further and further. Imagine in 10 years from now, all operating systems will scan users data to detect potential child porn material. It might even become required by law, or just by "social pressure". Just like it is now almost required by "social pressure" that social media platforms censor discourse and information. It's then very easy to expand this capability to also detect illegally downloaded music, or adult videos, or whatever deemed unacceptable. [0]: https://en.wikipedia.org/wiki/Overton_window https://en.wikipedia.org/wiki/Overton_window
        • simondotau 5y ago
          https://twitter.com/simondotau/status/1415564718375272455 https://twitter.com/simondotau/status/1415564718375272455
        • simondotau 5y ago
          I don't agree that what Apple is doing is sufficiently distinct from what Google has already been doing to represent an Overton window shift. To most people, they will see that the corpus of images being targeted is functionally identical to what Google has been doing for Android users for many years and therefore the difference is mere implementation detail. A distinction without a difference. Nothing more. Highly technical, libertarian-minded people see a big distinction between on-device and off-device scanning. But such people have always eschewed the "managed experience" model of Apple devices.
  • heavyset_go 5y ago
    It's entirely possible for one innocuous picture to look like an illegal picture in that database, and if they do look similar, they'll have similar perceptual hashes. That's the point of perceptual hashing.
    • tpush 5y ago
      'Similar' hashes wouldn't do anything; the system hashes perceptually similar images (cropped, resized etc.) to the same hash.
      • heavyset_go 5y ago
        Not all transformations result in the same hash, hence why fuzzy matching is necessary. Some perceptual hashing techniques try to incorporate common transformations like rotations and scaling, but if you change an image slightly, but enough, hashes will be different but still similar.
        • tpush 5y ago
          Apple's system only concerns exact hash matches, and is built so transformed images hash to the same number.
          • heavyset_go 5y ago
            > and is built so transformed images hash to the same number. That is the goal of all perceptual hashing algorithms. > Apple's system only concerns exact hash matches Then it is almost useless. All someone would need to do to evade the system is make minor enough adjustments to illegal images so that the distance between resulting hash and the original is minor. In reality, perceptual hashing systems use fuzzy matching on hashes by using a metric like hamming distance to calculate differences between two or more hashes.
            • tpush 5y ago
              > Then it is almost useless. I'm sure you're in a prime position to judge that. > In reality, perceptual hashing systems use fuzzy matching on hashes by using a metric like hamming distance to calculate differences between two or more hashes. Again, Apple's NeuralHash purposefully doesn't work like that [0]. [0] https://twitter.com/jonathanmayer/status/1423370142411476993 https://twitter.com/jonathanmayer/status/1423370142411476993