13 ms·

>There was no confusion at all. I don't know what you and tharne are talking about here. There was definitely confusion. HN is a tech forum and I still saw p

• hypothesis 5y ago
  I’m sorry, at what point there was any confusion that Apple is going to use our phones to do scanning?
  • slg 5y ago
    That is the where of the story. There was a inarguably confusion over the when, what, and how of the story. It was not universally understood that this would only apply to photos sent to iCloud. It was not universally understood that this was only looking for previously known CSAM. It was not universally understood that they were using some sort of hash matching so photos you took yourself would not trigger the system. I understand if you consider the where more important than the others, but it is simply a fact that there was confusion on what exactly was happening here.
    • lstodd 5y ago
      > some sort of hash matching so photos you took yourself would not trigger the system. This is ignorance in extreme.
    • hypothesis 5y ago
      I appreciate your more detailed clarification and agree with ‘where’ conclusion. To the extent that other parts of this story was explained to us by Apple, I did try to clarify some exaggeration in other thread.
    • asddubs 5y ago
      There's always going to be plenty of people commenting who didn't even bother to read the article at all. But by and large, from what I saw, people did understand the nuances of this and outlined how little stands in the way of expanding these policies' scope once the technology is in place
    • cwkoss 5y ago
      I think "universally understood" is doing a lot of work to portray a much higher degree of certainty about each of those statements than is justified. A lot of the contention wasn't about the specifics of their plan, but rather how subtle changes could vastly expand the scope of their plan. "this would only apply to photos sent to iCloud." for now, until scope creeps. "this was only looking for previously known CSAM." for now, until scope creeps. "using some sort of hash matching so photos you took yourself would not trigger the system." well this one is immediately concerning even within claimed scope because there ARE going to be false positives that apple records some database. Millions of iphone users are going to have a non-zero "possible childporn" score. They are building an engine for iphone users to self-incriminate. If they rigidly hold the scope to only what they announced and never expand, it could be argued that this is a reasonable concession to fight CSAM. However, in making the announcement, they boldly stepped past their existing hard line in privacy (local device content is private and not surveilled by apple), so it seems naive to expect that this announcement reflects the eventual scope of this self-incrimination engine for the next decade of apple updates.
      • slg 5y ago
        >A lot of the contention wasn't about the specifics of their plan, but rather how subtle changes could vastly expand the scope of their plan. The how helps show us how changing this system is not a subtle change. It isn't like they can flip a switch and suddenly they are identifying new suspected CSAM on people's phones. That would require a new system since the current one is only hash matching. >However, in making the announcement, they boldly stepped past their existing hard line in privacy (local device content is private and not surveilled by apple), so it seems naive to expect that this announcement reflects the eventual scope of this self-incrimination engine for the next decade of apple updates. This is an arbitrary line that is being drawn. These are photos that are marked for sending to iCloud. Whether the scanning happens on the phone before they are sent or in the cloud after they sent is largely immaterial when it comes to the impact of the code. People are acting as if the line Apple drew was motivated by technology. That was never the deciding factor. Technology is the easy part here. That line was only a policy line and that policy has not changed. Only photos that are sent to iCloud are scanned. If you fear Apple changing that policy going forward, you should have always feared Apple changing that policy.
    • insulanus 5y ago
      You forgot the who - Whose fault is it that those points were not clearly communicated? - Who wrote the perceptual hash matching code? - Who is allowed to audit the code, the review system, and the hash database? - Who updates this code? - Who decides if your phone OS is updated? - Who decides the iCloud upload defaults? - Who decides if you are reported? - Who asked for this feature?
      • romwell 5y ago
        - Who is going to have their dog killed by a misguided SWAT team that doesn't bother analyzing the automated report before acting on it?
    • romwell 5y ago
      It was not universally "understood" because it was not universally agreed upon. To wit: * It was not universally understood that this would only apply to photos sent to iCloud. Since the scanning doesn't happen on iCloud, this distinction is irrelevant. "We are going to intrusively scan the subset of your photos that you care enough to back up to the cloud that we've been pushing to you for years" is pretty clear. * It was not universally understood that this was only looking for previously known CSAM. It was only looking for whatever is in an opaque database which, according to a third party we don't have any contract with, contains CSAM. * It was not universally understood that they were using some sort of hash matching so photos you took yourself would not trigger the system. Yeah right, I feel totally safe knowing that I won't be falsely reported to FBI by a "some sort of" hash matching. Here's a hash function: f(x) = 0 for all x It's "some sort of" hash, too.
      • fingerlocks 5y ago
        > Since the scanning doesn't happen on iCloud, this distinction is irrelevant. It’s really not irrelevant. A third party photo library that avoids using the PhotoKit library would not be touched by the CSAM detector. There are many of these on the App Store. One step further and store the photos encrypted, with a custom renderer that decrypts the content on the heap, and that would take some tremendous performance-hitting detection abilities it’s extremely unlikely to ever happen.
        • romwell 5y ago
          "Look, there's a hack that allows you to avoid this at the cost of convenience that you bought the device for!" OK.
          • fingerlocks 5y ago
            “Don’t support this company and it’s entire product line because this one app is scanning for kiddie porn”. Misrepresenting each other’s arguments is not how you have a discussion.
    • tomp 5y ago
      Your problem is trusting evil people at face value.
  • notriddle 5y ago
    Nobody's confused about that. > HN is a tech forum and I still saw plenty of people here worried about how they would get in trouble for having innocent photos of their own children on their phone. They're confused about this. NeuralHash doesn't look for pictures of naked kids. It looks for pictures that are identical to the ones they've put in their signatures list. The problem is that Apple claims that the signatures in their list are all pictures of sexually-abused kids, but we have no way of verifying that. Heck, they don't even have any way of verifying that. Everyone just has to take NCMEC's word for it.
    • lazide 5y ago
      You’re also wrong no? The perceptual hashing doesn’t match EXACTLY the same photos, it intentionally matches photos similar to the same photos, so a minor rotate by a few degrees or crop or whatever also matches
    • cwkoss 5y ago
      Perceptual hashing can have collisions, and they will be at a higher rate than completely "identical". The public does not know what the false positive rate is for 'average iphone user pictures'. As engineers we can be certain the false positive rate is not zero. This means that some number of iphone users are going to have non zero "possible child pornographer" scores in the apple database. The false positive rate is crucial to understanding how concerning this should be. If the average iphone user has 1000 photos, and the false positive rate is the claimed 1 in 1 trillion, there is a 1 in a billion chance that you'll be flagged as a potential child pornographer. (~1 in the world will be falsely accused). This seems reasonable enough with the apple-internal screening step. If the chunking and perceptual hashing functionally ends up having a much higher false positive rate for images which have similarities to the dataset (parents' pictures of kids playing shirtless, legal adult porn, etc), the false positive rate could actually be more like 1 in 1 million or worse. In which case there are potentially hundreds of thousands of people who will be falsely accused by this system. How many matches will US judges require before they sign warrants for arrests, search and seizure of digital devices? If they are technically competent it shouldn't only be 1, but I don't trust all judges to understand probability well enough to require multiple matches.
  • benhurmarcel 5y ago
    Why does it matter that much which CPU runs the check?
• mixmastamyk 5y ago
  True, but a small point of contention compared to the introduction of on-device scanning for illegal activities. The policies of which could be changed at a moment's notice. Other details are relatively unimportant.
  • sharken 5y ago
    This is in essence the problem with the new on-device initiative from Apple (calling it a tool is rather misleading). If allowed to go forward, it is only a matter of time before the capability is expanded. So it's a big no to the scanning capability, you would think that Apple had gotten the message by now. And the other initiative is also open for abuse, by allowing the device administrator to spy on the user. Admittedly not as bad as the on-device scanning.
• smnrchrds 5y ago
  I don't think that was confusion either, because there were discussion and articles on how a hash collision is possible in the scenario you mention due to the way perceptual hashes work.
  • slg 5y ago
    They have said that the system won't be triggered on a single image. You would need to have multiple photos on your phone experience this hash collision which drops the odds of false positives considerably. EDIT: It has now come out that you need to trigger the system 30 times before Apple acts on it. I can't imagine the odds for someone to have 30 hash collisions.
    • smnrchrds 5y ago
      So they understand that their system is very susceptible to false positives, but they are saying their clients shouldn't worry because the black box hash gets compared with a black box inauditable threshold, both of which solely determined by Apple. I don't think the reaction was due to any confusion. People understood what Apple was trying to do and realized how much it sucked from a technical perspective.
      • slg 5y ago
        Where did "very susceptible to false positives" come from? If the odds of a collision are one in a million that is troublesome if they only need one match. If they ignore anyone that has less than 3 matches, we don't really have to worry about false positives. People who have CSAM generally don't have only 1 or 2 images.
        • JCharante 5y ago
          So why 30 matches before they do something? Why not 100? Why not 5?
          • smnrchrds 5y ago
• indymike 5y ago
  There was zero confusion. Apple's new feature was universally rejected and if it cone to market will cause a severe loss of market share.
  • megablast 5y ago
    I doubt that it was universal. Nothing ever is. But I haven't seen any positive discussions about it, which is odd.
    • geoduck14 5y ago
      Case in point, I'm pretty apathetic about the whole thing
  • defaultname 5y ago
    Severe loss of marketshare? It wouldn't even register. I don't like the feature. Putting this on the client device is dubious and should never have made it past the brainstorming stage. Having said that, technology companies, big and small, are bound in the US to do this. By law. If anything Apple was by far the laggard of the bunch (with reporting counts magnitudes lower than peers, despite a larger customer base). As I said in another comment, no company can protect you from your government. Much has been made about it being on device, which while a serious optics issue...the hot takes being given on here are manifestly absurd. Like, literally the company that holds all of your data, all of your passwords, all of your info and you need to invent slippery slopes to imagine up what they "might" do? If they want to have their way with your data, they could have been doing it for decades. They should never have announced two very different systems at the same time. Contrary to some of the insincere claims given in this very thread, there is massive disinformation and confusion about them. In the end I feel like 98% of the "the end is nigh!" comments are by long time Apple detractors who just see this glorious opening. And while I still hope that Apple says "Mea culpa, we're just going to scan on the ingress to iCloud Photos", whatever they do in a month this is going to be completely forgotten.
    • cm2012 5y ago
• notJim 5y ago
  There is a good amount of confusion throughout this very thread about what this is.
• devwastaken 5y ago
  Not confusion - we will eventually remove humans from CSAM checking and it will all be AI's. The "slippery slope" is already halfway down the hill.
  • monocasa 5y ago
    Or even without calling it 'AI', perceptual hashing like you typically see in applications like CSAM is pretty damn close to ML techniques. The normal thought process is "we don't want child predators to sneak through just by cropping, or sticking a water mark on the image, or some other way slightly modifying the image like the color balance. Can we come up with something that'll hash the same even with minor modifications to the image?". And you basically end up with intentionally overfit AI.
  • TheDudeMan 5y ago
    You could argue that the slippery slope started when we invented the transistor :)
• roenxi 5y ago
  It is a very tenuous theory that this will only end at CSAM with alleged trillion to 1 odds. It is unclear how they justify stopping there - all laws are roughly equal in the eyes of government, why shouldn't an iPhone enforce them all? This incident shows that Apple is comfortable with the idea of using my phone to get me imprisoned: 1. I don't want to financially support anyone going through my private things in conjunction with what is basically the police, looking for reasons to imprison me. Reasonable suspicion first, thankyou very much. 2. I don't trust them to use this in a politically neutral way. This is too much power. 3. I don't trust them to manage the false positive rate. Everyone knows how reliable software engineers are when they claim a system does something. At heart they aren't 1 in a trillion people. This is a great time to be outraged. I can't really do much about what they do on their servers, but I can certainly get antsy about what happens on my phone.
  • deleted 5y ago
    [deleted]
  • omikun 5y ago
    It's not like this is some new fancy technology no one could do before. There has never been any technical barriers to scanning your phone for insert-reason as part of the OS. The slippery slope argument could be made from the first smart phone. The government could always pressure any company to put a backdoor in any OS. Apple hasn't done this (to our knowledge) so far and this new CSAM scanning feature doesn't remove any non-existing barriers to that end. They just gave a talk at the USENIX Security Symposium on how this works and the safeguards put in place. The scanning functionality is part of the icloud photo upload module. Security researchers can determine if they change the scanning algorithm in future updates. I don't know what else you can hope for. Going back to the slipper slope argument, you could say nothing stops them (or Google on Android) from uploading your passcode and share it with X. It's all just software in the end that they write that powers the lock screen and security checks.
    • roenxi 5y ago
      > Going back to the slipper slope argument, you could say nothing stops them (or Google on Android) from uploading your passcode and share it with X. It's all just software in the end that they write that powers the lock screen and security checks. If they start claiming they do that then I'll get angry about that too. You'll notice that only the fringe is accusing Apple of being liars or acting in bad faith here. Apple are pretty up front about what they do. The problem with this plan is they're claiming they are only going to do selective law enforcement (only 1 US law, only things that are strongly supported by consensus and even then only sometimes). That isn't a position with a reasonable foundation, they are going to change their mind. I want them to change it in the "we don't snoop on people's phones" direction rather than open season.
• Zak 5y ago
  There's plenty of confusion, but what's telling in this story is that the people who are the least confused are among the most opposed to Apple's plans.
• PixyMisa 5y ago
  They are absolutely correct to be worried.
• colordrops 5y ago
  They are talking about how there is no confusion about the overreach in scanning on people's devices. You are arguing a straw man.