9 ms·

A Screeching Voice of the Minority

123 points by guiambros 5y ago

nullc 5y ago
> That’s so that criminals can’t just disguise the image with a trivial change. It's utterly trivial to modify the image to avoid matching. Most minor modifications will do so. The purpose of the hash is so that idiotic criminals that aren't trying to avoid getting caught don't accidentally avoid getting caught just by resizing or recompressing the images. The flawed 'hashing' function also creates plausible denyability: If its somehow proved, perhaps via a whistleblower, that Apple was in fact matching the historic tank man photo, Apple or the government could easily hand over a piece of child porn with the same neuralhash due to the extraordinary vulnerability of the function.
- raxxorrax 5y ago
  Don't know why you are downvoted because I think this is still true for the most part that you should be able to trivially create another hash for the same image with unintrusive operations. You might need to put some efforts in it and reverse engineer the hashing algorithm a bit and see what causes rolling a new hash value. A lot of research went into classifying images, mostly from the side that wanted visible or invisible watermarks in pictures. It is a cold war against image cropping, compression and modification that tends to not favor the classification side. But here you only want to have a different hash, the operations for that are numerous and probably indeed often trivial. I don't think you could just have a completely different picture create a collision though. Maybe they can because it is their hash-function, but that would be quite unusual and I doubt this is the case.
  - nullc 5y ago
    > I don't think you could just have a completely different picture create a collision though. Allow me to introduce you to my posts on github: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issues/1#issuecomment-903094036 https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue... Where I post good looking examples of standard test images altered fairly subtly to give the specific hashes. The apple neuralhash is broken as a 'hash function'. It's much much easier to modify images to just have a different hash. A simple blemish on the image-- or, with whiteboxing using the hash function, no visually noticeable change is required at all.
    - raxxorrax 5y ago
      That is seriously the hash function they used? I assumed a competent hash function, hell, any common hash function is better even if they fail at resize/compression. This one would too.
      - nullc 5y ago
        Apple 'neuralhash' usually survives resize (the first step it does is resize the image to 360x360) or minor compression lossy changes. But the price of that is that it's not secure against generating arbitrary preimages based on fairly arbitrarily targets, with results that look nothing like the original image you're matching (if one even exists). They could do some downsample, transform, quantize, sha256 approach that would still survive some tiny amount of differences. ... but apple favors easily constructed false positives over false negatives, even though its trivial for anyone trying to avoid their function to get a false negative. Their scheme only protects against accidental false negatives. It's just a symptom of systematically favoring factors other than the privacy and security of the actual owner and user of the computer.
        raxxorrax 5y ago
        Wow, that is seriously a naive approach. Honestly, I would have expected much more here. That makes the libraries of hashes for content almost useless. Hope they extracted features that could be used in a more sophisticated approach.
exporectomy 5y ago
"If you’re right wing, look at how privately taken video and photos have been used to track down the January 6th Capitol rioters" How is that supposed to appeal to right wingers? He thinks all right wingers want those rioters to escape justice? Most republicans oppose them according to several polls. The author may be a victim of political divisiveness due to paying too much attention to social media and news.
- redis_mlc 5y ago
  All Americans want the right to protest. In fact, Democrat protesters occupied the Capitol building in 2018. I'm not aware of a manhunt for those protesters. > Most republicans oppose them according to several polls. I suggest the poll be re-run after you tell people that the MSM narrative was a lie - protesters killed nobody, and police opened the doors. Nobody arrested had a gun. AOC has told 3 different stories, yet she was a mile away at the time. In fact, there was no "insurrection", confirmed by the FBI last week. But there was a Marxist-style narrative - they're very clever with wording, aren't they? Why didn't Pelosi add more security for 100,000 protesters? That's the only fact we really need to know, isn't it?
- roenxi 5y ago
  It does show an ignorance of how the right wing thinks. However, it is directionally accurate - being an effective right wing leader is likely to get someone's phone targeted. Not a partisan thing; effective left wing leaders also have their phones targeted. The swamp doesn't like change.
crummy 5y ago
I thought with enough matches Apple would decrypt your images, at which point they would be able to verify if they were CSAM (and notify authorities) or realise they weren't and that a government had tried to trick them (and do nothing or go public with this information). Is this not how it works?
- Jtsummers 5y ago
  That is how they've described the system. After 30 or so matches, they'll be able to decrypt a version (presumably scaled down, the description is vague) of the images and manually review them. With the present system targeting CSAM imagery, unless the images you have match their expectation of matching images, they wouldn't pass it on to law enforcement. This is where the slippery slope argument comes in. Apple would have to actively cooperate with governments to both detect and report other kinds of images. So if Tiananmen Square images were showing up in the review pile, Apple would have to know to pass those on to the Chinese government and be willing to. The only other way they'd be passed on is if Apple drops the manual review portion.
  - lazide 5y ago
    Not quite. Any manual review system is going to have infrastructure to pick reviewers, route the data around, etc. it would not be hard, and could be done under the radar, to intercept data for specific targets in a way that wouldn’t arouse suspicion, and it would be awfully hard to prove it had happened too.
    - Jtsummers 5y ago
      So you're agreeing with me. Apple would have to actively cooperate by providing reviewers who would go along with the kind of media besides CP that nations may want to detect, or Apple would have to actively cooperate by simply passing matches on and bypassing the review process. But Apple would have to be the ones doing it either way.
      - lazide 5y ago
KSteffensen 5y ago
"How much attention do you think police gave each of those 20 million reports from Facebook?" Killer argument. Why not focus here instead?
watt 5y ago
You know what I hate? Titles that say specific words that are never repeated in the article. A title that has so little to do with the contents of article, that it might as well be a random reference to some throw-away line in some comment somebody uttered sometime, which might as well be left alone so a proper title could be selected.
- BLKNSLVR 5y ago
  The "screeching voice of the minority" is that which Apple is catering to in it's image scanning, and since the image scanning is what the article is about, then I don't have a problem with it. Normally I would agree, but I'm fine with the subtle(?) implication of this one.
  - Jtsummers 5y ago
    That's not what the title is about. The title is establishing the author and the article as part of the "screeching voice of the minority", a reference to a press release that dismissed concerns about the system as, essentially, being unimportant and which used that phrase to dismiss, out of hand, opponents of the scanning system.
    - BLKNSLVR 5y ago
      Hah, you're right, thanks. Didn't know about the memo until reading other comments.
- edflsafoiewq 5y ago
  It's a reference to > We know that the days to come will be filled with the screeching voices of the minority. https://9to5mac.com/2021/08/06/apple-internal-memo-icloud-photo-scanning-concerns/ https://9to5mac.com/2021/08/06/apple-internal-memo-icloud-ph... from a letter from NCMEC that Apple circulated in an internal memo.
  - watt 5y ago
    It is, but why not address it in the article? The article does not have a word "minority" or "screeching" in the text.
thatsabadidea 5y ago
100% this will be used to track dissidents with non-official-narrative memes. For your own good.
justbored123 5y ago
The hysteria about this is extremely childish. After the Pegasus news that showed that a relatively small private company in a very small country could have complete remote access to your phone and sell it on the market, do you really think that a powerful government like China or the US can't already do it??? How could you be this naive? Especially after the Snowden revelations. And about Apple's having a change of hearth in the future and using something like this against you. Have you stopped to think for one second and consider that they have complete control over the updates that your phone receives?? They can quietly do what ever they want without you ever knowing and you can't do sh*t about it. So drop the concerned citizen act. We all abdicate privacy a long time ago when we sold our souls for cool free services like google and sexy gadgets. At the end of the day the "slippery slope" argument boils down to one thing: do you trust your institutions? Any law or service can be abused, and the only thing preventing that is the integrity of the people on your society and the institutions both public and private that they work for. The exact same law that is used to fight corruption in a great country like Norway or Denmark can be abused to persecute political opponents in a shitty country like North Korea or China. If the answer is no, I don't trust my institutions and my people, then you have a far more severe and fundamental problem than an specific law or service. In that scenario, focusing in an specific issue is like trying to cover the sun with your hands.
- noptd 5y ago
  And? These whataboutisms are a distraction and in no way support the idea that people shouldn't be worried about Apple implementing a new client-side system that's ripe for abuse. If things are already bad, we should be working towards making them better instead of rolling over and apathetically allowing them to get worse.
- Jensson 5y ago
  If we don't speak up when they push the boundaries they will just continue pushing. The only thing stopping Apple from doing all of those bad things today is that they are afraid of bad PR. You should be happy that others speak up now rather than you waiting until what they do becomes bad enough that you feel the need to speak up against it.
rossdavidh 5y ago
Good summary of the situation. I am, unfortunately, pretty much completely in agreement on all points.
da_chicken 5y ago
As usual, they trot out "think of the children!" whenever they want to start doing something horrifically invasive or trivial to abuse. Just what we want: social standing determined by YouTube algorithm.
darepublic 5y ago
I feel like it's a matter of time before chrome adopts similar bullshit. Just in time censorship
- r721 5y ago
  Chrome for Windows already scans local files for "unwanted software" (using ESET’s technology), other kinds of hashes could be easily added there: >As applied in Chrome Cleanup, ESET’s technology is used by Google to alert users about unwanted or potentially harmful software attempting to get on users’ devices through stealth, for example, by being bundled into the download of legitimate software or content. Google Chrome, using ESET’s security technology, then provides users with the option to remove the unwanted software. Chrome Cleanup operates in the background, without visibility or interruptions to the user. It deletes the unwanted software and notifies the user once the cleanup has been successfully completed. https://www.eset.com/int/about/newsroom/press-releases/company/google-chrome-and-eset-collaborate-in-fight-against-online-threats-3/ https://www.eset.com/int/about/newsroom/press-releases/compa...
- runjake 5y ago
  Chrome/ChromeOS/Android already do the same thing with cloud synced photos but it happens on the server side. Google has been scanning for years now. Same with Microsoft. And Dropbox. And…
  - mastazi 5y ago
    If it's private and/or sensitive, it probably shouldn't be stored with a cloud-synced service. Recently I've seen a lot of "experts" trying to reassure the non-technically oriented about the fact that it's not true that "the cloud is not someone else's computer", that that's just a meme and it's not accurate. But they never thoroughly explain why that wouldn't be the case. When you dig deeper, it turns out that more often than not those "experts" are people whose job is to sell cloud services, or who work for companies that sell cloud services.
  - judge2020 5y ago
    "Chrome/ChromeOS/Android" doesn't work when Android is a completely separate piece of software from the other two, and only Android/ChromeOS have any sort of file syncing to Google servers thus meaning no PhotoDNA scans if you only use the Chrome browser.
robertoandred 5y ago
Apple does not report you to law enforcement, as this article claims. The article also ignores human review. The article compares scanning to facial/object/scene recognition, which it can’t do. The article talks about making it easier to find who took a photo, which is not the problem the Jan 6 sleuths were trying to solve. The article worries about China, but ignores that matchable photos must appear in multiple jurisdictions’ databases. The article also ignores the several points of third-party auditing and several layers of human reviews.
- jcims 5y ago
  What do you think the public reception would’ve been if this technology had been available last year, and multiple jurisdictions published hashes of images from the riots at the capitol? Or maybe, more directly, what do you think Apple employees reviewing these hits would do if those types of images started showing up in the triage list.
  - robertoandred 5y ago
    What exactly do you think publishing hashes of Jan 6 pics would achieve? The photographers posted them publicly on Facebook and Twitter. That’s how we have them in the first place. The problem wasn’t knowing who took the photos, it was knowing who was in the photos. And that needs facial detection, which this system can’t do.
    - jcims 5y ago
      It would achieve exactly what the system is designed to do, mine personal devices for photos that authorities are aware of. Just because a lot of people were sloppy doesn’t mean every person was. However, my primary point is that there is no possibility of a technical control over the scope of the program. It’s a policy decision and is subject to explicit expansion or collusion/parallel construction.
      - robertoandred 5y ago
a3n 5y ago
It might be a good idea not to travel outside your home country with an iPhone or other Apple tech, because you don't necessarily know what kinds of photos or other digital files a country's government would object to. The picture of the man against the tanks is a good example. Indeed, you don't necessarily know what photos are on your phone.
- tjpnz 5y ago
  The less gadgets you bring across the border the better. In New Zealand for instance a customs official can now force you to unlock a device on your person for "suspicions" no matter how vague. The only way around it is to refuse to play and leave everything at home.
  - raxxorrax 5y ago
    Which is a completely archaic policy. As if there wasn't a worldwide network that makes information exchange beyond borders quite trivial. That said, leaving your devices at home is a sensible idea.
- secondaryacct 5y ago
  I wonder if someone cheeky could "swat" you by whatsapping a collection of known triggering images !
  - wizzwizz4 5y ago
    Swatting is not “cheeky”.
sigmaprimus 5y ago
A few years ago a little girl went missing and unfortunately was later found dead in a nearby lake. It was determined that she had been abducted and killed. The local news station reported that there were approximately 15000 pedophiles in the surrounding region. I found it incredible that there were so many deviants known by the police in my community, they eventually caught the killer. He turned out to be a neighbor and confessed. I think the police are well aware of every download/upload of child porn or are atleast they are able to look it up when the need arises. The time for concern of protecting ones privacy on any digital connected device has come and gone, best to assume anything You do on your device is readily available for the authorities(and others) to exploit. *it was 21 years ago, and the only news report I found says 100 known but I am positive they said 15K.* https://www.cbc.ca/news/canada/missing-girl-s-body-found-in-b-c-lake-1.232583 https://www.cbc.ca/news/canada/missing-girl-s-body-found-in-...
- judge2020 5y ago
  What's the chance that the news station said "out of 15000 potential perpetrators in the surrounding area, 100 are registered sex offenders"?
- michaelmrose 5y ago
  It is completely trivial to render your communications opaque to casual surveillance. If someone were truly committed to surveiling you it would be drastically harder. At the point where people are hacking your devices and watching you from across the street impossible. About 1/2 of 1% (estimates vary) of people are sexually attracted to children and an unknown number commit acts against children. Statistically if your kid gets abused it will probably be by someone you know and shouldn't have trusted. You should probably have a healthy mistrust regarding men alone with your kid and the government without being paranoid.
  - sigmaprimus 5y ago
    >>>It is completely trivial to render your communications opaque to casual surveillance... Im not sure I understand what You mean by this, my thought or possible concern is that all web traffic, especially that which goes through known proxy addresses is being logged and flagged by local carriers and ISPs. Then in the event of a crime being commited in my area, my actions online including stuff from decades past will invite a knock on my door by the authorities or worse the media. I must stress, Im not worried about anything sexual but have done a great deal of research on some other subjects that are not exactly above board. I can appreciate the concern regarding apple scanning personal files and reporting matching files to the government, but feel the odds of a virus, vulnerability or exploit being used by a criminal ngo is just as big if not bigger risk and has been for a long time now. Anyways, like I said...for a while now I have been of the mindset that anything I do on a connected device (even temporarily connected) leaves me exposed, now or in the future. Maybe cancel culture has encouraged my current way of thinking.
nbzso 5y ago
I moved on. Pop!_OS (no Ubuntu telemetry and OpenSnitch) over MacBook Air is working flawlessly. Company MacBooks are with Arch/Manjaro. Will run Catalina for specific tasks without internet until it breaks completely. The hostile design decisions of Apple are unbelievable. After 20 days of public backslash Apple is preparing for massive iPhone 13 campaign and pushing this obviously flawed intrusion with "common good" goals, normalizing surveillance on a new level. And all of this after Pegasus/NSO. But actually I am glad. No more shiny toys and marketing illusions for me. Expect everybody to follow suit. Year or two down the road this type of scanning and control will be mandatory. So the only way to respond is to invest in your privacy. Not in user-hostile, dark pattern designed products.
- judge2020 5y ago
  > Expect everybody to follow suit. Year or two down the road this type of scanning and control will be mandatory. Indeed, this article (despite many incorrect facts about the process) is right in that this is just a vocal minority 'screeching' to get the message out there to anyone willing to listen. The absolute majority of the population, or at least adult population, will always choose the path of least resistance - they just wants to live, work, avoid hardship, and partake in dopamine-inducing activity every day. There's a reason Google and Amazon are trillion-dollar enterprises and Hulu still has an ad-supported version that 70% of people choose[0] - people don't value privacy at all when dealing with these faceless companies. Society has always shifted the level of privacy they accept (think taxes, audits, search & seizure with probably cause), and this will just be another shift that people accept so that they continue on with their lives while not sacrificing any of their valuable time. 0: https://variety.com/2019/digital/news/hulu-ad-supported-subscribers-70-percent-1203227954/ https://variety.com/2019/digital/news/hulu-ad-supported-subs...
  - geeB 5y ago
    The hulu number was quite surprising to me, given that the ad supported tier is $6/mo vs $12/mo for ad free, and if someone watches enough the extra $6/mo would seem worth it. It might have a lot to do with their $1.99 for one year promotion they do on Black Fridays though, which would change things considerably ($2 vs $144 per year), especially for households without enough disposable income. Easy enough to switch to a different person every year and keep it going for a while, and also at the time of that article it had been less than 3 years since the free tier with ads was removed. I don't think privacy has much to do with it though. People that don't have any idea about these topics just wouldn't care, and at least in my case I care but I'm under no illusion that paying for ad-free changes anything other than the convenience of not having ads. iirc hulu's ads cannot be skipped anyway, so they pretty much can collect the same exact information from both tiers.
bigcorp-slave 5y ago
The core problem here is reporting to prevent CSAM (hopefully, it is uncontroversial that doing so is desirable) but rather that this makes on-device snitching on any topic a matter of -policy- rather than -technology-. It is much harder for a government to say, you must implement this large and complex feature, you must keep it secret, etc - especially a foreign government - than it is to say, here are some new hashes, please plug them in to your database that four people have access to. Or to say, please scan all images instead of just those uploaded. Even if Apple wants to right by its users, it is making it much easier for Apple to be compelled to do wrong. “We promise” is easily forgotten. “We can’t” requires dev teams to change.
- 3grdlurker 5y ago
  I agree about how the technology can be applied to other instances, but I feel that people who are criticizing Apple are crucially placing all the blame on Apple but not on the government that pressured them into doing this in the first place. This CSAM issue is proof that if even the most valuable company in the world can be pressured into compliance, then any company in the world could be, calling into question whether it is realistic in the first place to expect companies to act ethically while also creating highly profitable products. For all the things that have been said about the issue, I don’t remember a single comment that has been directed at the US government, or China itself, or any discussions about how nation-states should be run and genuine introspections on what could prevent surveillance states from existing in the first place.
  - judge2020 5y ago
    This is because people, for some reason, see Apple's 3-trillion-dollar valuation as being some purely self-made fortune to the point that Apple could rival these governments with their technology stance, when in reality Apple's success has only been enabled by the governments that house and profit from Apple's dominance - and now they're coming for their share of surveillance instead of dollars.
systemvoltage 5y ago
America is kneeling down to the same playbook that authoritarian regimes use, just a more vanilla version of it. Ready to be exploited one day. I am happy that the tech community on HN and elsewhere are speaking up about this. I was expecting it to be just another "think of the children" theatre ready to be swept under the rug. Major newspapers are talking about it. If anything else, next time they say the word "Privacy" at the Apple WWDC, we should boo that non-sense and let them pay the price in PR-damage. Apple spent last 3-4 years building their image as privacy-pro alternative to Google/FB/Microsoft in various product/service categories. They compltely fucked up. Privacy is about trust and it takes a single event to forever destroy it.
- audiodude 5y ago
  Perhaps they spent the time building capital as privacy respecting so they could "spend" it on a moment like this?
tamaharbor 5y ago
Alexa has been listening in on all your sexual adventures at home anyway, so what difference does this make?
- judge2020 5y ago
  If you have any proof that'd be nice. Just because there's a microphone in the room, such as via a flip phone or even rotary phone, doesn't mean it's actively recording audio to be shipped off for later blackmail/extortion/embarrassment.
  - perryizgr8 5y ago
    Echo devices' microphones are turned on all the time, unlike a flip/rotary phone, whose mics are physically disconnected when the handset is in the cradle.
    - judge2020 5y ago
      And? Just like Alexa, on a flip phone the software is the only thing preventing the microphone (which is never physically disconnected) from sending audio to some other device/person.
anigbrowl 5y ago
Strange argument: - government may use this in future to tag people for wrongthink imagery, but also - CP is rampant on FB and FB reports it all but nothing happens so what difference does it make
- jhanschoo 5y ago
  > but nothing happens Indeed; I'm not sure this premise holds. I personally don't trust Apple's promises on privacy given its actions in China shows that it values market access more than its privacy values. But I find this argument weak.
- cryptoquick 5y ago
  The argument those points are supporting, as the article explains, this is an ineffective way to fight CP, but it's being done anyway, and can be used for nefarious purposes. Could a government not have a different level of interest in law enforcement over CP vs, say, political material that undermines government authority?
leyafo 5y ago
step away from the fancy tools https://liyafu.com/2021-08-17-keep-away-the-fancy-tool/ https://liyafu.com/2021-08-17-keep-away-the-fancy-tool/

123 points by guiambros 5y ago

> That’s so that criminals can’t just disguise the image with a trivial change. It's utterly trivial to modify the image to avoid matching. Most minor modifications will do so. The purpose of the hash is so that idiotic criminals that aren't trying to avoid getting caught don't accidentally avoid getting caught just by resizing or recompressing the images. The flawed 'hashing' function also creates plausible denyability: If its somehow proved, perhaps via a whistleblower, that Apple was in fact matching the historic tank man photo, Apple or the government could easily hand over a piece of child porn with the same neuralhash due to the extraordinary vulnerability of the function.

raxxorrax 5y ago
Don't know why you are downvoted because I think this is still true for the most part that you should be able to trivially create another hash for the same image with unintrusive operations. You might need to put some efforts in it and reverse engineer the hashing algorithm a bit and see what causes rolling a new hash value. A lot of research went into classifying images, mostly from the side that wanted visible or invisible watermarks in pictures. It is a cold war against image cropping, compression and modification that tends to not favor the classification side. But here you only want to have a different hash, the operations for that are numerous and probably indeed often trivial. I don't think you could just have a completely different picture create a collision though. Maybe they can because it is their hash-function, but that would be quite unusual and I doubt this is the case.
- nullc 5y ago
  > I don't think you could just have a completely different picture create a collision though. Allow me to introduce you to my posts on github: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issues/1#issuecomment-903094036 https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue... Where I post good looking examples of standard test images altered fairly subtly to give the specific hashes. The apple neuralhash is broken as a 'hash function'. It's much much easier to modify images to just have a different hash. A simple blemish on the image-- or, with whiteboxing using the hash function, no visually noticeable change is required at all.
  - raxxorrax 5y ago
    That is seriously the hash function they used? I assumed a competent hash function, hell, any common hash function is better even if they fail at resize/compression. This one would too.
    - nullc 5y ago
      Apple 'neuralhash' usually survives resize (the first step it does is resize the image to 360x360) or minor compression lossy changes. But the price of that is that it's not secure against generating arbitrary preimages based on fairly arbitrarily targets, with results that look nothing like the original image you're matching (if one even exists). They could do some downsample, transform, quantize, sha256 approach that would still survive some tiny amount of differences. ... but apple favors easily constructed false positives over false negatives, even though its trivial for anyone trying to avoid their function to get a false negative. Their scheme only protects against accidental false negatives. It's just a symptom of systematically favoring factors other than the privacy and security of the actual owner and user of the computer.
      - raxxorrax 5y ago
        Wow, that is seriously a naive approach. Honestly, I would have expected much more here. That makes the libraries of hashes for content almost useless. Hope they extracted features that could be used in a more sophisticated approach.

exporectomy 5y ago

"If you’re right wing, look at how privately taken video and photos have been used to track down the January 6th Capitol rioters" How is that supposed to appeal to right wingers? He thinks all right wingers want those rioters to escape justice? Most republicans oppose them according to several polls. The author may be a victim of political divisiveness due to paying too much attention to social media and news.

redis_mlc 5y ago
All Americans want the right to protest. In fact, Democrat protesters occupied the Capitol building in 2018. I'm not aware of a manhunt for those protesters. > Most republicans oppose them according to several polls. I suggest the poll be re-run after you tell people that the MSM narrative was a lie - protesters killed nobody, and police opened the doors. Nobody arrested had a gun. AOC has told 3 different stories, yet she was a mile away at the time. In fact, there was no "insurrection", confirmed by the FBI last week. But there was a Marxist-style narrative - they're very clever with wording, aren't they? Why didn't Pelosi add more security for 100,000 protesters? That's the only fact we really need to know, isn't it?
roenxi 5y ago
It does show an ignorance of how the right wing thinks. However, it is directionally accurate - being an effective right wing leader is likely to get someone's phone targeted. Not a partisan thing; effective left wing leaders also have their phones targeted. The swamp doesn't like change.

crummy 5y ago

I thought with enough matches Apple would decrypt your images, at which point they would be able to verify if they were CSAM (and notify authorities) or realise they weren't and that a government had tried to trick them (and do nothing or go public with this information). Is this not how it works?

Jtsummers 5y ago
That is how they've described the system. After 30 or so matches, they'll be able to decrypt a version (presumably scaled down, the description is vague) of the images and manually review them. With the present system targeting CSAM imagery, unless the images you have match their expectation of matching images, they wouldn't pass it on to law enforcement. This is where the slippery slope argument comes in. Apple would have to actively cooperate with governments to both detect and report other kinds of images. So if Tiananmen Square images were showing up in the review pile, Apple would have to know to pass those on to the Chinese government and be willing to. The only other way they'd be passed on is if Apple drops the manual review portion.
- lazide 5y ago
  Not quite. Any manual review system is going to have infrastructure to pick reviewers, route the data around, etc. it would not be hard, and could be done under the radar, to intercept data for specific targets in a way that wouldn’t arouse suspicion, and it would be awfully hard to prove it had happened too.
  - Jtsummers 5y ago
    So you're agreeing with me. Apple would have to actively cooperate by providing reviewers who would go along with the kind of media besides CP that nations may want to detect, or Apple would have to actively cooperate by simply passing matches on and bypassing the review process. But Apple would have to be the ones doing it either way.
    - lazide 5y ago

KSteffensen 5y ago

"How much attention do you think police gave each of those 20 million reports from Facebook?" Killer argument. Why not focus here instead?

watt 5y ago

You know what I hate? Titles that say specific words that are never repeated in the article. A title that has so little to do with the contents of article, that it might as well be a random reference to some throw-away line in some comment somebody uttered sometime, which might as well be left alone so a proper title could be selected.

BLKNSLVR 5y ago
The "screeching voice of the minority" is that which Apple is catering to in it's image scanning, and since the image scanning is what the article is about, then I don't have a problem with it. Normally I would agree, but I'm fine with the subtle(?) implication of this one.
- Jtsummers 5y ago
  That's not what the title is about. The title is establishing the author and the article as part of the "screeching voice of the minority", a reference to a press release that dismissed concerns about the system as, essentially, being unimportant and which used that phrase to dismiss, out of hand, opponents of the scanning system.
  - BLKNSLVR 5y ago
    Hah, you're right, thanks. Didn't know about the memo until reading other comments.
edflsafoiewq 5y ago
It's a reference to > We know that the days to come will be filled with the screeching voices of the minority. https://9to5mac.com/2021/08/06/apple-internal-memo-icloud-photo-scanning-concerns/ https://9to5mac.com/2021/08/06/apple-internal-memo-icloud-ph... from a letter from NCMEC that Apple circulated in an internal memo.
- watt 5y ago
  It is, but why not address it in the article? The article does not have a word "minority" or "screeching" in the text.

thatsabadidea 5y ago

100% this will be used to track dissidents with non-official-narrative memes. For your own good.

justbored123 5y ago

The hysteria about this is extremely childish. After the Pegasus news that showed that a relatively small private company in a very small country could have complete remote access to your phone and sell it on the market, do you really think that a powerful government like China or the US can't already do it??? How could you be this naive? Especially after the Snowden revelations. And about Apple's having a change of hearth in the future and using something like this against you. Have you stopped to think for one second and consider that they have complete control over the updates that your phone receives?? They can quietly do what ever they want without you ever knowing and you can't do sh*t about it. So drop the concerned citizen act. We all abdicate privacy a long time ago when we sold our souls for cool free services like google and sexy gadgets. At the end of the day the "slippery slope" argument boils down to one thing: do you trust your institutions? Any law or service can be abused, and the only thing preventing that is the integrity of the people on your society and the institutions both public and private that they work for. The exact same law that is used to fight corruption in a great country like Norway or Denmark can be abused to persecute political opponents in a shitty country like North Korea or China. If the answer is no, I don't trust my institutions and my people, then you have a far more severe and fundamental problem than an specific law or service. In that scenario, focusing in an specific issue is like trying to cover the sun with your hands.

noptd 5y ago
And? These whataboutisms are a distraction and in no way support the idea that people shouldn't be worried about Apple implementing a new client-side system that's ripe for abuse. If things are already bad, we should be working towards making them better instead of rolling over and apathetically allowing them to get worse.
Jensson 5y ago
If we don't speak up when they push the boundaries they will just continue pushing. The only thing stopping Apple from doing all of those bad things today is that they are afraid of bad PR. You should be happy that others speak up now rather than you waiting until what they do becomes bad enough that you feel the need to speak up against it.

rossdavidh 5y ago

Good summary of the situation. I am, unfortunately, pretty much completely in agreement on all points.

da_chicken 5y ago

As usual, they trot out "think of the children!" whenever they want to start doing something horrifically invasive or trivial to abuse. Just what we want: social standing determined by YouTube algorithm.

darepublic 5y ago

I feel like it's a matter of time before chrome adopts similar bullshit. Just in time censorship

r721 5y ago
Chrome for Windows already scans local files for "unwanted software" (using ESET’s technology), other kinds of hashes could be easily added there: >As applied in Chrome Cleanup, ESET’s technology is used by Google to alert users about unwanted or potentially harmful software attempting to get on users’ devices through stealth, for example, by being bundled into the download of legitimate software or content. Google Chrome, using ESET’s security technology, then provides users with the option to remove the unwanted software. Chrome Cleanup operates in the background, without visibility or interruptions to the user. It deletes the unwanted software and notifies the user once the cleanup has been successfully completed. https://www.eset.com/int/about/newsroom/press-releases/company/google-chrome-and-eset-collaborate-in-fight-against-online-threats-3/ https://www.eset.com/int/about/newsroom/press-releases/compa...
runjake 5y ago
Chrome/ChromeOS/Android already do the same thing with cloud synced photos but it happens on the server side. Google has been scanning for years now. Same with Microsoft. And Dropbox. And…
- mastazi 5y ago
  If it's private and/or sensitive, it probably shouldn't be stored with a cloud-synced service. Recently I've seen a lot of "experts" trying to reassure the non-technically oriented about the fact that it's not true that "the cloud is not someone else's computer", that that's just a meme and it's not accurate. But they never thoroughly explain why that wouldn't be the case. When you dig deeper, it turns out that more often than not those "experts" are people whose job is to sell cloud services, or who work for companies that sell cloud services.
- judge2020 5y ago
  "Chrome/ChromeOS/Android" doesn't work when Android is a completely separate piece of software from the other two, and only Android/ChromeOS have any sort of file syncing to Google servers thus meaning no PhotoDNA scans if you only use the Chrome browser.

robertoandred 5y ago

Apple does not report you to law enforcement, as this article claims. The article also ignores human review. The article compares scanning to facial/object/scene recognition, which it can’t do. The article talks about making it easier to find who took a photo, which is not the problem the Jan 6 sleuths were trying to solve. The article worries about China, but ignores that matchable photos must appear in multiple jurisdictions’ databases. The article also ignores the several points of third-party auditing and several layers of human reviews.

jcims 5y ago
What do you think the public reception would’ve been if this technology had been available last year, and multiple jurisdictions published hashes of images from the riots at the capitol? Or maybe, more directly, what do you think Apple employees reviewing these hits would do if those types of images started showing up in the triage list.
- robertoandred 5y ago
  What exactly do you think publishing hashes of Jan 6 pics would achieve? The photographers posted them publicly on Facebook and Twitter. That’s how we have them in the first place. The problem wasn’t knowing who took the photos, it was knowing who was in the photos. And that needs facial detection, which this system can’t do.
  - jcims 5y ago
    It would achieve exactly what the system is designed to do, mine personal devices for photos that authorities are aware of. Just because a lot of people were sloppy doesn’t mean every person was. However, my primary point is that there is no possibility of a technical control over the scope of the program. It’s a policy decision and is subject to explicit expansion or collusion/parallel construction.
    - robertoandred 5y ago

a3n 5y ago

It might be a good idea not to travel outside your home country with an iPhone or other Apple tech, because you don't necessarily know what kinds of photos or other digital files a country's government would object to. The picture of the man against the tanks is a good example. Indeed, you don't necessarily know what photos are on your phone.

tjpnz 5y ago
The less gadgets you bring across the border the better. In New Zealand for instance a customs official can now force you to unlock a device on your person for "suspicions" no matter how vague. The only way around it is to refuse to play and leave everything at home.
- raxxorrax 5y ago
  Which is a completely archaic policy. As if there wasn't a worldwide network that makes information exchange beyond borders quite trivial. That said, leaving your devices at home is a sensible idea.
secondaryacct 5y ago
I wonder if someone cheeky could "swat" you by whatsapping a collection of known triggering images !
- wizzwizz4 5y ago
  Swatting is not “cheeky”.

sigmaprimus 5y ago

A few years ago a little girl went missing and unfortunately was later found dead in a nearby lake. It was determined that she had been abducted and killed. The local news station reported that there were approximately 15000 pedophiles in the surrounding region. I found it incredible that there were so many deviants known by the police in my community, they eventually caught the killer. He turned out to be a neighbor and confessed. I think the police are well aware of every download/upload of child porn or are atleast they are able to look it up when the need arises. The time for concern of protecting ones privacy on any digital connected device has come and gone, best to assume anything You do on your device is readily available for the authorities(and others) to exploit. *it was 21 years ago, and the only news report I found says 100 known but I am positive they said 15K.* https://www.cbc.ca/news/canada/missing-girl-s-body-found-in-b-c-lake-1.232583 https://www.cbc.ca/news/canada/missing-girl-s-body-found-in-...

judge2020 5y ago
What's the chance that the news station said "out of 15000 potential perpetrators in the surrounding area, 100 are registered sex offenders"?
michaelmrose 5y ago
It is completely trivial to render your communications opaque to casual surveillance. If someone were truly committed to surveiling you it would be drastically harder. At the point where people are hacking your devices and watching you from across the street impossible. About 1/2 of 1% (estimates vary) of people are sexually attracted to children and an unknown number commit acts against children. Statistically if your kid gets abused it will probably be by someone you know and shouldn't have trusted. You should probably have a healthy mistrust regarding men alone with your kid and the government without being paranoid.
- sigmaprimus 5y ago
  >>>It is completely trivial to render your communications opaque to casual surveillance... Im not sure I understand what You mean by this, my thought or possible concern is that all web traffic, especially that which goes through known proxy addresses is being logged and flagged by local carriers and ISPs. Then in the event of a crime being commited in my area, my actions online including stuff from decades past will invite a knock on my door by the authorities or worse the media. I must stress, Im not worried about anything sexual but have done a great deal of research on some other subjects that are not exactly above board. I can appreciate the concern regarding apple scanning personal files and reporting matching files to the government, but feel the odds of a virus, vulnerability or exploit being used by a criminal ngo is just as big if not bigger risk and has been for a long time now. Anyways, like I said...for a while now I have been of the mindset that anything I do on a connected device (even temporarily connected) leaves me exposed, now or in the future. Maybe cancel culture has encouraged my current way of thinking.

nbzso 5y ago

I moved on. Pop!_OS (no Ubuntu telemetry and OpenSnitch) over MacBook Air is working flawlessly. Company MacBooks are with Arch/Manjaro. Will run Catalina for specific tasks without internet until it breaks completely. The hostile design decisions of Apple are unbelievable. After 20 days of public backslash Apple is preparing for massive iPhone 13 campaign and pushing this obviously flawed intrusion with "common good" goals, normalizing surveillance on a new level. And all of this after Pegasus/NSO. But actually I am glad. No more shiny toys and marketing illusions for me. Expect everybody to follow suit. Year or two down the road this type of scanning and control will be mandatory. So the only way to respond is to invest in your privacy. Not in user-hostile, dark pattern designed products.

judge2020 5y ago
> Expect everybody to follow suit. Year or two down the road this type of scanning and control will be mandatory. Indeed, this article (despite many incorrect facts about the process) is right in that this is just a vocal minority 'screeching' to get the message out there to anyone willing to listen. The absolute majority of the population, or at least adult population, will always choose the path of least resistance - they just wants to live, work, avoid hardship, and partake in dopamine-inducing activity every day. There's a reason Google and Amazon are trillion-dollar enterprises and Hulu still has an ad-supported version that 70% of people choose[0] - people don't value privacy at all when dealing with these faceless companies. Society has always shifted the level of privacy they accept (think taxes, audits, search & seizure with probably cause), and this will just be another shift that people accept so that they continue on with their lives while not sacrificing any of their valuable time. 0: https://variety.com/2019/digital/news/hulu-ad-supported-subscribers-70-percent-1203227954/ https://variety.com/2019/digital/news/hulu-ad-supported-subs...
- geeB 5y ago
  The hulu number was quite surprising to me, given that the ad supported tier is $6/mo vs $12/mo for ad free, and if someone watches enough the extra $6/mo would seem worth it. It might have a lot to do with their $1.99 for one year promotion they do on Black Fridays though, which would change things considerably ($2 vs $144 per year), especially for households without enough disposable income. Easy enough to switch to a different person every year and keep it going for a while, and also at the time of that article it had been less than 3 years since the free tier with ads was removed. I don't think privacy has much to do with it though. People that don't have any idea about these topics just wouldn't care, and at least in my case I care but I'm under no illusion that paying for ad-free changes anything other than the convenience of not having ads. iirc hulu's ads cannot be skipped anyway, so they pretty much can collect the same exact information from both tiers.

bigcorp-slave 5y ago

The core problem here is reporting to prevent CSAM (hopefully, it is uncontroversial that doing so is desirable) but rather that this makes on-device snitching on any topic a matter of -policy- rather than -technology-. It is much harder for a government to say, you must implement this large and complex feature, you must keep it secret, etc - especially a foreign government - than it is to say, here are some new hashes, please plug them in to your database that four people have access to. Or to say, please scan all images instead of just those uploaded. Even if Apple wants to right by its users, it is making it much easier for Apple to be compelled to do wrong. “We promise” is easily forgotten. “We can’t” requires dev teams to change.

3grdlurker 5y ago
I agree about how the technology can be applied to other instances, but I feel that people who are criticizing Apple are crucially placing all the blame on Apple but not on the government that pressured them into doing this in the first place. This CSAM issue is proof that if even the most valuable company in the world can be pressured into compliance, then any company in the world could be, calling into question whether it is realistic in the first place to expect companies to act ethically while also creating highly profitable products. For all the things that have been said about the issue, I don’t remember a single comment that has been directed at the US government, or China itself, or any discussions about how nation-states should be run and genuine introspections on what could prevent surveillance states from existing in the first place.
- judge2020 5y ago
  This is because people, for some reason, see Apple's 3-trillion-dollar valuation as being some purely self-made fortune to the point that Apple could rival these governments with their technology stance, when in reality Apple's success has only been enabled by the governments that house and profit from Apple's dominance - and now they're coming for their share of surveillance instead of dollars.

systemvoltage 5y ago

America is kneeling down to the same playbook that authoritarian regimes use, just a more vanilla version of it. Ready to be exploited one day. I am happy that the tech community on HN and elsewhere are speaking up about this. I was expecting it to be just another "think of the children" theatre ready to be swept under the rug. Major newspapers are talking about it. If anything else, next time they say the word "Privacy" at the Apple WWDC, we should boo that non-sense and let them pay the price in PR-damage. Apple spent last 3-4 years building their image as privacy-pro alternative to Google/FB/Microsoft in various product/service categories. They compltely fucked up. Privacy is about trust and it takes a single event to forever destroy it.

audiodude 5y ago
Perhaps they spent the time building capital as privacy respecting so they could "spend" it on a moment like this?

tamaharbor 5y ago

Alexa has been listening in on all your sexual adventures at home anyway, so what difference does this make?

judge2020 5y ago
If you have any proof that'd be nice. Just because there's a microphone in the room, such as via a flip phone or even rotary phone, doesn't mean it's actively recording audio to be shipped off for later blackmail/extortion/embarrassment.
- perryizgr8 5y ago
  Echo devices' microphones are turned on all the time, unlike a flip/rotary phone, whose mics are physically disconnected when the handset is in the cradle.
  - judge2020 5y ago
    And? Just like Alexa, on a flip phone the software is the only thing preventing the microphone (which is never physically disconnected) from sending audio to some other device/person.

anigbrowl 5y ago

Strange argument: - government may use this in future to tag people for wrongthink imagery, but also - CP is rampant on FB and FB reports it all but nothing happens so what difference does it make

jhanschoo 5y ago
> but nothing happens Indeed; I'm not sure this premise holds. I personally don't trust Apple's promises on privacy given its actions in China shows that it values market access more than its privacy values. But I find this argument weak.
cryptoquick 5y ago
The argument those points are supporting, as the article explains, this is an ineffective way to fight CP, but it's being done anyway, and can be used for nefarious purposes. Could a government not have a different level of interest in law enforcement over CP vs, say, political material that undermines government authority?

leyafo 5y ago

step away from the fancy tools https://liyafu.com/2021-08-17-keep-away-the-fancy-tool/ https://liyafu.com/2021-08-17-keep-away-the-fancy-tool/