10 ms·

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates o

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in China", what do you think Apple is going to do? Is anyone so naive that they believe the CCP wouldn't deliver such an ultimatum? Apple's position seems to completely ignore recent Chinese history.

tyingq 5y ago
>(like a foreign government) It seems like it wouldn't take that. If you can generate a colliding pair of images, you could probably create a pair where one of the images might get attention with child porn groups and thus, shared around enough to end up in the CSAM database. And where the other was innocuous.
bsdetector 5y ago
> If the CCP says "put these hashes in your database or we will halt all iPhone sales in China", what do you think Apple is going to do? Or maybe China already said "put in this CSAM check or you can't make or sell phones in China". Since Apple's position is contrary to their previous privacy policy and doesn't seem to make a lot of sense, it's quite possible extortion already happened (and not necessarily by China).
- ballenf 5y ago
  I'd honestly believe such pressure came from US domestic intelligence or law enforcement agencies just as easily as from China.
  - adventured 5y ago
    It wouldn't specifically be from domestic intelligence, it would be from a powerful member of Congress with a relationship to Apple (specifically the board/management), acting as a go-between that would try to politically lay out the situation for them. Hey Apple, we can either turn up the anti-trust heat by a lot, or we can turn it down, which is it going to be? Except it would be couched in intellectually dishonest language meant to preserve a veneer that the US Government isn't a violent, quasi-psychotic bully ready to bash your face in if you don't do what they're asking. The interactions with intelligence about the new program would begin after they acquiesce to going along. It's too easy. There's an extraordinary amount of wilful naivety in the US about the nature of the government and its frequent power abuses (what it's willing to do), despite the rather comically massive demonstration of said abuses spanning the entire post WW2 era. Every time it happens the wilfully naive crowd feigns surprise.
slg 5y ago
I would expect Apple to say the same thing if the CCP proposed a system of scanning devices last month. I fail to see how this system changes the calculus for how Apple will deal with authoritarian governments. If Apple could stand up to them before this system, why can't they stand up to them with this system?
- tines 5y ago
  Tin-foil hat time: Who's to say that they could stand up to them before this system? The system itself could have been proposed by the CCP in the first place. I'll take my hat off now.
  - deleted 5y ago
    [deleted]
- cmelbye 5y ago
  Apple doesn't stand up to authoritarian governments. They give direct access of their systems to the US and Chinese governments already.
  - 99mans 5y ago
    This. We've known Apple is backdoored since at least 2013 with Snowden revelations. Why are people still debating this nearly a decade later? Apple products ARE NOT PRIVATE.
- foerbert 5y ago
  The difference is the ease with which they can demur. Before, it would be a whole heck of a lot of new, additional work. They also have the problem of actually introducing it without being noticed, or having to come up with some cover for the new behavior. Now? Well now it's real simple. It will even conveniently not expose the actual images it's checking for. Apple now has significantly less ability to rationally reject the request based on the effort and difficulty of the matter. Even Apple's own reasons to reject the request have decreased. It would have legitimately cost them more to fulfill this request before, even if China did want to play hardball. Now, they have greater incentive to go along with it.
orhmeh09 5y ago
Could you provide specific evidence that China has and would do this? I’ve a hard time recalling any specific cases. Maybe nation-states do this kind of thing, but I’m only aware of the countless times the United States has done this. What’s the recent history?
- waz0wski 5y ago
  https://www.reuters.com/article/us-china-apple-icloud-insight-idUSKCN1G8060 https://www.reuters.com/article/us-china-apple-icloud-insigh... https://support.apple.com/en-us/HT208351 https://support.apple.com/en-us/HT208351
  - dotcommand 5y ago
    > https://www.reuters.com/article/us-china-apple-icloud-insigh https://www.reuters.com/article/us-china-apple-icloud-insigh... Apple has to store data of chinese citizens in china? And Apple has to adhere to chinese laws in china? How insane. It's crazy how deluded the "CCP crowd" are. Apparently, the "CCP crowd" thinks companies are allowed to do business in another country and not abide by their laws. Are you going to go insane since the EU requires tech companies to store EU citizens data within the EU? https://blogs.microsoft.com/eupolicy/2021/05/06/eu-data-boundary/ https://blogs.microsoft.com/eupolicy/2021/05/06/eu-data-boun... You would think most countries would demand their citizens data be stored locally.
  - orhmeh09 5y ago
    Where do you think your iCloud keys are stored? Switzerland?
- willis936 5y ago
  Do you have access to this webpage? https://en.wikipedia.org/wiki/1989_Tiananmen_Square_protests https://en.wikipedia.org/wiki/1989_Tiananmen_Square_protests
  - orhmeh09 5y ago
    What does this have to do with forcing corporations’ hands?
programzeta 5y ago
Apple’s trying to mitigate this by putting themselves to a more internationally focused standard of matches against at least two separate sources of CSA hashes, if my understanding of their announcements/follow-ups is right. Separately, the US has even greater pressure on Apple in the case they want to unilaterally add database images, considering they have a actual chance and means to jail (and run through the legal ringer) whomever tells them ‘no’. And that’s just the overt pressure available; I think this is a more likely potential for trust violation here, even if both could come to pass.
breuleux 5y ago
Presumably Apple would be afraid that, say, the EU becomes suspicious, issues a court order to obtain the hashes, notices they cannot audit the CCP hashes, pointedly asks "what is this", becomes absolutely livid that their citizens are spied on by a country that is not them, fines Apple out the wazoo, then extradites whoever is responsible and puts them in prison. I mean, China's not the only player in this. Putting extra hashes to surveil Chinese citizens, yeah, they might do that, but it'd be suicide to put them anywhere else.
- riffraff 5y ago
  Seems pretty trivial to have different servers sides per country, and put there a different db. EU, China, Iran, US: everyone gets to spy on their own children and forbid whatever they want.
  - breuleux 5y ago
    The db is encrypted and uploaded to user devices. If each country gets a different db, the payload will be different in each country, which does not make sense if it's all supposed to be CSAM. So Apple would likely just say "these were mandated by the US government for US citizens," punting the ball in their court, unless they are forbidden to say so, in which case they'll say nothing, but we all know what it means. That's when you know you should change phones and stop using all cloud services, because obviously all cloud services scan for the same thing. On the flip side, though, at least Apple will have given us a canary. And that's why I don't think Apple will be asked to add these hashes: if the governments don't want their citizens to know what's being scanned server side, pushing the equivalent data to clients would tip their hand. They might just write Apple off as a loss and rely on Google, Facebook, etc.
    - sillysaurusx 5y ago
      This isn’t true. The db is blinded. We have no way of knowing what’s in it. It would be trivial to have the same payload on each device, and extract different answers using the matching server side db which varies by country. Perhaps not trivial, but just short.
pbhjpbhj 5y ago
I think one of the other stories on this talked about "watermarking" in order to create a hash collision. So it need not be a non-CSAM image, a TLA could just alter an image to make it collide with a file they want to track, other countries would agree that file's hash should be in the hash list and bingo: Apple presumably provide the TLA with a list of devices holding that file. ?
- duskwuff 5y ago
  Except that there's a threshold involved. A single matching file doesn't trigger an investigation; it takes multiple (10+, maybe more?) matches to do that.
  - kemayo 5y ago
    In the interview Craig Federighi gave on Friday, he said 30.[1] I have no idea if he was just throwing a number out or if that's the actual threshold. [1]: https://tidbits.com/2021/08/13/new-csam-detection-details-emerge-following-craig-federighi-interview/ https://tidbits.com/2021/08/13/new-csam-detection-details-em...
    - duskwuff 5y ago
      Either way, it's high enough that adding a single file to the set wouldn't be a useful way of finding people who have that file. One attack I can imagine would be to add a whole set of closely related photos (e.g. images posted online by a known dissident) to identify the person who took them, and even that would be a stretch.
beepbooptheory 5y ago
The sinophobia of this website is getting to be more than tiring and really just becoming worrisome. Would never of bet that such smart and thoughtful people in general could be so so stuck within certain political talking points, to an almost fanatical degree. If people on HN are generally like this, I have little hope for the rest of our world and its future. At the end of the day all the bad guys will still win, even if we did give a lot of attention to a subset of them for a while I guess.
yongjik 5y ago
These scenarios sound rather like "the wrong side of airlock" stories[1]. Why would China go through an elaborate scheme with fake child-porn hashes, when it can already arrest these people on made-up charges, and simply tell Apple to provide the private key for their phones, so that they can read and insert whatever real/fake evidences they want? [1] I'm stealing the expression from this excellent article: https://devblogs.microsoft.com/oldnewthing/20060508-22/?p=31283 https://devblogs.microsoft.com/oldnewthing/20060508-22/?p=31...
- axelf4 5y ago
  That is not a good comparison. The extra hashes would help China find out about more borderline citizens than it otherwise would have.
  - stjohnswarts 5y ago
    Have we established that a US NGO is accepting "CSAM" hashes from China or that they are cooperating with them at all? That seems unlikely and Apple hasn't yet announced plans with how they're going to scan phones in China, I mean wouldn't China just demand outright to have full scanning capabilities of anything on the phone since you don't have any protection at all from that in China?
    - tablespoon 5y ago
      > Have we established that a US NGO is accepting "CSAM" hashes from China or that they are cooperating with them at all? I believe Apple's intention is to accept hashes from all governments, not just one US organization. One of their ineffectual concessions to the criticism was to require two governments provide the same hash before they'd start using it.
      - avereveard 5y ago
        China can definitely find a state government requiring some cash injection to help push the hash of a certain uninteresting square where nothing happened into the db
captn3m0 5y ago
Related, the Indian Government (Telecom Department) bullied Apple into building an iOS feature for reporting phone calls and SMS by threatening to stop iPhone sales in India. Apple complied. https://indianexpress.com/article/technology/mobile-tabs/apple-ios-12-block-pesky-calls-texts-trai-5268885/ https://indianexpress.com/article/technology/mobile-tabs/app...
- deleted 5y ago
  [deleted]
- c7DJTLrn 5y ago
  I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technology. I just feel so lost and powerless these days, I don't know how much longer I can go on when every piece of technology I own is working against me - tools designed to serve a ruling class instead of the consumer. I don't like it one bit.
  - gsibble 5y ago
    The feeling is mutual. The devices we own are now being used to actively police us.
    - wobbly_bush 5y ago
      The parent comment might have misunderstood what the government asked for. It asked for a feature to "report spam" by end users.
      - c7DJTLrn 5y ago
        Maybe I did. But what difference does it make? There's plenty of other instances where Apple has reportedly been bullied into action or inaction (being dissuaded from implementing E2EE for iCloud is one example). I've really just reached a breaking point and I'm sorry if logic does not apply.
tablespoon 5y ago
>> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. > If the CCP says "put these hashes in your database or we will halt all iPhone sales in China", what do you think Apple is going to do? Is anyone so naive that they believe the CCP wouldn't deliver such an ultimatum? Apple's position seems to completely ignore recent Chinese history. Apple policy << Local laws and regulations. It's very hard to believe their policy is anything less than a deliberate PR smokescreen meant to disarm critics, because it has so many holes. Edit: just thought of another way Apple's policy could be easily circumvented and therefore cannot be regarded as a serious proposal: get two countries to collaborate to add politically sensitive hashes to the list (e.g. China and North Korea, or China and Cambodia). That doesn't even require Apple to be coerced.
Y_Y 5y ago
What about giving a censored version of the appropriate image? Like put a big black rectangle covering whatever awful thing is the subject, and just (e.g.) show some feet and hands and a background. Then you could provide a proof that an image which is the same as the "censored", one except for the masked part, has the perceptual hash specified. I don't know if this is technically feasible (but I'd be happy for someone knowledgeable to opine). I also admit that there are secondary concerns, like the possibility of recognising the background image, and this being used by someone to identify the location, or tipping off a criminal. Probably it would only be appropriate to do this in the case of someone being accused, and maybe then in a way where they couldn't relay the information, since apparently they don't want to make the hash database public. Also, for the record, I'm spitballing here about infosec. This isn't me volunteering to draw black boxes or be called by anyone's defense.
tshaddox 5y ago
If the CCP says “put this arbitrary software into your next iPhone software update or we will halt all iPhone sales in China,” what do you think Apple is going to do? Isn’t the answer to both questions the same?
- fsflover 5y ago
  The answers are probably the same. What different is how hard it is to discover.
  - tshaddox 5y ago
    It apparently wasn't hard to "discover" the fact that this CSAM database can and will change over time. In fact, Apple explained this in detail as well as how they are attempting to avoid the problem of governments abusing the system. Are you suggesting that a different software update might be even easier to discover?
    - deleted 5y ago
      [deleted]
- noptd 5y ago
  So we should simply accept systems with a high potential for abuse because of the possibility that something bad is already being done?
  - tshaddox 5y ago
    What has more potential for abuse than the fact that Apple can push any software they want to iPhones at any time?
    - drdeca 5y ago
      Can they force install updates? I think it’s been over a year since I updated my iPhone’s os.
btilly 5y ago
If the CCP says "put these hashes in your database or we will halt all iPhone sales in China", what do you think Apple is going to do? I think that the real risk is, "put these hashes in your database or we will stop all iPhone related manufacturing in China".
- vineyardmike 5y ago
  Both would be bad for apple. Chinese sales are already significant enough to their bottom line to comply.
vineyardmike 5y ago
The only way this matters [today] though is if apple turns it on for all pictures, not just icloud ones. Presumably "Chinese iCloud" already scans uploaded photos cloud side. Unless the goal is to simply the effort/expense of scanning by making it a client process.
- CheezeIt 5y ago
  > The only way this matters though is if apple turns it on for all pictures, And there’s no way the CCP would order Apple to do that.
true_religion 5y ago
Of course Apple would do it. They’d look like fools for saying they want to stop CP, then refusing the listen to the government of nearly a billion people when it says “Please ban this newly produced material”. At best, they would look biased. At worst, they would be sending a signal that they don’t care about Chinese children. You wouldn’t even need government strong arming, mobs of netizens would happily tear Apple down.
batch12 5y ago
From the post yesterday discussing collisions, it doesn't seem outside the realm of possibility to take an image of CSAM and modify it until it has the hash that matches another target wanted either.
godelski 5y ago
Or why not "Hey Vietnam, Pakistan, Russia, etc put these hashes into your database please and thanks." I mean the CCP has allies that are also authoritarian. Why would they have to threaten Apple directly? This is also how you get past the Apple human verification. Just pay those Apple workers to click confirm.
- adventured 5y ago
  > Why would they have to threaten Apple directly? They'd do it directly because it's expedient and useful. If you're operating such a sprawling authoritarian regime, it's important to occasionally make a show of your power and control, lest anyone forget. The CCP isn't afraid of Apple, Apple is afraid of the CCP. Lately the CCP has been on a rather showy demonstration of its total control. If you're them it's useful to remind Apple from time to time that they're basically a guest in China and can be removed at any time. You don't want them to forget, you want to be confrontational with Apple at times, you want to see their acknowledged subservience; you're not looking to avoid that power confrontation, the confrontation is part of the point. And the threat generally isn't made, it's understood. The CCP doesn't have to threaten in most cases, Apple will understand ahead of time. What gets made initially is a dictate (do this), not the threat. If something unusual happens, such as with Didi's listing on the NYSE against Beijing's wishes (whereas ByteDance did the opposite and kowtowed, pulling their plans to IPO), then, given that Didi obviously understood the confrontation risk ahead of time and tested you anyway, then you punish them. If that still isn't enough, you take them apart (or in the case of Apple, remove them from the country).
  - godelski 5y ago
    I'm just saying that there is another avenue. To be clear, this isn't a "vs" situation. It means that they have multiple avenues. To also clarify, the avenue of extortion isn't open to every country. But the avenue I presented is as long as that country has an ally. I'm not aware of any country not having an ally, so I presume that this avenue is pretty much open to any country.
dwighttk 5y ago
Apple doesn’t control the database
fossuser 5y ago
The CCP already runs iCloud themselves in country so this is a bit irrelevant. (Though I think this kind of capitulation to authoritarian countries is wrong, personally: https://zalberico.com/essay/2020/06/13/zoom-in-china.html https://zalberico.com/essay/2020/06/13/zoom-in-china.html) This policy really needs to be compared to the status quo (unencrypted on cloud image scanning). When you compare in transit client side hash checks that allow for on cloud encryption and only occur when using the cloud it's hard to see an argument against it that makes sense given the current setup. The abuse risk is no higher than the status quo and enabling encryption is a net win.
- clnhlzmn 5y ago
  Saying this is no worse than the status quo isn't a good argument. The status quo is the problem.
Xamayon 5y ago
At this point, with all the easily producible collisions, the Gov't could just modify some CSAM images to match the hash of various leaked documents/etc they want to track. Then they don't even have to go thru special channels. Just submit the modified image for inclusion normally! (Not quite that simple, as they would still need to find out about the matches, but maybe that's where various NSA intercepts could help...)
- Natsu 5y ago
  Not quite, a CSAM hash match triggers another match within Apple to avoid false positives and then a human review. It wouldn't be trivial for them to extract matches out of that, and they'd only be able to track files they already know the contents for. I would think they could more easily just make your phone carrier install a malware update on your phone, rather than jumping through all of these hoops to get them access they already have. Plenty of data is leaking out of people's phones already as can be seen from, e.g. the Parler hack.
  - Xamayon 5y ago
    I tried to address the issue with finding out about the match at the end of my comment. I agree it's not exactly practical without other serious work to intercept the alerts, have 'spies' in the apple review process, etc. Much easier ways would exist at that point, but it's somewhat amusing (in a horrifying way) that some bad actor could in theory use modified CSAM as a way to detect the likely presence of non CSAM content using generated collisions.
- nodamage 5y ago
  All of the major tech companies already scan images uploaded to their services so isn't this already theoretically possible now? How is the situation changed by Apple using on-device scanning instead of cloud scanning (considering these images were going to be uploaded into iCloud anyway).
pharrington 5y ago
Apple's CSAM detection has nothing to do with this. The vector for an authoritarian government getting blacklists into tech is that government telling the tech vendor "ban this content. We don't care how."
FabHK 5y ago
Look, if the government tells Apple to do something, then Apple can push back, but then has to do it or pull out of the country. That's the way it was, and is. Now, what has actually changed? The two compelling push-back against a country's demands for more surveillance etc. are: a) it is not technically feasible (eg, wiretapping E2EE chats), and b) it is making the device insecure vis-a-vis hackers and criminals (eg, putting in a backdoor) The interesting question is: Have these defences against government demands been weakened by this new technology? Maybe they have, that would be my gut feeling. But it is not enough to assert it, one must show it.
JohnJamesRambo 5y ago
Tank Man from Tiananmen Square image comes to mind.
ljm 5y ago
> two countries 1. USA 2. UK, Canada, Australia, New Zealand Pick one from the first list, and one from the second list
jrockway 5y ago
Why does the CCP even need to talk to Apple? They have a database of CSAM, and can modify an image in this set to collide with a special image they're looking for on people's phones. They then share their new modified cache of CSAM with other countries ("hey, China is helping us! that's great!") and it gets added to Apple's database for the next iOS release, because it looks to humans like CSAM. Only the CCP knows that it collides with something special they're looking for. Now that we know that collisions are not just easy -- but happen by themselves with no human input (as evidenced by the Imagenet collisions), we know this system can't work. Apple has two remaining safeguards: a second set of hashes (they say), and human reviewers. The human reviewers are likely trained to prefer false-positives when unsure, and so while a thorough human review would clearly indicate "not CSAM" for the images the malicious collisions match, it doesn't feel like much of a safeguard to me. (Remember the leaked memo from the US's organization -- they called our objections "screeching voices". I'm sure the actual reviewers think similarly.) I assume the people reviewing CSAM for the CCP will be in China, so they can be in on the whole scheme. (In the US, we have slightly better checks and balances. Eventually the image will be in front of a court and a jury of your peers, and it's not illegal to have photos that embarrass the CCP, so you'll personally be fine modulo the stress of a criminal investigation. But that dissident in China, probably not going to get a fair trial -- despite the image not being CSAM, it's still illegal. And Apple handed it directly to the government for you.) I don't know, I just find this thing exceedingly worrying. When faced with a government-level adversary, this doesn't sound like a very good system. I think if we're okay with this, we might as well mandate CCTV in everyone's home, so we can catch real child abusers in the act.

tyingq 5y ago

>(like a foreign government) It seems like it wouldn't take that. If you can generate a colliding pair of images, you could probably create a pair where one of the images might get attention with child porn groups and thus, shared around enough to end up in the CSAM database. And where the other was innocuous.

bsdetector 5y ago

> If the CCP says "put these hashes in your database or we will halt all iPhone sales in China", what do you think Apple is going to do? Or maybe China already said "put in this CSAM check or you can't make or sell phones in China". Since Apple's position is contrary to their previous privacy policy and doesn't seem to make a lot of sense, it's quite possible extortion already happened (and not necessarily by China).

ballenf 5y ago
I'd honestly believe such pressure came from US domestic intelligence or law enforcement agencies just as easily as from China.
- adventured 5y ago
  It wouldn't specifically be from domestic intelligence, it would be from a powerful member of Congress with a relationship to Apple (specifically the board/management), acting as a go-between that would try to politically lay out the situation for them. Hey Apple, we can either turn up the anti-trust heat by a lot, or we can turn it down, which is it going to be? Except it would be couched in intellectually dishonest language meant to preserve a veneer that the US Government isn't a violent, quasi-psychotic bully ready to bash your face in if you don't do what they're asking. The interactions with intelligence about the new program would begin after they acquiesce to going along. It's too easy. There's an extraordinary amount of wilful naivety in the US about the nature of the government and its frequent power abuses (what it's willing to do), despite the rather comically massive demonstration of said abuses spanning the entire post WW2 era. Every time it happens the wilfully naive crowd feigns surprise.

slg 5y ago

I would expect Apple to say the same thing if the CCP proposed a system of scanning devices last month. I fail to see how this system changes the calculus for how Apple will deal with authoritarian governments. If Apple could stand up to them before this system, why can't they stand up to them with this system?

tines 5y ago
Tin-foil hat time: Who's to say that they could stand up to them before this system? The system itself could have been proposed by the CCP in the first place. I'll take my hat off now.
- deleted 5y ago
  [deleted]
cmelbye 5y ago
Apple doesn't stand up to authoritarian governments. They give direct access of their systems to the US and Chinese governments already.
- 99mans 5y ago
  This. We've known Apple is backdoored since at least 2013 with Snowden revelations. Why are people still debating this nearly a decade later? Apple products ARE NOT PRIVATE.
foerbert 5y ago
The difference is the ease with which they can demur. Before, it would be a whole heck of a lot of new, additional work. They also have the problem of actually introducing it without being noticed, or having to come up with some cover for the new behavior. Now? Well now it's real simple. It will even conveniently not expose the actual images it's checking for. Apple now has significantly less ability to rationally reject the request based on the effort and difficulty of the matter. Even Apple's own reasons to reject the request have decreased. It would have legitimately cost them more to fulfill this request before, even if China did want to play hardball. Now, they have greater incentive to go along with it.

orhmeh09 5y ago

Could you provide specific evidence that China has and would do this? I’ve a hard time recalling any specific cases. Maybe nation-states do this kind of thing, but I’m only aware of the countless times the United States has done this. What’s the recent history?

waz0wski 5y ago
https://www.reuters.com/article/us-china-apple-icloud-insight-idUSKCN1G8060 https://www.reuters.com/article/us-china-apple-icloud-insigh... https://support.apple.com/en-us/HT208351 https://support.apple.com/en-us/HT208351
- dotcommand 5y ago
  > https://www.reuters.com/article/us-china-apple-icloud-insigh https://www.reuters.com/article/us-china-apple-icloud-insigh... Apple has to store data of chinese citizens in china? And Apple has to adhere to chinese laws in china? How insane. It's crazy how deluded the "CCP crowd" are. Apparently, the "CCP crowd" thinks companies are allowed to do business in another country and not abide by their laws. Are you going to go insane since the EU requires tech companies to store EU citizens data within the EU? https://blogs.microsoft.com/eupolicy/2021/05/06/eu-data-boundary/ https://blogs.microsoft.com/eupolicy/2021/05/06/eu-data-boun... You would think most countries would demand their citizens data be stored locally.
- orhmeh09 5y ago
  Where do you think your iCloud keys are stored? Switzerland?
willis936 5y ago
Do you have access to this webpage? https://en.wikipedia.org/wiki/1989_Tiananmen_Square_protests https://en.wikipedia.org/wiki/1989_Tiananmen_Square_protests
- orhmeh09 5y ago
  What does this have to do with forcing corporations’ hands?

programzeta 5y ago

Apple’s trying to mitigate this by putting themselves to a more internationally focused standard of matches against at least two separate sources of CSA hashes, if my understanding of their announcements/follow-ups is right. Separately, the US has even greater pressure on Apple in the case they want to unilaterally add database images, considering they have a actual chance and means to jail (and run through the legal ringer) whomever tells them ‘no’. And that’s just the overt pressure available; I think this is a more likely potential for trust violation here, even if both could come to pass.

breuleux 5y ago

Presumably Apple would be afraid that, say, the EU becomes suspicious, issues a court order to obtain the hashes, notices they cannot audit the CCP hashes, pointedly asks "what is this", becomes absolutely livid that their citizens are spied on by a country that is not them, fines Apple out the wazoo, then extradites whoever is responsible and puts them in prison. I mean, China's not the only player in this. Putting extra hashes to surveil Chinese citizens, yeah, they might do that, but it'd be suicide to put them anywhere else.

riffraff 5y ago
Seems pretty trivial to have different servers sides per country, and put there a different db. EU, China, Iran, US: everyone gets to spy on their own children and forbid whatever they want.
- breuleux 5y ago
  The db is encrypted and uploaded to user devices. If each country gets a different db, the payload will be different in each country, which does not make sense if it's all supposed to be CSAM. So Apple would likely just say "these were mandated by the US government for US citizens," punting the ball in their court, unless they are forbidden to say so, in which case they'll say nothing, but we all know what it means. That's when you know you should change phones and stop using all cloud services, because obviously all cloud services scan for the same thing. On the flip side, though, at least Apple will have given us a canary. And that's why I don't think Apple will be asked to add these hashes: if the governments don't want their citizens to know what's being scanned server side, pushing the equivalent data to clients would tip their hand. They might just write Apple off as a loss and rely on Google, Facebook, etc.
  - sillysaurusx 5y ago
    This isn’t true. The db is blinded. We have no way of knowing what’s in it. It would be trivial to have the same payload on each device, and extract different answers using the matching server side db which varies by country. Perhaps not trivial, but just short.

pbhjpbhj 5y ago

I think one of the other stories on this talked about "watermarking" in order to create a hash collision. So it need not be a non-CSAM image, a TLA could just alter an image to make it collide with a file they want to track, other countries would agree that file's hash should be in the hash list and bingo: Apple presumably provide the TLA with a list of devices holding that file. ?

duskwuff 5y ago
Except that there's a threshold involved. A single matching file doesn't trigger an investigation; it takes multiple (10+, maybe more?) matches to do that.
- kemayo 5y ago
  In the interview Craig Federighi gave on Friday, he said 30.[1] I have no idea if he was just throwing a number out or if that's the actual threshold. [1]: https://tidbits.com/2021/08/13/new-csam-detection-details-emerge-following-craig-federighi-interview/ https://tidbits.com/2021/08/13/new-csam-detection-details-em...
  - duskwuff 5y ago
    Either way, it's high enough that adding a single file to the set wouldn't be a useful way of finding people who have that file. One attack I can imagine would be to add a whole set of closely related photos (e.g. images posted online by a known dissident) to identify the person who took them, and even that would be a stretch.

beepbooptheory 5y ago

The sinophobia of this website is getting to be more than tiring and really just becoming worrisome. Would never of bet that such smart and thoughtful people in general could be so so stuck within certain political talking points, to an almost fanatical degree. If people on HN are generally like this, I have little hope for the rest of our world and its future. At the end of the day all the bad guys will still win, even if we did give a lot of attention to a subset of them for a while I guess.

yongjik 5y ago

These scenarios sound rather like "the wrong side of airlock" stories[1]. Why would China go through an elaborate scheme with fake child-porn hashes, when it can already arrest these people on made-up charges, and simply tell Apple to provide the private key for their phones, so that they can read and insert whatever real/fake evidences they want? [1] I'm stealing the expression from this excellent article: https://devblogs.microsoft.com/oldnewthing/20060508-22/?p=31283 https://devblogs.microsoft.com/oldnewthing/20060508-22/?p=31...

axelf4 5y ago
That is not a good comparison. The extra hashes would help China find out about more borderline citizens than it otherwise would have.
- stjohnswarts 5y ago
  Have we established that a US NGO is accepting "CSAM" hashes from China or that they are cooperating with them at all? That seems unlikely and Apple hasn't yet announced plans with how they're going to scan phones in China, I mean wouldn't China just demand outright to have full scanning capabilities of anything on the phone since you don't have any protection at all from that in China?
  - tablespoon 5y ago
    > Have we established that a US NGO is accepting "CSAM" hashes from China or that they are cooperating with them at all? I believe Apple's intention is to accept hashes from all governments, not just one US organization. One of their ineffectual concessions to the criticism was to require two governments provide the same hash before they'd start using it.
    - avereveard 5y ago
      China can definitely find a state government requiring some cash injection to help push the hash of a certain uninteresting square where nothing happened into the db

captn3m0 5y ago

Related, the Indian Government (Telecom Department) bullied Apple into building an iOS feature for reporting phone calls and SMS by threatening to stop iPhone sales in India. Apple complied. https://indianexpress.com/article/technology/mobile-tabs/apple-ios-12-block-pesky-calls-texts-trai-5268885/ https://indianexpress.com/article/technology/mobile-tabs/app...

deleted 5y ago
[deleted]
c7DJTLrn 5y ago
I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technology. I just feel so lost and powerless these days, I don't know how much longer I can go on when every piece of technology I own is working against me - tools designed to serve a ruling class instead of the consumer. I don't like it one bit.
- gsibble 5y ago
  The feeling is mutual. The devices we own are now being used to actively police us.
  - wobbly_bush 5y ago
    The parent comment might have misunderstood what the government asked for. It asked for a feature to "report spam" by end users.
    - c7DJTLrn 5y ago
      Maybe I did. But what difference does it make? There's plenty of other instances where Apple has reportedly been bullied into action or inaction (being dissuaded from implementing E2EE for iCloud is one example). I've really just reached a breaking point and I'm sorry if logic does not apply.

tablespoon 5y ago

>> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. > If the CCP says "put these hashes in your database or we will halt all iPhone sales in China", what do you think Apple is going to do? Is anyone so naive that they believe the CCP wouldn't deliver such an ultimatum? Apple's position seems to completely ignore recent Chinese history. Apple policy << Local laws and regulations. It's very hard to believe their policy is anything less than a deliberate PR smokescreen meant to disarm critics, because it has so many holes. Edit: just thought of another way Apple's policy could be easily circumvented and therefore cannot be regarded as a serious proposal: get two countries to collaborate to add politically sensitive hashes to the list (e.g. China and North Korea, or China and Cambodia). That doesn't even require Apple to be coerced.

Y_Y 5y ago

What about giving a censored version of the appropriate image? Like put a big black rectangle covering whatever awful thing is the subject, and just (e.g.) show some feet and hands and a background. Then you could provide a proof that an image which is the same as the "censored", one except for the masked part, has the perceptual hash specified. I don't know if this is technically feasible (but I'd be happy for someone knowledgeable to opine). I also admit that there are secondary concerns, like the possibility of recognising the background image, and this being used by someone to identify the location, or tipping off a criminal. Probably it would only be appropriate to do this in the case of someone being accused, and maybe then in a way where they couldn't relay the information, since apparently they don't want to make the hash database public. Also, for the record, I'm spitballing here about infosec. This isn't me volunteering to draw black boxes or be called by anyone's defense.

tshaddox 5y ago

If the CCP says “put this arbitrary software into your next iPhone software update or we will halt all iPhone sales in China,” what do you think Apple is going to do? Isn’t the answer to both questions the same?

fsflover 5y ago
The answers are probably the same. What different is how hard it is to discover.
- tshaddox 5y ago
  It apparently wasn't hard to "discover" the fact that this CSAM database can and will change over time. In fact, Apple explained this in detail as well as how they are attempting to avoid the problem of governments abusing the system. Are you suggesting that a different software update might be even easier to discover?
  - deleted 5y ago
    [deleted]
noptd 5y ago
So we should simply accept systems with a high potential for abuse because of the possibility that something bad is already being done?
- tshaddox 5y ago
  What has more potential for abuse than the fact that Apple can push any software they want to iPhones at any time?
  - drdeca 5y ago
    Can they force install updates? I think it’s been over a year since I updated my iPhone’s os.

btilly 5y ago

If the CCP says "put these hashes in your database or we will halt all iPhone sales in China", what do you think Apple is going to do? I think that the real risk is, "put these hashes in your database or we will stop all iPhone related manufacturing in China".

vineyardmike 5y ago
Both would be bad for apple. Chinese sales are already significant enough to their bottom line to comply.

vineyardmike 5y ago

The only way this matters [today] though is if apple turns it on for all pictures, not just icloud ones. Presumably "Chinese iCloud" already scans uploaded photos cloud side. Unless the goal is to simply the effort/expense of scanning by making it a client process.

CheezeIt 5y ago
> The only way this matters though is if apple turns it on for all pictures, And there’s no way the CCP would order Apple to do that.

true_religion 5y ago

Of course Apple would do it. They’d look like fools for saying they want to stop CP, then refusing the listen to the government of nearly a billion people when it says “Please ban this newly produced material”. At best, they would look biased. At worst, they would be sending a signal that they don’t care about Chinese children. You wouldn’t even need government strong arming, mobs of netizens would happily tear Apple down.

batch12 5y ago

From the post yesterday discussing collisions, it doesn't seem outside the realm of possibility to take an image of CSAM and modify it until it has the hash that matches another target wanted either.

godelski 5y ago

Or why not "Hey Vietnam, Pakistan, Russia, etc put these hashes into your database please and thanks." I mean the CCP has allies that are also authoritarian. Why would they have to threaten Apple directly? This is also how you get past the Apple human verification. Just pay those Apple workers to click confirm.

adventured 5y ago
> Why would they have to threaten Apple directly? They'd do it directly because it's expedient and useful. If you're operating such a sprawling authoritarian regime, it's important to occasionally make a show of your power and control, lest anyone forget. The CCP isn't afraid of Apple, Apple is afraid of the CCP. Lately the CCP has been on a rather showy demonstration of its total control. If you're them it's useful to remind Apple from time to time that they're basically a guest in China and can be removed at any time. You don't want them to forget, you want to be confrontational with Apple at times, you want to see their acknowledged subservience; you're not looking to avoid that power confrontation, the confrontation is part of the point. And the threat generally isn't made, it's understood. The CCP doesn't have to threaten in most cases, Apple will understand ahead of time. What gets made initially is a dictate (do this), not the threat. If something unusual happens, such as with Didi's listing on the NYSE against Beijing's wishes (whereas ByteDance did the opposite and kowtowed, pulling their plans to IPO), then, given that Didi obviously understood the confrontation risk ahead of time and tested you anyway, then you punish them. If that still isn't enough, you take them apart (or in the case of Apple, remove them from the country).
- godelski 5y ago
  I'm just saying that there is another avenue. To be clear, this isn't a "vs" situation. It means that they have multiple avenues. To also clarify, the avenue of extortion isn't open to every country. But the avenue I presented is as long as that country has an ally. I'm not aware of any country not having an ally, so I presume that this avenue is pretty much open to any country.

dwighttk 5y ago

Apple doesn’t control the database

fossuser 5y ago

The CCP already runs iCloud themselves in country so this is a bit irrelevant. (Though I think this kind of capitulation to authoritarian countries is wrong, personally: https://zalberico.com/essay/2020/06/13/zoom-in-china.html https://zalberico.com/essay/2020/06/13/zoom-in-china.html) This policy really needs to be compared to the status quo (unencrypted on cloud image scanning). When you compare in transit client side hash checks that allow for on cloud encryption and only occur when using the cloud it's hard to see an argument against it that makes sense given the current setup. The abuse risk is no higher than the status quo and enabling encryption is a net win.

clnhlzmn 5y ago
Saying this is no worse than the status quo isn't a good argument. The status quo is the problem.

Xamayon 5y ago

At this point, with all the easily producible collisions, the Gov't could just modify some CSAM images to match the hash of various leaked documents/etc they want to track. Then they don't even have to go thru special channels. Just submit the modified image for inclusion normally! (Not quite that simple, as they would still need to find out about the matches, but maybe that's where various NSA intercepts could help...)

Natsu 5y ago
Not quite, a CSAM hash match triggers another match within Apple to avoid false positives and then a human review. It wouldn't be trivial for them to extract matches out of that, and they'd only be able to track files they already know the contents for. I would think they could more easily just make your phone carrier install a malware update on your phone, rather than jumping through all of these hoops to get them access they already have. Plenty of data is leaking out of people's phones already as can be seen from, e.g. the Parler hack.
- Xamayon 5y ago
  I tried to address the issue with finding out about the match at the end of my comment. I agree it's not exactly practical without other serious work to intercept the alerts, have 'spies' in the apple review process, etc. Much easier ways would exist at that point, but it's somewhat amusing (in a horrifying way) that some bad actor could in theory use modified CSAM as a way to detect the likely presence of non CSAM content using generated collisions.
nodamage 5y ago
All of the major tech companies already scan images uploaded to their services so isn't this already theoretically possible now? How is the situation changed by Apple using on-device scanning instead of cloud scanning (considering these images were going to be uploaded into iCloud anyway).

pharrington 5y ago

Apple's CSAM detection has nothing to do with this. The vector for an authoritarian government getting blacklists into tech is that government telling the tech vendor "ban this content. We don't care how."

FabHK 5y ago

Look, if the government tells Apple to do something, then Apple can push back, but then has to do it or pull out of the country. That's the way it was, and is. Now, what has actually changed? The two compelling push-back against a country's demands for more surveillance etc. are: a) it is not technically feasible (eg, wiretapping E2EE chats), and b) it is making the device insecure vis-a-vis hackers and criminals (eg, putting in a backdoor) The interesting question is: Have these defences against government demands been weakened by this new technology? Maybe they have, that would be my gut feeling. But it is not enough to assert it, one must show it.

JohnJamesRambo 5y ago

Tank Man from Tiananmen Square image comes to mind.

ljm 5y ago

> two countries 1. USA 2. UK, Canada, Australia, New Zealand Pick one from the first list, and one from the second list

jrockway 5y ago

Why does the CCP even need to talk to Apple? They have a database of CSAM, and can modify an image in this set to collide with a special image they're looking for on people's phones. They then share their new modified cache of CSAM with other countries ("hey, China is helping us! that's great!") and it gets added to Apple's database for the next iOS release, because it looks to humans like CSAM. Only the CCP knows that it collides with something special they're looking for. Now that we know that collisions are not just easy -- but happen by themselves with no human input (as evidenced by the Imagenet collisions), we know this system can't work. Apple has two remaining safeguards: a second set of hashes (they say), and human reviewers. The human reviewers are likely trained to prefer false-positives when unsure, and so while a thorough human review would clearly indicate "not CSAM" for the images the malicious collisions match, it doesn't feel like much of a safeguard to me. (Remember the leaked memo from the US's organization -- they called our objections "screeching voices". I'm sure the actual reviewers think similarly.) I assume the people reviewing CSAM for the CCP will be in China, so they can be in on the whole scheme. (In the US, we have slightly better checks and balances. Eventually the image will be in front of a court and a jury of your peers, and it's not illegal to have photos that embarrass the CCP, so you'll personally be fine modulo the stress of a criminal investigation. But that dissident in China, probably not going to get a fair trial -- despite the image not being CSAM, it's still illegal. And Apple handed it directly to the government for you.) I don't know, I just find this thing exceedingly worrying. When faced with a government-level adversary, this doesn't sound like a very good system. I think if we're okay with this, we might as well mandate CCTV in everyone's home, so we can catch real child abusers in the act.