5 ms·

I would really like people to start answering this: what exactly do you think has changed? e.g, >That’s very different from authorities taking a sneak peek int

I would really like people to start answering this: what exactly do you think has changed? e.g, >That’s very different from authorities taking a sneak peek into my stuff. To be very blunt: - The opt out of this is to not use iCloud Photos. - If you _currently_ use iCloud Photos, your photos are _already_ hash compared. - Thus the existing opt out is to... not use iCloud Photos. The exact same outcome can happen regardless of whether it's done on or off device. iCloud has _always_ been a known vector for authorities to peek. >I’m big Apple fanboy, but I’m not going to carry a snitch in my pocket. If you use iCloud, you arguably already do.

xuki 5y ago
It makes even less sense, given that they are currently doing this with your iCloud photos. Now they have this tool that can match to a database of photos, how do we know they wouldn't use this to identify non-sexual photos? Maybe Tim Cook wouldn't, what about the next CEO? And the one after that?
- Klonoar 5y ago
  The questions re: what the CEO would sign off on here don't really matter, as the question could apply whether it's server side or client side. It _does_ make sense client side if you view it being done server side as a blocker for E2EE on iCloud. There is absolutely no world where Apple could implement that without keeping the ability to say "yes, we're blocking child porn".
- tialaramex 5y ago
  What makes you think that Apple has a database of actual child sex abuse images? Does that feel like a thing you'd be OK with? "Oh, this is Jim, he's the guy who keeps our archive of sex abuse photographs here at One Infinite Loop" ? If you feel OK with that at Apple, how about at Facebook? Tencent? What about the new ten-person SV start-up would-be Facebook killer whose main founder had a felony conviction in 1996 for violating the Mann Act. Still comfortable? Far more likely Apple takes a bunch of hashes from a third party in the law enforcement side of things (ie cops) and trust that the third party is definitely giving them hashes to protect against the Very Bad Thing that Apple's customers are worried about. Whereupon what you're actually trusting isn't Tim Cook, it's a cop. I'm told there are good cops. Maybe all this is done exclusively by good cops. For now. Now, I don't know about the USA, but around here we don't let cops just snoop about in our stuff, on the off-chance that by doing so they might find kiddie porn. So it should be striking that apparently Apple expects you to be OK with that.
  - megous 5y ago
    Any of these large services allowing user uploaded content can build such a database in a heartbeat. And with a list of known hashes it can even be automated.
coldtea 5y ago
>I would really like people to start answering this: what exactly do you think has changed? e.g, Apple has announced they'll be doing this check? What exactly do you think is the same as before? >The exact same outcome can happen regardless of whether it's done on or off device. iCloud has _always_ been a known vector for authorities to peek. That's neither here, nor there. It's another thing to peak selectively with a warrant of sorts, than to (a) peak automatically in everybody, (b) with a false-positive-prone technique, especially since the mere accusation on a false match can be disastrous for a person, even if they eventually are proven innocent...
- Klonoar 5y ago
  >What exactly do you think is the same as before? The same checking when you synced things to iCloud. As has been repeated over and over again, this check happens for iCloud Photos. It's not running arbitrarily. Your photos were compared before and they're being compared now... if you're using iCloud Photos.
  - coldtea 5y ago
    >The same checking when you synced things to iCloud. As has been repeated over and over again, this check happens for iCloud Photos. It's not running arbitrarily. Who said it's running "arbitrarily"? Who said it's not about iCloud Photos? >Your photos were compared before and they're being compared now... if you're using iCloud Photos. They weren't always compared, they started being compared a few years ago, and they moved to comparing them with a new scheme now. Both are bad, and not the responsibility of a company selling phones - and also a bad precedent (now it's "think of the children", tomorrow "think of the country", then "think of those with wrong ideas", then "think how much money insurance companies can save" and what have you). As for your suggestions to just "stop using iCloud Photos", how about we get to enjoy the features we bought our devices for, without stuff we didn't ask for and don't want?
    - Klonoar 5y ago
      >Both are bad, and not the responsibility of a company selling phones Apple is not just a hardware company and there is no obligation for them to host offending contents on their servers - just as Dropbox, Google, and so on would maintain with theirs. >As for your suggestions to just "stop using iCloud Photos", how about we get to enjoy the features we bought our devices for, without stuff we didn't ask for and don't want? It's odd to say that a business shouldn't be allowed to police what's on their platform, given we're on a forum explicitly enabling entrepreneurs.
Renaud 5y ago
What has changed is the inclusion of spyware technology on the device that can be weaponised to basically report on anything. Today it's only geared toward iCloud and CSAM. How many lines of codes do you think it will take before it scans all your local pictures? How hard do you think it will be for an authoritarian regime like China, that Apple bends over backwards to please, to start including other hashes that are not CSAM? iCloud is opt-out. They can scan server-side like everyone does. Your device is your device, and it now contains, deeply embedded into it, the ability to perform actions that are not under your control and can silently report you directly to the authorities. If you don't see a deep change there, I don't know what to say. I live in a country that is getting more authoritarian by the day, where people are sent to prison (some for life) for criticizing the government, sometime just for chanting or printing a slogan. This is the kind of crap that makes me extremely angry at Apple. Under the guise of something no-one can genuinely be against (think of the children!), they have now included a pretty generic snitch into your phone and made everyone accept it.
- wonnage 5y ago
  We gotta stop with the China bogeyman every time a privacy issue comes up. This is a feature designed by an American company for American government surveillance purposes. China is perfectly capable of doing the same surveillance or worse on its own citizens, with or without Apple. China has nothing to do with why American tech is progressively implementing more authoritarian features in a supposedly democratic country.
  - dannyw 5y ago
    China is just an example. In Australia, the law allows our executive department to order tech companies to build backdoors for the investigation of any crime punishable by more than 2 years imprisonment. We actually had the anti terror department arrest a popular, left-leaning YouTube influencer for harassment while physically assaulting his mum (all on video). That's something that is literally unprecedented in Hong Kong just 3 years ago.
  - bigiain 5y ago
    Ok. How about the Saudi Arabian bogeymen then? Who took Jamal Kashoggi apart with bonesaws as he screamed? Or the Israeli bogeymen who exploited his phone for them? Or the Turkish bogeymen who also a customers of that Israeli phone exploitation company? (Or Facebook who wanted to buy those tools but got turned down, because Facebook is “too far” even for NSO who happily take Saudi and Turkish money?) There are without doubt enough privacy bogeymen to go around, trying to derail a valid argument over its use of the Chinese as the placeholder bogeyman detracts from the discussion pointlessly.
foerbert 5y ago
I think one of the major factors that changes how people perceive this is that it's happen on their own device. If you upload a thing to a server and the server does something... I mean sure. You gave a thing to somebody else, and they did something with it. That's a very understandable and largely accepted situation. This is different. This is your own device doing that thing, out of your control. Alright sure, it's doing the same thing as the other server did and under the same circumstances* so maybe functionally nothing has changed. But the philosophical difference is quite huge between somebody else's server watching over what you upload and your own device doing it. I'm struggling to come up with a good analogy. The closest I can really think of is the difference between a reasonably trusted work friend and your own family member reporting you to the authorities for suspicious behavior in your workplace and home respectively. The end result is the same, but I suspect few people would feel the same about those situations. * There is no inherent limitation for your own device to only be able to check photos you upload to iCloud. There is however such a limitation for the iCloud servers. A very reasonably and potentially functional difference is the ability for this surveillance to be easily expanded beyond iCloud uploads in the future.
matheusmoreira 5y ago
What changed is we are not the masters of our technology anymore. If I tell my computer to do something, it should do it without question. It doesn't matter if it's a crime. The computer is supposed to be my tool and obey my commands. Now what's going to happen instead is the computer will report me to its real masters: corporations, governments. How is this acceptable in any way?
- brokenmachine 5y ago
  Vote with your wallet.
bigiain 5y ago
> The opt out of this is to not use iCloud Photos. Wasn’t yesterday’s version of this sorry about how Apple is implementing this as a client side service on iPhones? https://news.ycombinator.com/item?id=28068741 https://news.ycombinator.com/item?id=28068741 I don’t know if the implication there is “don’t use the stock Apple camera app and photo albums”, or “don’t store any images on yours Phone any more” if they are scanning files from other apps for perceptual hash matches as well…
- Klonoar 5y ago
  ...yes, and the client-side check is only run before syncing to iCloud Photos, which is basically just shifting the hash check from before upload (client side) to after upload (server side).
  - aix1 5y ago
    Thanks for this clarification. This, I think, is an important aspect that seems to often get overlooked. Apple's explanation: <quote> Before an image is stored in iCloud Photos, an on-device matching process is performed for that image against the known CSAM hashes. This matching process is powered by a cryptographic technology called private set intersection, which determines if there is a match without revealing the result. The device creates a cryptographic safety voucher that encodes the match result along with additional encrypted data about the image. This voucher is uploaded to iCloud Photos along with the image. Using another technology called threshold secret sharing, the system ensures the contents of the safety vouchers cannot be interpreted by Apple unless the iCloud Photos account crosses a threshold of known CSAM content. The threshold is set to provide an extremely high level of accuracy and ensures less than a one in one trillion chance per year of incorrectly flagging a given account. </quote> https://www.apple.com/child-safety/ https://www.apple.com/child-safety/
    - bigiain 5y ago
      I'm actually super impressed at the lengths they went to make this as private as they did. With the proviso that you have to trust the code they push onto your device actually does what they claim in the paper. But it does, on the face of it, prevent the "cops adding the latest cop on black person murder viral image to the CSAM content hashes and easily seeing who has it on their device and when it showed up there" concern. They are at least going to need to add enough _other_ hashes to the list to get everybody with their target image over the threshold, then get it past whatever Apple has in place for their manual review and "visual derivatives". And surely that sort of abuse of the system would set off alarms instantly at Apple when a big list of hashes of images that are common enough to all push BLM activists and sympathisers over the CASM count threshold. (I also wonder what those "visual derivatives" they get are, and how well they are going to work to filter out false positives, and how Apple are going to take care of whoever ends up doing that review work. While I have a fairly positive impression of the care and effort Apple put into ensuring my privacy, I have a somewhat less positive impression of how they treat outsourced or low-skill workers. I won't be _too_ surprised to hear the same sort of horror stories about both the work, and the management overseeing the workers that do this sort of job at Facebook... I can't imagine a worse job description than "review images to check if they are real child sexual abuse images", and doing that for minimum wage and no benefits with supervisors threatening to fire you if you take toilet breaks or miss your 150 images an hour targets - is sadly something I totally expect to read about in a year or two's time.)

xuki 5y ago

It makes even less sense, given that they are currently doing this with your iCloud photos. Now they have this tool that can match to a database of photos, how do we know they wouldn't use this to identify non-sexual photos? Maybe Tim Cook wouldn't, what about the next CEO? And the one after that?

Klonoar 5y ago
The questions re: what the CEO would sign off on here don't really matter, as the question could apply whether it's server side or client side. It _does_ make sense client side if you view it being done server side as a blocker for E2EE on iCloud. There is absolutely no world where Apple could implement that without keeping the ability to say "yes, we're blocking child porn".
tialaramex 5y ago
What makes you think that Apple has a database of actual child sex abuse images? Does that feel like a thing you'd be OK with? "Oh, this is Jim, he's the guy who keeps our archive of sex abuse photographs here at One Infinite Loop" ? If you feel OK with that at Apple, how about at Facebook? Tencent? What about the new ten-person SV start-up would-be Facebook killer whose main founder had a felony conviction in 1996 for violating the Mann Act. Still comfortable? Far more likely Apple takes a bunch of hashes from a third party in the law enforcement side of things (ie cops) and trust that the third party is definitely giving them hashes to protect against the Very Bad Thing that Apple's customers are worried about. Whereupon what you're actually trusting isn't Tim Cook, it's a cop. I'm told there are good cops. Maybe all this is done exclusively by good cops. For now. Now, I don't know about the USA, but around here we don't let cops just snoop about in our stuff, on the off-chance that by doing so they might find kiddie porn. So it should be striking that apparently Apple expects you to be OK with that.
- megous 5y ago
  Any of these large services allowing user uploaded content can build such a database in a heartbeat. And with a list of known hashes it can even be automated.

coldtea 5y ago

>I would really like people to start answering this: what exactly do you think has changed? e.g, Apple has announced they'll be doing this check? What exactly do you think is the same as before? >The exact same outcome can happen regardless of whether it's done on or off device. iCloud has _always_ been a known vector for authorities to peek. That's neither here, nor there. It's another thing to peak selectively with a warrant of sorts, than to (a) peak automatically in everybody, (b) with a false-positive-prone technique, especially since the mere accusation on a false match can be disastrous for a person, even if they eventually are proven innocent...

Klonoar 5y ago
>What exactly do you think is the same as before? The same checking when you synced things to iCloud. As has been repeated over and over again, this check happens for iCloud Photos. It's not running arbitrarily. Your photos were compared before and they're being compared now... if you're using iCloud Photos.
- coldtea 5y ago
  >The same checking when you synced things to iCloud. As has been repeated over and over again, this check happens for iCloud Photos. It's not running arbitrarily. Who said it's running "arbitrarily"? Who said it's not about iCloud Photos? >Your photos were compared before and they're being compared now... if you're using iCloud Photos. They weren't always compared, they started being compared a few years ago, and they moved to comparing them with a new scheme now. Both are bad, and not the responsibility of a company selling phones - and also a bad precedent (now it's "think of the children", tomorrow "think of the country", then "think of those with wrong ideas", then "think how much money insurance companies can save" and what have you). As for your suggestions to just "stop using iCloud Photos", how about we get to enjoy the features we bought our devices for, without stuff we didn't ask for and don't want?
  - Klonoar 5y ago
    >Both are bad, and not the responsibility of a company selling phones Apple is not just a hardware company and there is no obligation for them to host offending contents on their servers - just as Dropbox, Google, and so on would maintain with theirs. >As for your suggestions to just "stop using iCloud Photos", how about we get to enjoy the features we bought our devices for, without stuff we didn't ask for and don't want? It's odd to say that a business shouldn't be allowed to police what's on their platform, given we're on a forum explicitly enabling entrepreneurs.

Renaud 5y ago

What has changed is the inclusion of spyware technology on the device that can be weaponised to basically report on anything. Today it's only geared toward iCloud and CSAM. How many lines of codes do you think it will take before it scans all your local pictures? How hard do you think it will be for an authoritarian regime like China, that Apple bends over backwards to please, to start including other hashes that are not CSAM? iCloud is opt-out. They can scan server-side like everyone does. Your device is your device, and it now contains, deeply embedded into it, the ability to perform actions that are not under your control and can silently report you directly to the authorities. If you don't see a deep change there, I don't know what to say. I live in a country that is getting more authoritarian by the day, where people are sent to prison (some for life) for criticizing the government, sometime just for chanting or printing a slogan. This is the kind of crap that makes me extremely angry at Apple. Under the guise of something no-one can genuinely be against (think of the children!), they have now included a pretty generic snitch into your phone and made everyone accept it.

wonnage 5y ago
We gotta stop with the China bogeyman every time a privacy issue comes up. This is a feature designed by an American company for American government surveillance purposes. China is perfectly capable of doing the same surveillance or worse on its own citizens, with or without Apple. China has nothing to do with why American tech is progressively implementing more authoritarian features in a supposedly democratic country.
- dannyw 5y ago
  China is just an example. In Australia, the law allows our executive department to order tech companies to build backdoors for the investigation of any crime punishable by more than 2 years imprisonment. We actually had the anti terror department arrest a popular, left-leaning YouTube influencer for harassment while physically assaulting his mum (all on video). That's something that is literally unprecedented in Hong Kong just 3 years ago.
- bigiain 5y ago
  Ok. How about the Saudi Arabian bogeymen then? Who took Jamal Kashoggi apart with bonesaws as he screamed? Or the Israeli bogeymen who exploited his phone for them? Or the Turkish bogeymen who also a customers of that Israeli phone exploitation company? (Or Facebook who wanted to buy those tools but got turned down, because Facebook is “too far” even for NSO who happily take Saudi and Turkish money?) There are without doubt enough privacy bogeymen to go around, trying to derail a valid argument over its use of the Chinese as the placeholder bogeyman detracts from the discussion pointlessly.

foerbert 5y ago

I think one of the major factors that changes how people perceive this is that it's happen on their own device. If you upload a thing to a server and the server does something... I mean sure. You gave a thing to somebody else, and they did something with it. That's a very understandable and largely accepted situation. This is different. This is your own device doing that thing, out of your control. Alright sure, it's doing the same thing as the other server did and under the same circumstances* so maybe functionally nothing has changed. But the philosophical difference is quite huge between somebody else's server watching over what you upload and your own device doing it. I'm struggling to come up with a good analogy. The closest I can really think of is the difference between a reasonably trusted work friend and your own family member reporting you to the authorities for suspicious behavior in your workplace and home respectively. The end result is the same, but I suspect few people would feel the same about those situations. * There is no inherent limitation for your own device to only be able to check photos you upload to iCloud. There is however such a limitation for the iCloud servers. A very reasonably and potentially functional difference is the ability for this surveillance to be easily expanded beyond iCloud uploads in the future.

matheusmoreira 5y ago

What changed is we are not the masters of our technology anymore. If I tell my computer to do something, it should do it without question. It doesn't matter if it's a crime. The computer is supposed to be my tool and obey my commands. Now what's going to happen instead is the computer will report me to its real masters: corporations, governments. How is this acceptable in any way?

brokenmachine 5y ago
Vote with your wallet.

bigiain 5y ago

> The opt out of this is to not use iCloud Photos. Wasn’t yesterday’s version of this sorry about how Apple is implementing this as a client side service on iPhones? https://news.ycombinator.com/item?id=28068741 https://news.ycombinator.com/item?id=28068741 I don’t know if the implication there is “don’t use the stock Apple camera app and photo albums”, or “don’t store any images on yours Phone any more” if they are scanning files from other apps for perceptual hash matches as well…

Klonoar 5y ago
...yes, and the client-side check is only run before syncing to iCloud Photos, which is basically just shifting the hash check from before upload (client side) to after upload (server side).
- aix1 5y ago
  Thanks for this clarification. This, I think, is an important aspect that seems to often get overlooked. Apple's explanation: <quote> Before an image is stored in iCloud Photos, an on-device matching process is performed for that image against the known CSAM hashes. This matching process is powered by a cryptographic technology called private set intersection, which determines if there is a match without revealing the result. The device creates a cryptographic safety voucher that encodes the match result along with additional encrypted data about the image. This voucher is uploaded to iCloud Photos along with the image. Using another technology called threshold secret sharing, the system ensures the contents of the safety vouchers cannot be interpreted by Apple unless the iCloud Photos account crosses a threshold of known CSAM content. The threshold is set to provide an extremely high level of accuracy and ensures less than a one in one trillion chance per year of incorrectly flagging a given account. </quote> https://www.apple.com/child-safety/ https://www.apple.com/child-safety/
  - bigiain 5y ago
    I'm actually super impressed at the lengths they went to make this as private as they did. With the proviso that you have to trust the code they push onto your device actually does what they claim in the paper. But it does, on the face of it, prevent the "cops adding the latest cop on black person murder viral image to the CSAM content hashes and easily seeing who has it on their device and when it showed up there" concern. They are at least going to need to add enough _other_ hashes to the list to get everybody with their target image over the threshold, then get it past whatever Apple has in place for their manual review and "visual derivatives". And surely that sort of abuse of the system would set off alarms instantly at Apple when a big list of hashes of images that are common enough to all push BLM activists and sympathisers over the CASM count threshold. (I also wonder what those "visual derivatives" they get are, and how well they are going to work to filter out false positives, and how Apple are going to take care of whoever ends up doing that review work. While I have a fairly positive impression of the care and effort Apple put into ensuring my privacy, I have a somewhat less positive impression of how they treat outsourced or low-skill workers. I won't be _too_ surprised to hear the same sort of horror stories about both the work, and the management overseeing the workers that do this sort of job at Facebook... I can't imagine a worse job description than "review images to check if they are real child sexual abuse images", and doing that for minimum wage and no benefits with supervisors threatening to fire you if you take toilet breaks or miss your 150 images an hour targets - is sadly something I totally expect to read about in a year or two's time.)