15 ms·

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outc

by hughrr 5y ago

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back about how this is good for the world. No way am I going back. Edit: I’m going to reply to some child comments here because I can’t be bothered to reply to each one. This is step one down the stairs to hell. If even one person is caught with this it will be leverage to move to step 2. Within a few years your entire frame buffer and camera will be working against you full time.

user-the-name 5y ago
It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone has, unless they have actual child pornography.
- atrus 5y ago
  I doubt someone who is able/wanting to attack such a system would be unwilling to own some of the material themselves.
- reuben_scratton 5y ago
  "Verified by a human" - and that human will be an overworked, underpaid, overseas subcontractor who may well have an incentive to mash the "Confirm match" button from time to time to improve his performance.
  - short_sells_poo 5y ago
    Beyond that, are we seriously going to ignore the fact that eventually this system will share the private photos of someone's naked kid with some random subcontractor? How is that even remotely OK? Or that it will find and share actual CSAM with said subcontractor?
    - FabHK 5y ago
      It only shares "visual derivatives" of images whose NeuralHash match the NeuralHash of known CSAM (either by being the same image ("perceptually") or a collision).
      - short_sells_poo 5y ago
        That to me just sounds like weasel words to avoid having to say that it shares images. Let's not beat about the bush, the "visual derivative" has to be good enough to identify what's going on in it for the manual confirmation. Are you actually arguing in good faith here at all? Because I can't see how a "visual derivative" that's nevertheless good enough for manual confirmation is any better than the source image?
        cyanite 5y ago
        > Are you actually arguing in good faith here at all? Are you? Because you just seemed to claim that it could match against innocent pictures of your naked children, but this tells me that you don’t understand that this system looks for known pictures, not for something that looks like naked children. Edit: if you do, apologies, but then I’d say that Apple has suggested that it’s a low resolution version of the picture. This should be contrasted with server side scanning, where the server accesses all pictures fully.
    - stetrain 5y ago
      This system does not use ML to find new CSAM images. It only checks for ones already in a known database. Your pictures of kids in the bathtub are not on the list. What is show to the reviewer is a "visual derivative" which hasn't been clearly defined. A thumbnail image? Something with a censored section? We don't really know.
  - apetrovic 5y ago
    I find it very hard to believe that Apple will put literal future of iPhone (just imagine the amount of bad press if one false negative comes out of review process and is reported to the authorities) to "overworked, underpaid overseas subcontractor". Apple is greedy, ruthless, tone-deaf machine, but I don't think they're stupid.
    - notriddle 5y ago
      That’s already how app reviews work.
      - apetrovic 5y ago
        I think rejecting one of million apps and angering one developer is a bit different than reporting someone about a child pornography/abuse, ruining their life and making a global scandal that can put future iPhone sales in jeopardy.
        read_if_gay_ 5y ago
        Barely anyone ever cared about privacy. Convenience, no matter how insignificant, always wins. I will be surprised if this is the last straw that finally gets the masses to care about privacy. In reality there might be a few scandals but at some point no one will care anymore and Apple will remain popular.
  - nomel 5y ago
    The "verified by a human" is only the the last step of the process for Apple. After that, it's given to NCMEC for review, then it's reported to authorities. The Apple subcontractor isn't the only one verifying things here. What you're saying is not realistic.
- hughrr 5y ago
  That assumes that the human review process is competent, your own images aren’t poisoned in some way (consider your own kids in the bath with some noise added) etc. In the mean time they lock your account which means your entire digital life stops dead until their review process is done. No way do I accept any of this. Also the hashes are on the device I understand and it’s not going to be that difficult to extract them.
  - user-the-name 5y ago
    The human reviewer would be able to check against the exact image that generated the hash in the first place. Taking another completely unrelated image and perturbing it would be immediately obvious.
    - hughrr 5y ago
      So there’s an office somewhere with computers full of illegal child porn that people are staring at and comparing your photos to? There’s some irony in that.
      - henrikeh 5y ago
        Yes. That is called NCMEC in the US and it is a core aspect of how this whole process works. If you don’t understand the details of this, I’ll recommend this podcast episode which sums it up and discusses the implications https://atp.fm/443 https://atp.fm/443
        hughrr 5y ago
        I am aware of the details. The episode of Brass Eye comes into context here, the relevant clip being as follows, showing exactly the issues of competence. https://youtu.be/_U-7L1tmBAo https://youtu.be/_U-7L1tmBAo
        henrikeh 5y ago
        I’m sorry, what is your point? I legitimately don’t understand what you mean.
- csande17 5y ago
  > It also requires getting a hold of actual blacklisted hashes, which I doubt anyone has, unless they have actual child pornography. I've never personally seen or looked for any images like this, but if they weren't already proliferating online and widely available to criminals, why would we need to build an elaborate client-side scanning system to detect and report people who have copies of them?
  - user-the-name 5y ago
    Do you have access to darknet child pornography trading networks? I don't.
    - deleted 5y ago
      [deleted]
    - tzs 5y ago
      OK, now I'm curious. How does darknet stuff actually work? Is there some darknet search engine that you access over Tor and type whatever illegal thing you seek such as "child porn" or "stolen credit cards" or "heroin" or "money laundering" into and it points you to providers of those illegal goods and services reachable over some reasonably secure channel? Or is it one of those things where someone already using a particular child porn or stolen card or money laundering or heroin selling site has to put you in contact with them?
      - quenix 5y ago
        Hmm. Tor onion addresses are generally super long and have a large random element to them, such as facebookwkhpilnemxj7asaniu7vnjj.onion where the "Facebook" part is there for convenience and may likely not be present for actual illegal addresses. As such, you probably won't be able to find a CP website unless you actively brute force the entire onion address space. I don't know how the sharing actually happens, but I don't think a real search engine for the dark net exists. There are websites that try (Ahlia, I think?) but they are super limited in scope. I suspect addresses for drugs and CP are shared person-to-person.
      - 5y ago
- nabakin 5y ago
  Agreed. This attack still has consequences for people but they will not run into any legal trouble as far as I'm aware.
- shmatt 5y ago
  the blacklisted hashes are openly given to thousands of companies by a foundation called the National Center for Missing & Exploited Children, in both PhotoDNA and MD5 versions of the hashes. Yes, I can't give you an open link, but I imagine at least hundreds of thousands of people have access to download them. Some companies with better security than others It only takes 1 guy to sell it to an NSO-type company, who would then use it to target attacks for $$$
  - user-the-name 5y ago
    Where would that $$$ come from? Like I said, the only attack possible is to overload the human reviewers, and even that is difficult to pull off except as a coordinated effort by lots of people.
elisbce 5y ago
You are insane. 1. Dumping iOS ecosystem and pick what? You think Android is better and won't have this? iOS is the strongest mobile system in terms of privacy protection available to this date. Hell, the FBI doesn't even need to ask Google to decrypt an Android. 2. Theoretically you can target attacks against anyone. It is just a matter of efforts. If you are a political target, they can already implant spywares around you to track and monitor you. They don't even need to break your phone. 3. If you are not possessing CSAM materials and not one of those targets, then you are not worth the efforts to be attacked or monitored. They don't care. And to be honest, this is the best(might be the only real) way to stay private.
- dang 5y ago
  Your account has been breaking the site guidelines by using HN for political battle and also with personal attacks, like here and https://news.ycombinator.com/item?id=27686351 https://news.ycombinator.com/item?id=27686351. We ban such accounts, so please read and follow the rules if you want to keep posting here: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. (In case anyone is wondering, we don't enforce the rules based on political positions and I have no idea what this account's political positions actually are. It's faster for moderation to skim comments without looking at that.)
nbzso 5y ago
Don't feel bad at all. I dumped macOS entirely from production workflow. I cannot work on computer knowing that something is "scanning" me and I am glad that my "paranoid" feeling stopped me to upgrade all office macs. Billionaires at (Apple) don't give a flying f*ck about users privacy. It is all vertical integration in the name of world domination. How removed from reality they are. This is week after Pegasus/NSO and there is no law who requires them to "scan" on device. So the logical explanation is "growth". Way to go Apple.
- mschuster91 5y ago
  > and there is no law who requires them to "scan" on device Yet. The demands by "concerned parents" (aka fronts for secret services, puritans/other religious fundamentalists and law-and-order hardliners) to "do something against child porn" have grown ever more strong and insane over the last years. (And you can bet that what is used on CSAM will immediately be used to go after legal pornography, sex work, drug enforcement, ...) Many current and powerful politicians don't understand a single bit about computers, to the point of bragging of never having used one (e.g. Japan's cybersecurity minister) or having assistants print out emails daily and transcribing handwritten responses (can't say more than that this is the situation for at least two German MPs) - but there are young politicians who do, and will replace them over the next decade hopefully. That means it's last chance for lobbying groups to get devastating laws passed through, and we must all be vigorous in spotting and preventing at least the worst of them. And what is suspiciously lacking in Apple's response: what are they going to do when they are compelled to extend the CSAM scanner by law in the US, India, China and/or EU? It's feasible for Apple to say they'll just be sticking the middle finger towards markets such as Russia, Saudi-Arabia or similar tiny dictatorships, but the big markets? Apple can't ignore these, and especially India and China are heavyweights.
  - Grustaf 5y ago
    Why would they extend the CSAM scanner? It would be much simpler to just use all the OCR and image classification functions they have already deployed. CSAM scanning is only useful for areas where Apple really doesn't want to even look at the actual material until they are extremely certain that it's a match. If they want to detect anti-government propaganda or something, there would be no such concerns, they would just do a regex search of your inbox. Infinitely more convenient.
vegetablepotpie 5y ago
> I’ve dumped the entire iOS ecosystem in the last week. Not to go off topic from your main point, but what did you move to?
- hughrr 5y ago
  Linux, dumbphone, DSLR. This was the final straw on a planned exit to be honest.
  - rootsudo 5y ago
    That's interesting, I may just do the same thing as well - the camera is the largest thing of why I want to be on an phone. I may just go LineageOS.
    - Sanzig 5y ago
      If you have a Pixel, or are willing to buy one, CalyxOS [1] may be worth a look. It's a privacy-focused ROM that still integrates microG, so it's compatible with most apps (unlike the other popular privacy-focused ROM GrapheneOS [2] which doesn't support microG). The big advantage to CalyxOS or GrapheneOS versus Lineage is they support re-locking the bootloader, which means that verified boot still works - important if your phone gets swiped. [1] https://calyxos.org/ https://calyxos.org/ [2] https://grapheneos.org/ https://grapheneos.org/
      - rootsudo 5y ago
        Thank you for the info, I will go buy and experiment about this.
      - sphinxcdi 5y ago
        > It's a privacy-focused ROM that still integrates microG, so it's compatible with most apps (unlike the other popular privacy-focused ROM GrapheneOS [2] which doesn't support microG). Many apps are also compatible with GrapheneOS and installing https://grapheneos.org/usage#sandboxed-play-services https://grapheneos.org/usage#sandboxed-play-services provides broader app compatibility.
Grustaf 5y ago
Working against you? This technology doesn't make it even an ounce easier for Apple, or some evil three letter agency, to spy on you. If they want to see if you're a Trump supporter, anti-vaxxer or whatever, they already have access to all your photos and emails, on device. They even have OCR and classification of your photos. An intern could add a spying function in an afternoon. if OCR(photo[i]).containsString("MAGA") { reportUser() }
zepto 5y ago
Vindicated? This can’t be used for a targeted attack. It’s no different from the previous false posting making this claim.
- zekrioca 5y ago
  Imagine if WhatsApp and other apps added received photos to iPhone's iCloud Photos gallery by default.. wait, they do.
  - zepto 5y ago
    Imagine if Apps had to ask permission before they could save photos… Imagine if the local photo storage is not the same as iCloud Photo Library… Imagine if anyone who cared could simply switch off iCloud Photo Library…
    - zekrioca 5y ago
      Imagine if our cognitive capability wasn’t exploited.
gjsman-1000 5y ago
It's not really end-game, because the original hashes are, themselves, hashed and not available (so you don't have a hash to work towards). And second, even if you somehow managed to get over that huge leap, raw noise won't pass Apple's review, so you have to reverse-engineer a new image that looks like CSAM, to match a hash you don't have. Big leaps required.
- cyanite 5y ago
  Technical nitpick: the hashes are not hashed (again), but rather cryptographically blinded. This is not reversible, so seems like a hash, but computations can be done on it which result can in some cases be seen by the server. This is actually all very clever, and described in a paper by Apple, linked from the technical summary article.
devwastaken 5y ago
The collisions are supposedly reviewed, my concern is that the process for photodna isn't going to always be the same, and we don't actually have any knowledge as to whether their claims are true. Eventually they will phase out human intervention and replace with gameable AI. 3 letter agencies don't need to review the actual images, they can easily get federal warrants based on some numbers. Apple is content with putting in backdoors. Theres collusion behind the scenes here, they know it's bad, but hey, it's apple, it's ran by manipulators and liars and forms it's own cult.
- atoav 5y ago
  One thing about this is that we are farther and farther away from "if there is evidence for a crime a jury can understand it and rationally decide what do with it" and we are getting closer to "if this lightbulb is glowing the machine says they are guilty, so better trust us". This kind of stuff should not be evidence, if anything it should be a indicator where to look.
  - deleted 5y ago
    [deleted]
  - hannasanarion 5y ago
    In what case do you think the hash will be introduced as evidence, where the actual image being hashed, which is a grey blob, cannot?
argvargc 5y ago
It didn't even last a week. Introducing the people asking us to trust them with the responsibility of mass, automated crime accusations.
- hannasanarion 5y ago
  Why do you think somebody will accuse you of a crime because you have a photo of a grey blob? The real world isn't as stupid as the computer one. The justice system is not deterministic and automatic. Nobody is going to look at this grey blob and go "welp, we have no choice but to throw you in prison forever"
  - argvargc 5y ago
    The real world and the computer world intersect. This is precisely typified by what is being discussed, surely? Apple is trying to automate and computerise a process that was not automated previously, apply it to a huge number of people, and with disastrous potential consequences should their wonderful design be found lacking. And within days, they have already utterly failed to provide one of their own self-stated and incredibly obvious requirements. What else could possibly go wrong? Further, if you think that because the first pre-image failure found (in days...) was a grey blob, that this means all future possible cultivated collisions will only ever be grey blobs, well - good luck with that.
    - hannasanarion 5y ago
      The process of making accusations and arrests is not automatic. The Apple system sends flagged photos to reviewers, and if the reviewers find them suspicious, they send them to the poor souls at NCMEC, who will compare the flagged photo with the original illegal photo that's supposedly a match, and inform law enforcement if they are in fact a match. Nobody will get cops at their door because somebody sent them a grey blob image. The process that's being "automated" is merely an automatic flag that initiates a several-step process of human review. Apple isn't rolling out robocops. Seriously, what are the "disasterous consequences" that you envision? What is the sequence of events where a hash collision leads to any inconvenience whatsoever for a user?
ksec 5y ago
I actually want Apple to stand ground and implement this feature. Like you said the double down on PR and marketing was enough for me. I may not be dumping all iOS and Mac for now. But it was "the" definite signal and evidence this is no longer the old Steve Jobs's Apple. It is like watching Mark Zuckerberg talking about privacy when he doesn't understand anything about it. ( Or more like he has a different understand of privacy than most people ) If they stand ground it makes the decision a lot easier for others. If they dont they will continue to use the boiling frog tactics. After all they dont think they are wrong and they are still righteous. And It would be far more interesting to see how the world folds. The whole computing market, Google, Apple and Microsoft are currently operating with a moat that is literally impenetrable. This will be a small push towards a new world of possibilities. Yes. I really hope they stand ground. I wish I won the lottery I could put a few million into the Social Media echo chamber to support this. Or you know, cough certain Apple competitor could do this.
- deleted 5y ago
  [deleted]
- danuker 5y ago
  Chances are by the next sales report it will be forgotten. Let's see how deep the memory hole goes.
- floatingatoll 5y ago
  Steve Jobs would have implemented this in secret and never told us at all, same as they already did for iCloud photos so many years ago. That would have been a far better approach than today’s Apple is taking. Oh well.
  - dmix 5y ago
    There’s a significant leap from implementing something server side to on the consumer handheld devices themselves. Even if it’s just similar software it is regardless much more serious. I personally thought the unencrypted backups was enough of a death-knell as it provides everything on your phone but anything with real-time on device access is always a gold mine for surveillance hawks.
dgb99 5y ago
can someone upvote me please? I need to add my account to my keybase profile.
robertoandred 5y ago
You feel vindicated because someone made one image look like another?
ballenf 5y ago
Is it theoretically possible that photo filter effect could raise the false positive rate to make the 1 / a trillion number off by several order of magnitude? Could be either malicious or accidental (less likely presumably).
Shorel 5y ago
That's just a checkbox that indicates further action is required. All hashing algorithms have potential collisions. Now, the hashes' database is not public, AFAIK it is in the hands of the FBI.
drvdevd 5y ago
> Within a few years your entire frame buffer and camera will be working against you full time. This. This is hard enough to explain but with all the PR around whether the feature is good or bad it's even more difficult. Putting this functionality on device is equivalent to modifying the hardware directly, because software is eating the world.
spullara 5y ago
What exactly is the targeted attack? To get a bunch of someone's random looking photos reviewed by a human?

by hughrr 5y ago

user-the-name 5y ago

It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone has, unless they have actual child pornography.

atrus 5y ago
I doubt someone who is able/wanting to attack such a system would be unwilling to own some of the material themselves.
reuben_scratton 5y ago
"Verified by a human" - and that human will be an overworked, underpaid, overseas subcontractor who may well have an incentive to mash the "Confirm match" button from time to time to improve his performance.
- short_sells_poo 5y ago
  Beyond that, are we seriously going to ignore the fact that eventually this system will share the private photos of someone's naked kid with some random subcontractor? How is that even remotely OK? Or that it will find and share actual CSAM with said subcontractor?
  - FabHK 5y ago
    It only shares "visual derivatives" of images whose NeuralHash match the NeuralHash of known CSAM (either by being the same image ("perceptually") or a collision).
    - short_sells_poo 5y ago
      That to me just sounds like weasel words to avoid having to say that it shares images. Let's not beat about the bush, the "visual derivative" has to be good enough to identify what's going on in it for the manual confirmation. Are you actually arguing in good faith here at all? Because I can't see how a "visual derivative" that's nevertheless good enough for manual confirmation is any better than the source image?
      - cyanite 5y ago
        > Are you actually arguing in good faith here at all? Are you? Because you just seemed to claim that it could match against innocent pictures of your naked children, but this tells me that you don’t understand that this system looks for known pictures, not for something that looks like naked children. Edit: if you do, apologies, but then I’d say that Apple has suggested that it’s a low resolution version of the picture. This should be contrasted with server side scanning, where the server accesses all pictures fully.
  - stetrain 5y ago
    This system does not use ML to find new CSAM images. It only checks for ones already in a known database. Your pictures of kids in the bathtub are not on the list. What is show to the reviewer is a "visual derivative" which hasn't been clearly defined. A thumbnail image? Something with a censored section? We don't really know.
- apetrovic 5y ago
  I find it very hard to believe that Apple will put literal future of iPhone (just imagine the amount of bad press if one false negative comes out of review process and is reported to the authorities) to "overworked, underpaid overseas subcontractor". Apple is greedy, ruthless, tone-deaf machine, but I don't think they're stupid.
  - notriddle 5y ago
    That’s already how app reviews work.
    - apetrovic 5y ago
      I think rejecting one of million apps and angering one developer is a bit different than reporting someone about a child pornography/abuse, ruining their life and making a global scandal that can put future iPhone sales in jeopardy.
      - read_if_gay_ 5y ago
        Barely anyone ever cared about privacy. Convenience, no matter how insignificant, always wins. I will be surprised if this is the last straw that finally gets the masses to care about privacy. In reality there might be a few scandals but at some point no one will care anymore and Apple will remain popular.
- nomel 5y ago
  The "verified by a human" is only the the last step of the process for Apple. After that, it's given to NCMEC for review, then it's reported to authorities. The Apple subcontractor isn't the only one verifying things here. What you're saying is not realistic.
hughrr 5y ago
That assumes that the human review process is competent, your own images aren’t poisoned in some way (consider your own kids in the bath with some noise added) etc. In the mean time they lock your account which means your entire digital life stops dead until their review process is done. No way do I accept any of this. Also the hashes are on the device I understand and it’s not going to be that difficult to extract them.
- user-the-name 5y ago
  The human reviewer would be able to check against the exact image that generated the hash in the first place. Taking another completely unrelated image and perturbing it would be immediately obvious.
  - hughrr 5y ago
    So there’s an office somewhere with computers full of illegal child porn that people are staring at and comparing your photos to? There’s some irony in that.
    - henrikeh 5y ago
      Yes. That is called NCMEC in the US and it is a core aspect of how this whole process works. If you don’t understand the details of this, I’ll recommend this podcast episode which sums it up and discusses the implications https://atp.fm/443 https://atp.fm/443
      - hughrr 5y ago
        I am aware of the details. The episode of Brass Eye comes into context here, the relevant clip being as follows, showing exactly the issues of competence. https://youtu.be/_U-7L1tmBAo https://youtu.be/_U-7L1tmBAo
        henrikeh 5y ago
        I’m sorry, what is your point? I legitimately don’t understand what you mean.
csande17 5y ago
> It also requires getting a hold of actual blacklisted hashes, which I doubt anyone has, unless they have actual child pornography. I've never personally seen or looked for any images like this, but if they weren't already proliferating online and widely available to criminals, why would we need to build an elaborate client-side scanning system to detect and report people who have copies of them?
- user-the-name 5y ago
  Do you have access to darknet child pornography trading networks? I don't.
  - deleted 5y ago
    [deleted]
  - tzs 5y ago
    OK, now I'm curious. How does darknet stuff actually work? Is there some darknet search engine that you access over Tor and type whatever illegal thing you seek such as "child porn" or "stolen credit cards" or "heroin" or "money laundering" into and it points you to providers of those illegal goods and services reachable over some reasonably secure channel? Or is it one of those things where someone already using a particular child porn or stolen card or money laundering or heroin selling site has to put you in contact with them?
    - quenix 5y ago
      Hmm. Tor onion addresses are generally super long and have a large random element to them, such as facebookwkhpilnemxj7asaniu7vnjj.onion where the "Facebook" part is there for convenience and may likely not be present for actual illegal addresses. As such, you probably won't be able to find a CP website unless you actively brute force the entire onion address space. I don't know how the sharing actually happens, but I don't think a real search engine for the dark net exists. There are websites that try (Ahlia, I think?) but they are super limited in scope. I suspect addresses for drugs and CP are shared person-to-person.
    - 5y ago
nabakin 5y ago
Agreed. This attack still has consequences for people but they will not run into any legal trouble as far as I'm aware.
shmatt 5y ago
the blacklisted hashes are openly given to thousands of companies by a foundation called the National Center for Missing & Exploited Children, in both PhotoDNA and MD5 versions of the hashes. Yes, I can't give you an open link, but I imagine at least hundreds of thousands of people have access to download them. Some companies with better security than others It only takes 1 guy to sell it to an NSO-type company, who would then use it to target attacks for $$$
- user-the-name 5y ago
  Where would that $$$ come from? Like I said, the only attack possible is to overload the human reviewers, and even that is difficult to pull off except as a coordinated effort by lots of people.

elisbce 5y ago

You are insane. 1. Dumping iOS ecosystem and pick what? You think Android is better and won't have this? iOS is the strongest mobile system in terms of privacy protection available to this date. Hell, the FBI doesn't even need to ask Google to decrypt an Android. 2. Theoretically you can target attacks against anyone. It is just a matter of efforts. If you are a political target, they can already implant spywares around you to track and monitor you. They don't even need to break your phone. 3. If you are not possessing CSAM materials and not one of those targets, then you are not worth the efforts to be attacked or monitored. They don't care. And to be honest, this is the best(might be the only real) way to stay private.

dang 5y ago
Your account has been breaking the site guidelines by using HN for political battle and also with personal attacks, like here and https://news.ycombinator.com/item?id=27686351 https://news.ycombinator.com/item?id=27686351. We ban such accounts, so please read and follow the rules if you want to keep posting here: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. (In case anyone is wondering, we don't enforce the rules based on political positions and I have no idea what this account's political positions actually are. It's faster for moderation to skim comments without looking at that.)

nbzso 5y ago

Don't feel bad at all. I dumped macOS entirely from production workflow. I cannot work on computer knowing that something is "scanning" me and I am glad that my "paranoid" feeling stopped me to upgrade all office macs. Billionaires at (Apple) don't give a flying f*ck about users privacy. It is all vertical integration in the name of world domination. How removed from reality they are. This is week after Pegasus/NSO and there is no law who requires them to "scan" on device. So the logical explanation is "growth". Way to go Apple.

mschuster91 5y ago
> and there is no law who requires them to "scan" on device Yet. The demands by "concerned parents" (aka fronts for secret services, puritans/other religious fundamentalists and law-and-order hardliners) to "do something against child porn" have grown ever more strong and insane over the last years. (And you can bet that what is used on CSAM will immediately be used to go after legal pornography, sex work, drug enforcement, ...) Many current and powerful politicians don't understand a single bit about computers, to the point of bragging of never having used one (e.g. Japan's cybersecurity minister) or having assistants print out emails daily and transcribing handwritten responses (can't say more than that this is the situation for at least two German MPs) - but there are young politicians who do, and will replace them over the next decade hopefully. That means it's last chance for lobbying groups to get devastating laws passed through, and we must all be vigorous in spotting and preventing at least the worst of them. And what is suspiciously lacking in Apple's response: what are they going to do when they are compelled to extend the CSAM scanner by law in the US, India, China and/or EU? It's feasible for Apple to say they'll just be sticking the middle finger towards markets such as Russia, Saudi-Arabia or similar tiny dictatorships, but the big markets? Apple can't ignore these, and especially India and China are heavyweights.
- Grustaf 5y ago
  Why would they extend the CSAM scanner? It would be much simpler to just use all the OCR and image classification functions they have already deployed. CSAM scanning is only useful for areas where Apple really doesn't want to even look at the actual material until they are extremely certain that it's a match. If they want to detect anti-government propaganda or something, there would be no such concerns, they would just do a regex search of your inbox. Infinitely more convenient.

vegetablepotpie 5y ago

> I’ve dumped the entire iOS ecosystem in the last week. Not to go off topic from your main point, but what did you move to?

hughrr 5y ago
Linux, dumbphone, DSLR. This was the final straw on a planned exit to be honest.
- rootsudo 5y ago
  That's interesting, I may just do the same thing as well - the camera is the largest thing of why I want to be on an phone. I may just go LineageOS.
  - Sanzig 5y ago
    If you have a Pixel, or are willing to buy one, CalyxOS [1] may be worth a look. It's a privacy-focused ROM that still integrates microG, so it's compatible with most apps (unlike the other popular privacy-focused ROM GrapheneOS [2] which doesn't support microG). The big advantage to CalyxOS or GrapheneOS versus Lineage is they support re-locking the bootloader, which means that verified boot still works - important if your phone gets swiped. [1] https://calyxos.org/ https://calyxos.org/ [2] https://grapheneos.org/ https://grapheneos.org/
    - rootsudo 5y ago
      Thank you for the info, I will go buy and experiment about this.
    - sphinxcdi 5y ago
      > It's a privacy-focused ROM that still integrates microG, so it's compatible with most apps (unlike the other popular privacy-focused ROM GrapheneOS [2] which doesn't support microG). Many apps are also compatible with GrapheneOS and installing https://grapheneos.org/usage#sandboxed-play-services https://grapheneos.org/usage#sandboxed-play-services provides broader app compatibility.

Grustaf 5y ago

Working against you? This technology doesn't make it even an ounce easier for Apple, or some evil three letter agency, to spy on you. If they want to see if you're a Trump supporter, anti-vaxxer or whatever, they already have access to all your photos and emails, on device. They even have OCR and classification of your photos. An intern could add a spying function in an afternoon. if OCR(photo[i]).containsString("MAGA") { reportUser() }

zepto 5y ago

Vindicated? This can’t be used for a targeted attack. It’s no different from the previous false posting making this claim.

zekrioca 5y ago
Imagine if WhatsApp and other apps added received photos to iPhone's iCloud Photos gallery by default.. wait, they do.
- zepto 5y ago
  Imagine if Apps had to ask permission before they could save photos… Imagine if the local photo storage is not the same as iCloud Photo Library… Imagine if anyone who cared could simply switch off iCloud Photo Library…
  - zekrioca 5y ago
    Imagine if our cognitive capability wasn’t exploited.

gjsman-1000 5y ago

It's not really end-game, because the original hashes are, themselves, hashed and not available (so you don't have a hash to work towards). And second, even if you somehow managed to get over that huge leap, raw noise won't pass Apple's review, so you have to reverse-engineer a new image that looks like CSAM, to match a hash you don't have. Big leaps required.

cyanite 5y ago
Technical nitpick: the hashes are not hashed (again), but rather cryptographically blinded. This is not reversible, so seems like a hash, but computations can be done on it which result can in some cases be seen by the server. This is actually all very clever, and described in a paper by Apple, linked from the technical summary article.

devwastaken 5y ago

The collisions are supposedly reviewed, my concern is that the process for photodna isn't going to always be the same, and we don't actually have any knowledge as to whether their claims are true. Eventually they will phase out human intervention and replace with gameable AI. 3 letter agencies don't need to review the actual images, they can easily get federal warrants based on some numbers. Apple is content with putting in backdoors. Theres collusion behind the scenes here, they know it's bad, but hey, it's apple, it's ran by manipulators and liars and forms it's own cult.

atoav 5y ago
One thing about this is that we are farther and farther away from "if there is evidence for a crime a jury can understand it and rationally decide what do with it" and we are getting closer to "if this lightbulb is glowing the machine says they are guilty, so better trust us". This kind of stuff should not be evidence, if anything it should be a indicator where to look.
- deleted 5y ago
  [deleted]
- hannasanarion 5y ago
  In what case do you think the hash will be introduced as evidence, where the actual image being hashed, which is a grey blob, cannot?

argvargc 5y ago

It didn't even last a week. Introducing the people asking us to trust them with the responsibility of mass, automated crime accusations.

hannasanarion 5y ago
Why do you think somebody will accuse you of a crime because you have a photo of a grey blob? The real world isn't as stupid as the computer one. The justice system is not deterministic and automatic. Nobody is going to look at this grey blob and go "welp, we have no choice but to throw you in prison forever"
- argvargc 5y ago
  The real world and the computer world intersect. This is precisely typified by what is being discussed, surely? Apple is trying to automate and computerise a process that was not automated previously, apply it to a huge number of people, and with disastrous potential consequences should their wonderful design be found lacking. And within days, they have already utterly failed to provide one of their own self-stated and incredibly obvious requirements. What else could possibly go wrong? Further, if you think that because the first pre-image failure found (in days...) was a grey blob, that this means all future possible cultivated collisions will only ever be grey blobs, well - good luck with that.
  - hannasanarion 5y ago
    The process of making accusations and arrests is not automatic. The Apple system sends flagged photos to reviewers, and if the reviewers find them suspicious, they send them to the poor souls at NCMEC, who will compare the flagged photo with the original illegal photo that's supposedly a match, and inform law enforcement if they are in fact a match. Nobody will get cops at their door because somebody sent them a grey blob image. The process that's being "automated" is merely an automatic flag that initiates a several-step process of human review. Apple isn't rolling out robocops. Seriously, what are the "disasterous consequences" that you envision? What is the sequence of events where a hash collision leads to any inconvenience whatsoever for a user?

ksec 5y ago

I actually want Apple to stand ground and implement this feature. Like you said the double down on PR and marketing was enough for me. I may not be dumping all iOS and Mac for now. But it was "the" definite signal and evidence this is no longer the old Steve Jobs's Apple. It is like watching Mark Zuckerberg talking about privacy when he doesn't understand anything about it. ( Or more like he has a different understand of privacy than most people ) If they stand ground it makes the decision a lot easier for others. If they dont they will continue to use the boiling frog tactics. After all they dont think they are wrong and they are still righteous. And It would be far more interesting to see how the world folds. The whole computing market, Google, Apple and Microsoft are currently operating with a moat that is literally impenetrable. This will be a small push towards a new world of possibilities. Yes. I really hope they stand ground. I wish I won the lottery I could put a few million into the Social Media echo chamber to support this. Or you know, cough certain Apple competitor could do this.

deleted 5y ago
[deleted]
danuker 5y ago
Chances are by the next sales report it will be forgotten. Let's see how deep the memory hole goes.
floatingatoll 5y ago
Steve Jobs would have implemented this in secret and never told us at all, same as they already did for iCloud photos so many years ago. That would have been a far better approach than today’s Apple is taking. Oh well.
- dmix 5y ago
  There’s a significant leap from implementing something server side to on the consumer handheld devices themselves. Even if it’s just similar software it is regardless much more serious. I personally thought the unencrypted backups was enough of a death-knell as it provides everything on your phone but anything with real-time on device access is always a gold mine for surveillance hawks.

dgb99 5y ago

can someone upvote me please? I need to add my account to my keybase profile.

robertoandred 5y ago

You feel vindicated because someone made one image look like another?

ballenf 5y ago

Is it theoretically possible that photo filter effect could raise the false positive rate to make the 1 / a trillion number off by several order of magnitude? Could be either malicious or accidental (less likely presumably).

Shorel 5y ago

That's just a checkbox that indicates further action is required. All hashing algorithms have potential collisions. Now, the hashes' database is not public, AFAIK it is in the hands of the FBI.

drvdevd 5y ago

> Within a few years your entire frame buffer and camera will be working against you full time. This. This is hard enough to explain but with all the PR around whether the feature is good or bad it's even more difficult. Putting this functionality on device is equivalent to modifying the hardware directly, because software is eating the world.

spullara 5y ago

What exactly is the targeted attack? To get a bunch of someone's random looking photos reviewed by a human?