6 ms·

The practical difference is that with on-device scanning, the system is just a few bit flips away from scanning every photo on your device, instead of just the

The practical difference is that with on-device scanning, the system is just a few bit flips away from scanning every photo on your device, instead of just the ones that are about to be uploaded. With server-side scanning, the separation is clear—what's on Apple's server can be scanned, and what's only on your phone cannot. This all plays so perfectly into my long-time fears about the locked down nature of the iPhone. Sure, it's more secure if Apple is a good steward, but no one outside of Apple can inspect or audit what it's doing!

tmdg 5y ago
I feel like this is a false sense of security. Even before this change, they can easily access and scan photos on your device. If they do any post-processing of the image on device, they already do.
- shuckles 5y ago
  Apple already scans your photos for faces and syncs found faces through iCloud. I’d imagine updating that machine learning model is at least as straightforward as this one.
  - Wowfunhappy 5y ago
    They're searching for different things though. To my knowledge, before now iOS has never scanned for fingerprints of specific photographs. It would be so darn easy to replace the CSAM database with fingerprints of known tiananmen square photos...
    - shuckles 5y ago
      That is a distinction without a difference. I’m sure you could put together quite a good tank man classifier (proof: Google Reverse Image Search works quite well), and it’d catch variations which a perceptual hash wouldn’t. The only difference is intent. The technical risk has not changed at all.
      - dpedu 5y ago
        That is to say face scanning is equally insidious as the new feature?
        shuckles 5y ago
        The technical risk to user privacy - if your threat model is a coerced Apple building surveillance features for nation state actors - is exactly the same between CSAM detection and Photos intelligence which sync results through iCloud. In fact, the latter is more generalizable, has no threshold protections, and so is likely worse.
        int_19h 5y ago
        It's the legal risk that is the biggest problem here. Now that every politician out there knows that this can be done for child porn, there'll be plenty demanding the same for other stuff. And this puts Apple in a rather difficult position, since, with every such demand, they have to either accede, or explain why it's not "important enough" - which then is easily weaponized to bash them. And not just Apple. Once technical feasibility is proven, I can easily see governments mandating this scheme for all devices sold. At that point, it can get even more ugly, since e.g. custom ROMs and such could be seen as a loophole, and cracked down upon.
        shuckles 5y ago
        This hypothetical lacks an explanation for why every politician has not demanded Apple (or say Google) do this scope creep already for photos stored in the cloud where the technical feasibility and legal precedent has already been established by existing CSAM scanning solutions deployed at scale.
        int_19h 5y ago
        I have to note that one of those solutions deployed at scale is Google's. But the big difference is that when those were originally rolled out, they didn't make quite that big of a splash, especially outside of tech circles. I will also note that, while it may be a hypothetical in this particular instance as yet, EU already went from passing a law that allows companies to do something similar voluntarily (previously, they'd be running afoul of privacy regulations), to a proposed bill making it mandatory - in less than a year's time. I don't see why US would be any different in that regard.
    - samstave 5y ago
      So you have User A - they upload a pic with User A and Peoples B,C,D,E,Z icloud scans for those faces finds those faces and ties them to other ID accounts via face - then via fingerprint recognition to a device, and to a location based on IMEI etc. Apple's platform is literally the foundation for the most dystopian digital tool-set in history... Once the government is able to crack the apple war chest, everything is fucked.
      - samstave 5y ago
        --MBS has entered the chat. Go fuck yourself.
  - fraa-orolo 5y ago
    A false positive in matching faces results in a click to fix it or a wrongly categorized photo. A false positive in this new thing may land you in jail or have your life destroyed. Even an allegation of something so heinous is enough to ruin a life. The "one in trillion" chance of false positives is Apple's invention. They haven't scanned trillions of photos and it's a guess. And you need multiple false positives, yet no one says how many, so it could be a low number. Either way, even with how small the chance of it being wrong is, the consequences for the individual are catastrophic. No one sane should accept that kind of risk/reward ratio. "Oh, and one more thing, and we think you'll love it. You can back up your entire camera roll for just $10 a month and a really infinitesimally minuscule chance that you and your family will be completely disgraced in the public eye, and you'll get raped and murdered in prison for nothing."
    - systoll 5y ago
      Ok. So iCloud Photos circa 2020 [and Google Photos and Facebook and Dropbox and OneDrive] aren’t a risk you should be willing to take. This feature doesn’t change anything in that regard; the scanning was already happening.
      - XorNot 5y ago
        I literally do not take that risk in 2021. I do, currently, make the reasoned assurance that the computational overhead of pushing changes down to my phone, and the general international security community, are keeping me approximately abreast of whether my private device is actively spying on me (short answer: it definitely is, longer answer: but to what specific intent?) Apple's new policy is: "of course your phone is scanning and flagging your private files to our server - that's normal behavior! Don't worry about it".
- lifty 5y ago
  It’s not a false sense of security, it’s a clear delimitation between theirs and mine; Debian package maintainers can also slip a scanner on your machine but that is a big line to cross on purpose and without notifying the user.
  - totetsu 5y ago
    But with a debian package you can choose not to accept the upgrade and see any funny business in the release source code..
    - gpm 5y ago
      Somewhere along the line someone is producing and signing the binaries that find their way onto their computer, they could produce those binaries from different source code and I would be none the wiser. Debian tries to be reproducible, so to avoid being caught they might need to control the mirror to so that they could send it to only me. I.e. if I'm lucky it would take a total of 2 people to put malicious binaries on my computer (1 with a signing key, 1 with access to the mirror I download things from).
    - least 5y ago
      That is technically true but in a real very practical sense everyone here using OSS absolutely is trusting a third party because they are not auditing every bit of code they run. For less technical people there is effectively zero difference between open and closed software. It’s really disingenuous to suggest that open source isn’t dependent on trust, you just change who you are trusting. Even if the case is someone else is auditing that code, you’re trusting that person instead of the repository owners. I’ll concede that at least that possibility to audit exists but personally I do have to trust to a certain extent that third parties aren’t trying to fuck me over.
      - totetsu 5y ago
        Thinking about this.. I guess my trust, is that someone smarter than I will notice it, cause a fuss, and the community will raise pitch forks and.. git forks. My trust is in the community, I hope it can stay healthy and diverse for all time.
- stjohnswarts 5y ago
  The only way what you said is not true for any networked device is to just go down to the river and throw it in and never use a digital device again. It's not a false sense of security, it's a calculated position on security and what you will accept, moving spying from the server to the phone was the last straw for a lot of people.
slg 5y ago
There is also an argument that the cloud scanning is a few bit flips away from allowing random Apple employees from looking through all my photos while on device scanning means they have to be specifically looking for certain content. On device scanning is probably preferred if one's fear is someone stealing their nudes, which is one of the more common fears about photo privacy and one that Apple has already shown is a problem for them.
- Wowfunhappy 5y ago
  Sure, but I don't particularly care because I don't upload sensitive photos to iCloud. As I'd recommend to everyone else. If it's on someone else's computer, it's not yours. But, I suppose my iPhone was never really mine either. I knew that, I just never quite put two and two together...
  - slg 5y ago
    >Sure, but I don't particularly care because I don't upload sensitive photos to iCloud So you are saying you have nothing to fear because you aren't hiding anything? Is this a tacit admission that you don't want them scanning your phone for CSAM because they will find something? I'm obviously not seriously accusing you of anything, just pointing out how your line of argument applies equally to privacy whether on the cloud or on your device.
    - Wowfunhappy 5y ago
      > So you are saying you have nothing to fear because you aren't hiding anything? No, I'm saying that if you're hiding something, maybe don't give it to someone else.
      - slg 5y ago
        "I'm not giving my photos to anyone. I am just taking photos on my phone and Apple is automatically backing them up to iCloud. I have no idea how that feature was turned on or how to turn it off." ~ most normal people. People here too often only think of these issues from the perspective of the type of person who browses HN. Apple is thinking of this from the perspective of an average iPhone users.
throaway46546 5y ago
I agree. Our phones tend to hold all of our most intimate data. Violating them is akin to violating our homes. It would be nice if the laws saw it this way.
FooHentai 5y ago
>the system is just a few bit flips away from scanning every photo on your device I prefer to think of it as being just one national security letter away from that happening. Which is of course a schroedinger's cat kinda thing. It's already been sent. Or it hasn't. But why would the national security apparatus not take advantage of this obvious opportunity? Anyone giving benefit of the doubt to this kind of stuff nowadays is IMO very naive or just poorly informed of recent digital history.
- jdavis703 5y ago
  How would the agency issuing an NSL be able to generate a hash of a photo they’re looking for? Presumably if they already had a photo to derive the hash they’d already have whatever it is they’re searching for.
  - gpm 5y ago
    Because you're searching for people, not documents. For a normal law enforcement context, say you bust 1 drug dealer, and are trying to find more. Maybe they have an image of a "contract" (document outlining terms of a drug deal) on their phone. You could find the other parties to that contract by searching everyone's phones for that image. For a national security context, you could imagine equivalents to the above, you could also imagine searching for classified documents that are believed to have been leaked, or maybe searching for documents you stole from an adversary that you believe there spies are likely to be storing on their phones. I'm saying documents here instead of images, but plenty of documents are just images, and I have little doubt that they could get this program to expand to include "scan PDFs, because you could make a CSAM into a pdf" (if it doesn't already?).
  - silverpepsi 5y ago
    I think you already got one great reply, I have just one thing to add to it: your post literally presupposes a 1.0 version of this software that never has its feature set expanded. I don't think that's a reasonable assumption. After all, with 1.0 the goal of catching this class of person is barely achieved. They'll likely arrest people, 99% of whom are just CSAM jpeg collectors who get an extra kick out of viewing a class of image that is super illegal. And nothing more. Then for version 2.0, they'll realize the INTERPOL, FBI, whatever can provide them a combo hash plus one string of text that can nail a known active producer. The holy grail. This small feature add to get so much closer to the original "moral goal" will prove too appealing to pass up. Now all the code is in place for the govt to pass over requests for data pinpointing mere common criminals.
nitrogen 5y ago
With server-side scanning, the separation is clear In all these threads everyone is coming close to the crux of the issue, but I want to restate it in clearer terms: There is a sacrosanct line between "public" and "private," "mine" and "yours." That line cannot be crossed by Western governments without a warrant. Cloud computing has deliberately blurred this line over time. This on-device scanning implementation blows right past that line. Our tools before the computing revolution, and our devices after, become a part of us. Our proprioception extends to include them as part of "self." A personal device -- a tool that should be wholly owned and wholly dependable, like pen and paper -- that betrays its user, is a self that betrays itself.
- N00bN00b 5y ago
  > that betrays its user, is a self that betrays itself. And cloud computing are corporations that take your self away from you. And that was always going to happen. And people have been fighting that idea in the shadows for a while. This issue is just bringing it to the foreground in a most intense way. "Think of the children." They are coming for your self. They will take it. There's not doubt. There never was. Your self is too valuable and you've just been squandering it anyway. Eventually you become Apple. Unless you start resisting, but that's hard. Just slooooowly warm the water. This froggy is fine. Nice and warm. Just don't touch the bottom of the pot.
- inkyoto 5y ago
  > There is a sacrosanct line between "public" and "private," "mine" and "yours." That line cannot be crossed by Western governments without a warrant. This is a self-delusion, I am afraid. The line has been crossed more than once, and it will be crossed again. UK and Australian governments are just two prime examples of waving terrorism and pedobear banners as a pretext to get invasive with each new legislation, and the Oz government already has a new legislation draft to make it a crime to refuse cooperation with law enforcing services when they request access to the encrypted content (think Signal messages). Also, refer to the Witness K case to see how cases that are unfavourable to the standing government completely bypass a «trustworthy» Western judicial system, including the Minister of Justice. CSAM is guaranteed to be abused under whatever new pretext politicians can come up with, and we won't even know that Apple has been quietly subjugated to comply with it in those jurisdictions. There will be even less transparency and even more abuse of CSAM in «non-Western» countries. That is the actual worry.
zepto 5y ago
How is it any more ‘bit flips’ away from scanning every photo on your device than it was before?
- Wowfunhappy 5y ago
  Because if (willBeUploaded) { scanPhoto(); } can become if (true) { scanPhoto(); } Obviously, this is stupidly oversimplified, I have no idea how Apple has structured their code. But the fact of the matter is, if the scanning routine is already on the phone, and the photos are on the phone, all anyone has to do is change which photos get scanned by the routine...
  - jkcxn 5y ago
    By the way they already scan photos that aren’t uploaded to iCloud. I’ve never used iCloud and I can go on the photos app and search for food for example
    - ninkendo 5y ago
      Right, the difference is that now, you can be reported to the authorities for a photo that the CSAM algorithm mistook for child pornography, whereas before the image classifying was purely for your own use.
      - zepto 5y ago
        This is simply a complete falsehood, because photos that are not uploaded are not scanned for CSAM. Photos for your own use that you do not upload to Apple’s cloud are completely unaffected.
    - zepto 5y ago
      Exactly. They already analysed photos on your phone for search. They don’t in fact do CSAM detection on the photos on your phone. But why are people only objecting now when they were already doing on-device analysis? CSAM detection is a complete red herring. It is less of a general purpose scanning mechanism than the search function that was already there.
jonplackett 5y ago
Yes. This is it. When I wrote the above I couldn’t quite put my finger on why I felt like this but this is the reason. It’s the slippery slope of mine / yours.
jasamer 5y ago
You could enable scanning of all files using a few bit-flips, but that doesn't have any effect - the results need to be evaluated by someone. Afaik, the system is designed such that the client side does not know whether a match occurred. So to actually report anyone based on the results of the proposed CSAM scanner, Apple also needs to upload all of the scanned files, including the client-side scan result, to figure out whether there's a match.

tmdg 5y ago

I feel like this is a false sense of security. Even before this change, they can easily access and scan photos on your device. If they do any post-processing of the image on device, they already do.

shuckles 5y ago
Apple already scans your photos for faces and syncs found faces through iCloud. I’d imagine updating that machine learning model is at least as straightforward as this one.
- Wowfunhappy 5y ago
  They're searching for different things though. To my knowledge, before now iOS has never scanned for fingerprints of specific photographs. It would be so darn easy to replace the CSAM database with fingerprints of known tiananmen square photos...
  - shuckles 5y ago
    That is a distinction without a difference. I’m sure you could put together quite a good tank man classifier (proof: Google Reverse Image Search works quite well), and it’d catch variations which a perceptual hash wouldn’t. The only difference is intent. The technical risk has not changed at all.
    - dpedu 5y ago
      That is to say face scanning is equally insidious as the new feature?
      - shuckles 5y ago
        The technical risk to user privacy - if your threat model is a coerced Apple building surveillance features for nation state actors - is exactly the same between CSAM detection and Photos intelligence which sync results through iCloud. In fact, the latter is more generalizable, has no threshold protections, and so is likely worse.
        int_19h 5y ago
        It's the legal risk that is the biggest problem here. Now that every politician out there knows that this can be done for child porn, there'll be plenty demanding the same for other stuff. And this puts Apple in a rather difficult position, since, with every such demand, they have to either accede, or explain why it's not "important enough" - which then is easily weaponized to bash them. And not just Apple. Once technical feasibility is proven, I can easily see governments mandating this scheme for all devices sold. At that point, it can get even more ugly, since e.g. custom ROMs and such could be seen as a loophole, and cracked down upon.
        shuckles 5y ago
        This hypothetical lacks an explanation for why every politician has not demanded Apple (or say Google) do this scope creep already for photos stored in the cloud where the technical feasibility and legal precedent has already been established by existing CSAM scanning solutions deployed at scale.
        int_19h 5y ago
        I have to note that one of those solutions deployed at scale is Google's. But the big difference is that when those were originally rolled out, they didn't make quite that big of a splash, especially outside of tech circles. I will also note that, while it may be a hypothetical in this particular instance as yet, EU already went from passing a law that allows companies to do something similar voluntarily (previously, they'd be running afoul of privacy regulations), to a proposed bill making it mandatory - in less than a year's time. I don't see why US would be any different in that regard.
  - samstave 5y ago
    So you have User A - they upload a pic with User A and Peoples B,C,D,E,Z icloud scans for those faces finds those faces and ties them to other ID accounts via face - then via fingerprint recognition to a device, and to a location based on IMEI etc. Apple's platform is literally the foundation for the most dystopian digital tool-set in history... Once the government is able to crack the apple war chest, everything is fucked.
    - samstave 5y ago
      --MBS has entered the chat. Go fuck yourself.
- fraa-orolo 5y ago
  A false positive in matching faces results in a click to fix it or a wrongly categorized photo. A false positive in this new thing may land you in jail or have your life destroyed. Even an allegation of something so heinous is enough to ruin a life. The "one in trillion" chance of false positives is Apple's invention. They haven't scanned trillions of photos and it's a guess. And you need multiple false positives, yet no one says how many, so it could be a low number. Either way, even with how small the chance of it being wrong is, the consequences for the individual are catastrophic. No one sane should accept that kind of risk/reward ratio. "Oh, and one more thing, and we think you'll love it. You can back up your entire camera roll for just $10 a month and a really infinitesimally minuscule chance that you and your family will be completely disgraced in the public eye, and you'll get raped and murdered in prison for nothing."
  - systoll 5y ago
    Ok. So iCloud Photos circa 2020 [and Google Photos and Facebook and Dropbox and OneDrive] aren’t a risk you should be willing to take. This feature doesn’t change anything in that regard; the scanning was already happening.
    - XorNot 5y ago
      I literally do not take that risk in 2021. I do, currently, make the reasoned assurance that the computational overhead of pushing changes down to my phone, and the general international security community, are keeping me approximately abreast of whether my private device is actively spying on me (short answer: it definitely is, longer answer: but to what specific intent?) Apple's new policy is: "of course your phone is scanning and flagging your private files to our server - that's normal behavior! Don't worry about it".
lifty 5y ago
It’s not a false sense of security, it’s a clear delimitation between theirs and mine; Debian package maintainers can also slip a scanner on your machine but that is a big line to cross on purpose and without notifying the user.
- totetsu 5y ago
  But with a debian package you can choose not to accept the upgrade and see any funny business in the release source code..
  - gpm 5y ago
    Somewhere along the line someone is producing and signing the binaries that find their way onto their computer, they could produce those binaries from different source code and I would be none the wiser. Debian tries to be reproducible, so to avoid being caught they might need to control the mirror to so that they could send it to only me. I.e. if I'm lucky it would take a total of 2 people to put malicious binaries on my computer (1 with a signing key, 1 with access to the mirror I download things from).
  - least 5y ago
    That is technically true but in a real very practical sense everyone here using OSS absolutely is trusting a third party because they are not auditing every bit of code they run. For less technical people there is effectively zero difference between open and closed software. It’s really disingenuous to suggest that open source isn’t dependent on trust, you just change who you are trusting. Even if the case is someone else is auditing that code, you’re trusting that person instead of the repository owners. I’ll concede that at least that possibility to audit exists but personally I do have to trust to a certain extent that third parties aren’t trying to fuck me over.
    - totetsu 5y ago
      Thinking about this.. I guess my trust, is that someone smarter than I will notice it, cause a fuss, and the community will raise pitch forks and.. git forks. My trust is in the community, I hope it can stay healthy and diverse for all time.
stjohnswarts 5y ago
The only way what you said is not true for any networked device is to just go down to the river and throw it in and never use a digital device again. It's not a false sense of security, it's a calculated position on security and what you will accept, moving spying from the server to the phone was the last straw for a lot of people.

slg 5y ago

There is also an argument that the cloud scanning is a few bit flips away from allowing random Apple employees from looking through all my photos while on device scanning means they have to be specifically looking for certain content. On device scanning is probably preferred if one's fear is someone stealing their nudes, which is one of the more common fears about photo privacy and one that Apple has already shown is a problem for them.

Wowfunhappy 5y ago
Sure, but I don't particularly care because I don't upload sensitive photos to iCloud. As I'd recommend to everyone else. If it's on someone else's computer, it's not yours. But, I suppose my iPhone was never really mine either. I knew that, I just never quite put two and two together...
- slg 5y ago
  >Sure, but I don't particularly care because I don't upload sensitive photos to iCloud So you are saying you have nothing to fear because you aren't hiding anything? Is this a tacit admission that you don't want them scanning your phone for CSAM because they will find something? I'm obviously not seriously accusing you of anything, just pointing out how your line of argument applies equally to privacy whether on the cloud or on your device.
  - Wowfunhappy 5y ago
    > So you are saying you have nothing to fear because you aren't hiding anything? No, I'm saying that if you're hiding something, maybe don't give it to someone else.
    - slg 5y ago
      "I'm not giving my photos to anyone. I am just taking photos on my phone and Apple is automatically backing them up to iCloud. I have no idea how that feature was turned on or how to turn it off." ~ most normal people. People here too often only think of these issues from the perspective of the type of person who browses HN. Apple is thinking of this from the perspective of an average iPhone users.

throaway46546 5y ago

I agree. Our phones tend to hold all of our most intimate data. Violating them is akin to violating our homes. It would be nice if the laws saw it this way.

FooHentai 5y ago

>the system is just a few bit flips away from scanning every photo on your device I prefer to think of it as being just one national security letter away from that happening. Which is of course a schroedinger's cat kinda thing. It's already been sent. Or it hasn't. But why would the national security apparatus not take advantage of this obvious opportunity? Anyone giving benefit of the doubt to this kind of stuff nowadays is IMO very naive or just poorly informed of recent digital history.

jdavis703 5y ago
How would the agency issuing an NSL be able to generate a hash of a photo they’re looking for? Presumably if they already had a photo to derive the hash they’d already have whatever it is they’re searching for.
- gpm 5y ago
  Because you're searching for people, not documents. For a normal law enforcement context, say you bust 1 drug dealer, and are trying to find more. Maybe they have an image of a "contract" (document outlining terms of a drug deal) on their phone. You could find the other parties to that contract by searching everyone's phones for that image. For a national security context, you could imagine equivalents to the above, you could also imagine searching for classified documents that are believed to have been leaked, or maybe searching for documents you stole from an adversary that you believe there spies are likely to be storing on their phones. I'm saying documents here instead of images, but plenty of documents are just images, and I have little doubt that they could get this program to expand to include "scan PDFs, because you could make a CSAM into a pdf" (if it doesn't already?).
- silverpepsi 5y ago
  I think you already got one great reply, I have just one thing to add to it: your post literally presupposes a 1.0 version of this software that never has its feature set expanded. I don't think that's a reasonable assumption. After all, with 1.0 the goal of catching this class of person is barely achieved. They'll likely arrest people, 99% of whom are just CSAM jpeg collectors who get an extra kick out of viewing a class of image that is super illegal. And nothing more. Then for version 2.0, they'll realize the INTERPOL, FBI, whatever can provide them a combo hash plus one string of text that can nail a known active producer. The holy grail. This small feature add to get so much closer to the original "moral goal" will prove too appealing to pass up. Now all the code is in place for the govt to pass over requests for data pinpointing mere common criminals.

nitrogen 5y ago

With server-side scanning, the separation is clear In all these threads everyone is coming close to the crux of the issue, but I want to restate it in clearer terms: There is a sacrosanct line between "public" and "private," "mine" and "yours." That line cannot be crossed by Western governments without a warrant. Cloud computing has deliberately blurred this line over time. This on-device scanning implementation blows right past that line. Our tools before the computing revolution, and our devices after, become a part of us. Our proprioception extends to include them as part of "self." A personal device -- a tool that should be wholly owned and wholly dependable, like pen and paper -- that betrays its user, is a self that betrays itself.

N00bN00b 5y ago
> that betrays its user, is a self that betrays itself. And cloud computing are corporations that take your self away from you. And that was always going to happen. And people have been fighting that idea in the shadows for a while. This issue is just bringing it to the foreground in a most intense way. "Think of the children." They are coming for your self. They will take it. There's not doubt. There never was. Your self is too valuable and you've just been squandering it anyway. Eventually you become Apple. Unless you start resisting, but that's hard. Just slooooowly warm the water. This froggy is fine. Nice and warm. Just don't touch the bottom of the pot.
inkyoto 5y ago
> There is a sacrosanct line between "public" and "private," "mine" and "yours." That line cannot be crossed by Western governments without a warrant. This is a self-delusion, I am afraid. The line has been crossed more than once, and it will be crossed again. UK and Australian governments are just two prime examples of waving terrorism and pedobear banners as a pretext to get invasive with each new legislation, and the Oz government already has a new legislation draft to make it a crime to refuse cooperation with law enforcing services when they request access to the encrypted content (think Signal messages). Also, refer to the Witness K case to see how cases that are unfavourable to the standing government completely bypass a «trustworthy» Western judicial system, including the Minister of Justice. CSAM is guaranteed to be abused under whatever new pretext politicians can come up with, and we won't even know that Apple has been quietly subjugated to comply with it in those jurisdictions. There will be even less transparency and even more abuse of CSAM in «non-Western» countries. That is the actual worry.

zepto 5y ago

How is it any more ‘bit flips’ away from scanning every photo on your device than it was before?

Wowfunhappy 5y ago
Because if (willBeUploaded) { scanPhoto(); } can become if (true) { scanPhoto(); } Obviously, this is stupidly oversimplified, I have no idea how Apple has structured their code. But the fact of the matter is, if the scanning routine is already on the phone, and the photos are on the phone, all anyone has to do is change which photos get scanned by the routine...
- jkcxn 5y ago
  By the way they already scan photos that aren’t uploaded to iCloud. I’ve never used iCloud and I can go on the photos app and search for food for example
  - ninkendo 5y ago
    Right, the difference is that now, you can be reported to the authorities for a photo that the CSAM algorithm mistook for child pornography, whereas before the image classifying was purely for your own use.
    - zepto 5y ago
      This is simply a complete falsehood, because photos that are not uploaded are not scanned for CSAM. Photos for your own use that you do not upload to Apple’s cloud are completely unaffected.
  - zepto 5y ago
    Exactly. They already analysed photos on your phone for search. They don’t in fact do CSAM detection on the photos on your phone. But why are people only objecting now when they were already doing on-device analysis? CSAM detection is a complete red herring. It is less of a general purpose scanning mechanism than the search function that was already there.

jonplackett 5y ago

Yes. This is it. When I wrote the above I couldn’t quite put my finger on why I felt like this but this is the reason. It’s the slippery slope of mine / yours.

jasamer 5y ago

You could enable scanning of all files using a few bit-flips, but that doesn't have any effect - the results need to be evaluated by someone. Afaik, the system is designed such that the client side does not know whether a match occurred. So to actually report anyone based on the results of the proposed CSAM scanner, Apple also needs to upload all of the scanned files, including the client-side scan result, to figure out whether there's a match.