4 ms·

>What exactly do you think is the same as before? The same checking when you synced things to iCloud. As has been repeated over and over again, this check happ

>What exactly do you think is the same as before? The same checking when you synced things to iCloud. As has been repeated over and over again, this check happens for iCloud Photos. It's not running arbitrarily. Your photos were compared before and they're being compared now... if you're using iCloud Photos.

coldtea 5y ago
>The same checking when you synced things to iCloud. As has been repeated over and over again, this check happens for iCloud Photos. It's not running arbitrarily. Who said it's running "arbitrarily"? Who said it's not about iCloud Photos? >Your photos were compared before and they're being compared now... if you're using iCloud Photos. They weren't always compared, they started being compared a few years ago, and they moved to comparing them with a new scheme now. Both are bad, and not the responsibility of a company selling phones - and also a bad precedent (now it's "think of the children", tomorrow "think of the country", then "think of those with wrong ideas", then "think how much money insurance companies can save" and what have you). As for your suggestions to just "stop using iCloud Photos", how about we get to enjoy the features we bought our devices for, without stuff we didn't ask for and don't want?
- Klonoar 5y ago
  >Both are bad, and not the responsibility of a company selling phones Apple is not just a hardware company and there is no obligation for them to host offending contents on their servers - just as Dropbox, Google, and so on would maintain with theirs. >As for your suggestions to just "stop using iCloud Photos", how about we get to enjoy the features we bought our devices for, without stuff we didn't ask for and don't want? It's odd to say that a business shouldn't be allowed to police what's on their platform, given we're on a forum explicitly enabling entrepreneurs.
  - macintux 5y ago
    In this case, they're explicitly required by law to report this material if it shows up on their servers.
    - coldtea 5y ago
      Well, Jim Crow legislation was also a thing once.
      - macintux 5y ago
        This definitely feels like a bad solution provoked by a dubious law; the complaints should be directed at our elected officials, not Apple.
  - visarga 5y ago
    With many more images, many more false positives. One has as a consequence a message or account being deleted, the other - being reported to the police. Very different!
    - jeromegv 5y ago
      They were reporting to the authorities before as well with what was found on iCloud photos.
  - coldtea 5y ago
    >It's odd to say that a business shouldn't be allowed to police what's on their platform, given we're on a forum explicitly enabling entrepreneurs. It's odd to say that a business should be allowed to police private user content, given we're on a forum with the name "Hacker" on it, built by ex-hackers, and with part of its member's interests heritage not in and "enabling enterpreneurs" but in hacking (in the MIT sense of yore).
stetrain 5y ago
The post office scans your mail through various machines in transit. We accept that when we put the mail in the mailbox. What if the post office announced they were installing a man with a scanning machine in your home who would scan your letters before they left your house? It's the same outcome. The same process. Just inside your house instead of out in the mail system. They're exactly the same, except somehow it's not.
- Klonoar 5y ago
  This example changes with regards to emotional weight if you remove "a man" and leave it at just "a scanning machine". There is no human scanning your photos on an iPhone, so let's compares apples to apples here. If that scanning machine didn't reveal the contents of my mail, and then ensured that it wasn't able to be given out in-transit? Yeah, I'd potentially be fine with it - but I'll leave this answer as a hypothetical since it's all theory anyway. The point here is that you're choosing to use the mail system and you're thus choosing to play by those rules. Given that these checks happen for iCloud you're effectively making the same agreement.
  - dannyw 5y ago
    There actually is a man involved: enough similarities and a human will review the photos. Every algorithm, especially perceptual hashing, will have false positives, and at Apple's scale, some people's private and intimate photos will be false positives and be exposed to a man looking at it.
    - visarga 5y ago
      > some people's private and intimate photos will be false positives and be exposed to a man looking at it and deciding who gets reported to police based on their cultural views on nudity
    - jeromegv 5y ago
      I think the point of the OP is that it was already the case before when you were using iCloud photos. The scan was server side.
visarga 5y ago
It's not the same because before the hashing was done in the cloud, but now the model is accessible locally, you just need to take pictures. This means it's easier to hack. If someone discovers a way to reliably generate adversarial images they can send such images to someone else to iSWAT them.
- Klonoar 5y ago
  If your definition of "hack" is "get bob to accept bad file", no, this model is not easier - it's just different. You could literally piggyback on the directories that Macs use to sync to iCloud Drive, get an image in there, and then it gets scanned by iCloud. This is not some new theoretical attack - and in fact, this would be the "hack" for the new one as well since it requires iCloud sync to trigger anyway.
  - visarga 5y ago
    I was referring to a "white box" adversarial attack, meaning you can try variations of the input to see how the outcome changes, and then construct an input that will fool the system. That's only possible if you have access to the perceptual hashing model, which you do now since it runs locally. Of course generating an adversarial image is not the final step, the hacker still need to place it in the victim's account somehow, but it's easier now because the image looks legit.

coldtea 5y ago

>The same checking when you synced things to iCloud. As has been repeated over and over again, this check happens for iCloud Photos. It's not running arbitrarily. Who said it's running "arbitrarily"? Who said it's not about iCloud Photos? >Your photos were compared before and they're being compared now... if you're using iCloud Photos. They weren't always compared, they started being compared a few years ago, and they moved to comparing them with a new scheme now. Both are bad, and not the responsibility of a company selling phones - and also a bad precedent (now it's "think of the children", tomorrow "think of the country", then "think of those with wrong ideas", then "think how much money insurance companies can save" and what have you). As for your suggestions to just "stop using iCloud Photos", how about we get to enjoy the features we bought our devices for, without stuff we didn't ask for and don't want?

Klonoar 5y ago
>Both are bad, and not the responsibility of a company selling phones Apple is not just a hardware company and there is no obligation for them to host offending contents on their servers - just as Dropbox, Google, and so on would maintain with theirs. >As for your suggestions to just "stop using iCloud Photos", how about we get to enjoy the features we bought our devices for, without stuff we didn't ask for and don't want? It's odd to say that a business shouldn't be allowed to police what's on their platform, given we're on a forum explicitly enabling entrepreneurs.
- macintux 5y ago
  In this case, they're explicitly required by law to report this material if it shows up on their servers.
  - coldtea 5y ago
    Well, Jim Crow legislation was also a thing once.
    - macintux 5y ago
      This definitely feels like a bad solution provoked by a dubious law; the complaints should be directed at our elected officials, not Apple.
- visarga 5y ago
  With many more images, many more false positives. One has as a consequence a message or account being deleted, the other - being reported to the police. Very different!
  - jeromegv 5y ago
    They were reporting to the authorities before as well with what was found on iCloud photos.
- coldtea 5y ago
  >It's odd to say that a business shouldn't be allowed to police what's on their platform, given we're on a forum explicitly enabling entrepreneurs. It's odd to say that a business should be allowed to police private user content, given we're on a forum with the name "Hacker" on it, built by ex-hackers, and with part of its member's interests heritage not in and "enabling enterpreneurs" but in hacking (in the MIT sense of yore).

stetrain 5y ago

The post office scans your mail through various machines in transit. We accept that when we put the mail in the mailbox. What if the post office announced they were installing a man with a scanning machine in your home who would scan your letters before they left your house? It's the same outcome. The same process. Just inside your house instead of out in the mail system. They're exactly the same, except somehow it's not.

Klonoar 5y ago
This example changes with regards to emotional weight if you remove "a man" and leave it at just "a scanning machine". There is no human scanning your photos on an iPhone, so let's compares apples to apples here. If that scanning machine didn't reveal the contents of my mail, and then ensured that it wasn't able to be given out in-transit? Yeah, I'd potentially be fine with it - but I'll leave this answer as a hypothetical since it's all theory anyway. The point here is that you're choosing to use the mail system and you're thus choosing to play by those rules. Given that these checks happen for iCloud you're effectively making the same agreement.
- dannyw 5y ago
  There actually is a man involved: enough similarities and a human will review the photos. Every algorithm, especially perceptual hashing, will have false positives, and at Apple's scale, some people's private and intimate photos will be false positives and be exposed to a man looking at it.
  - visarga 5y ago
    > some people's private and intimate photos will be false positives and be exposed to a man looking at it and deciding who gets reported to police based on their cultural views on nudity
  - jeromegv 5y ago
    I think the point of the OP is that it was already the case before when you were using iCloud photos. The scan was server side.

visarga 5y ago

It's not the same because before the hashing was done in the cloud, but now the model is accessible locally, you just need to take pictures. This means it's easier to hack. If someone discovers a way to reliably generate adversarial images they can send such images to someone else to iSWAT them.

Klonoar 5y ago
If your definition of "hack" is "get bob to accept bad file", no, this model is not easier - it's just different. You could literally piggyback on the directories that Macs use to sync to iCloud Drive, get an image in there, and then it gets scanned by iCloud. This is not some new theoretical attack - and in fact, this would be the "hack" for the new one as well since it requires iCloud sync to trigger anyway.
- visarga 5y ago
  I was referring to a "white box" adversarial attack, meaning you can try variations of the input to see how the outcome changes, and then construct an input that will fool the system. That's only possible if you have access to the perceptual hashing model, which you do now since it runs locally. Of course generating an adversarial image is not the final step, the hacker still need to place it in the victim's account somehow, but it's easier now because the image looks legit.