4 ms·

In theory they are simply matching against a known database of existing images, and as such your baby/family images will not be a match - they are personal imag

In theory they are simply matching against a known database of existing images, and as such your baby/family images will not be a match - they are personal images that have not been catalogued elsewhere. There is also meant to be human verification of any flagged images. In practice ? (Edit: This is for the iCloud scanning. The (i)Message warning around sexting is a whole other issue)

tjpnz 5y ago
>There is also meant to be human verification of any flagged images. That shouldn't make anyone sleep soundly at night. The person at the other end has no incentive to not turn your life upside down.
lm28469 5y ago
> There is also meant to be human verification of any flagged images. "Rest assured our system is safe, a random guy working in a digital sweatshop at the end of the world will get to see your naked baby pictures just to make sure you're not a criminal"
- mobiuscog 5y ago
  This is definitely an 'added' risk they need to qualify - how they will stop any verification from leaking private images. One would hope that such checks should be minimal and therefore performed by suitably qualified (and equally audited) personnel. As you allude, it is more likely that such work is outsourced in the same way that the manufacturing of the phone is.
  - samrolken 5y ago
    Apple never gets to see your photo. Even if the photo matches the database and even if you have enough photos matched to reach the threshold that lets Apple decrypt the hashes, the photo itself is not decrypted or sent to anyone as part of this process. Only the related metadata.
    - rubatuga 5y ago
      Source?
  - jbarrs 5y ago
    Amazon in the pursuit of Alexa quality control has most recordings manually reviewed by real people to see whether the response was appropriate. This includes any accidental triggers. It probably does not surprise that these devices have recorded sensitive information like bank details and passwords, or have recorded instances of domestic abuse which employees have been forced to listen to [1]. It absolutely wouldn't surprise me to learn that Apple will have a team of people reviewing everything that gets flagged, who will get to see everyone's private photos and also probably be exposed to real CSAM. [1] https://www.theguardian.com/technology/2019/apr/11/amazon-staff-listen-to-customers-alexa-recordings-report-says https://www.theguardian.com/technology/2019/apr/11/amazon-st...
- rich_sasha 5y ago
  Jeepers good point, actually this went from an inconvenience to a total show stopper for me. What will the job interview sound like for this job? “Do you have any experience in recognising child porn?”, “yes, many years, and great enthusiasm for the job”. I’m looking for my old Huawei phone now.
Popegaf 5y ago
> There is also meant to be human verification of any flagged images. So Apple does have access to everything you store on your phone? Either they're lying about being concerned about privacy or you don't have your facts straight.
- TazeTSchnitzel 5y ago
  The only images scanned are ones uploaded to iCloud photos, so these are images Apple already has.
pjerem 5y ago
Wait, if the Apple algorithm flags a photo of my kid in his bath, some unknown human is going to check the picture for verification purpose ? That's even more cringe.
- samrolken 5y ago
  Your kid in the bath won’t be in the database being matched against.
  - rich_sasha 5y ago
    If it’s a fixed db, not an “artificial sort-of intelligence”. But that’s a stupid way around it, surely. Child porn is surely, and tragically, in continuous production. And really, it’s the production that is the biggest issue, this is the continued abuse of children. And that’s the one bit the algo won’t match against?
    - samrolken 5y ago
      Apple has documentation explaining how it all works with a database of known images.
  - fragileone 5y ago
    Out of 200k photos being matched against I'd be extremely surprised if there aren't at least some false positives generated.

tjpnz 5y ago

>There is also meant to be human verification of any flagged images. That shouldn't make anyone sleep soundly at night. The person at the other end has no incentive to not turn your life upside down.

lm28469 5y ago

> There is also meant to be human verification of any flagged images. "Rest assured our system is safe, a random guy working in a digital sweatshop at the end of the world will get to see your naked baby pictures just to make sure you're not a criminal"

mobiuscog 5y ago
This is definitely an 'added' risk they need to qualify - how they will stop any verification from leaking private images. One would hope that such checks should be minimal and therefore performed by suitably qualified (and equally audited) personnel. As you allude, it is more likely that such work is outsourced in the same way that the manufacturing of the phone is.
- samrolken 5y ago
  Apple never gets to see your photo. Even if the photo matches the database and even if you have enough photos matched to reach the threshold that lets Apple decrypt the hashes, the photo itself is not decrypted or sent to anyone as part of this process. Only the related metadata.
  - rubatuga 5y ago
    Source?
- jbarrs 5y ago
  Amazon in the pursuit of Alexa quality control has most recordings manually reviewed by real people to see whether the response was appropriate. This includes any accidental triggers. It probably does not surprise that these devices have recorded sensitive information like bank details and passwords, or have recorded instances of domestic abuse which employees have been forced to listen to [1]. It absolutely wouldn't surprise me to learn that Apple will have a team of people reviewing everything that gets flagged, who will get to see everyone's private photos and also probably be exposed to real CSAM. [1] https://www.theguardian.com/technology/2019/apr/11/amazon-staff-listen-to-customers-alexa-recordings-report-says https://www.theguardian.com/technology/2019/apr/11/amazon-st...
rich_sasha 5y ago
Jeepers good point, actually this went from an inconvenience to a total show stopper for me. What will the job interview sound like for this job? “Do you have any experience in recognising child porn?”, “yes, many years, and great enthusiasm for the job”. I’m looking for my old Huawei phone now.

Popegaf 5y ago

> There is also meant to be human verification of any flagged images. So Apple does have access to everything you store on your phone? Either they're lying about being concerned about privacy or you don't have your facts straight.

TazeTSchnitzel 5y ago
The only images scanned are ones uploaded to iCloud photos, so these are images Apple already has.

pjerem 5y ago

Wait, if the Apple algorithm flags a photo of my kid in his bath, some unknown human is going to check the picture for verification purpose ? That's even more cringe.

samrolken 5y ago
Your kid in the bath won’t be in the database being matched against.
- rich_sasha 5y ago
  If it’s a fixed db, not an “artificial sort-of intelligence”. But that’s a stupid way around it, surely. Child porn is surely, and tragically, in continuous production. And really, it’s the production that is the biggest issue, this is the continued abuse of children. And that’s the one bit the algo won’t match against?
  - samrolken 5y ago
    Apple has documentation explaining how it all works with a database of known images.
- fragileone 5y ago
  Out of 200k photos being matched against I'd be extremely surprised if there aren't at least some false positives generated.