4 ms·

Because it’s based on machine learning, why can’t some of the network and weights used for the hash be shared? So we can be sure it is not possible to match any

Because it’s based on machine learning, why can’t some of the network and weights used for the hash be shared? So we can be sure it is not possible to match anything else than children. The sub narrow network used for detecting only CP is of course secret, but then we know it can’t be used for revealing pictures of police, activists etc

Dylan16807 5y ago
It doesn't seem to work that way. It's a bunch of arbitrary hashes that might not show up as any content in particular except "this hash matches". In particular, there was someone claiming that images of empty rooms where abuse happened are part of the database, along with other images that appear in the same sets as CSAM.
doubtfuluser 5y ago
Like stated below, I’m seriously worried about Adversarial examples that fool the network into A) hiding bad pictures being not recognized B) triggering false positives on harmless pictures to discredit people. Opening up the network would mate it even easier to create the adversarials… Im not sure there is a winning position in the approach they used
- punnerud 5y ago
  «even easier to create the adversarials», agree. But if they find one of this pictures they can retrain in a way that find all of them on all phones. So it will never be safe to have CP on the phone/iCloud. And easier to keep Apple accountable on not matching anything else than the purpose.
- matwood 5y ago
  Do you use iCloud photos now? CSAM checking has already been used for years (on almost all photos services btw).
pjerem 5y ago
Because the technology is not the point. I just don’t want Apple to search anything in my photos. Because those are my photos. Not theirs. My phone doesn’t need a program that search for abused child photography since there is not such material on it. I don’t care if their thing is open or closed source : My iPhone is mine and I don’t want them to do anything on it that can get me arrested because of a false positive. We must not accept those intrusions. This thing is as serious as if IKEA regularly sent robots in your house to check all the photo albums on your IKEA shelf to check them against some database. Why does it sounds dumb when it’s IKEA but more acceptable when it’s your phone constructor ?

Dylan16807 5y ago

It doesn't seem to work that way. It's a bunch of arbitrary hashes that might not show up as any content in particular except "this hash matches". In particular, there was someone claiming that images of empty rooms where abuse happened are part of the database, along with other images that appear in the same sets as CSAM.

doubtfuluser 5y ago

Like stated below, I’m seriously worried about Adversarial examples that fool the network into A) hiding bad pictures being not recognized B) triggering false positives on harmless pictures to discredit people. Opening up the network would mate it even easier to create the adversarials… Im not sure there is a winning position in the approach they used

punnerud 5y ago
«even easier to create the adversarials», agree. But if they find one of this pictures they can retrain in a way that find all of them on all phones. So it will never be safe to have CP on the phone/iCloud. And easier to keep Apple accountable on not matching anything else than the purpose.
matwood 5y ago
Do you use iCloud photos now? CSAM checking has already been used for years (on almost all photos services btw).

pjerem 5y ago

Because the technology is not the point. I just don’t want Apple to search anything in my photos. Because those are my photos. Not theirs. My phone doesn’t need a program that search for abused child photography since there is not such material on it. I don’t care if their thing is open or closed source : My iPhone is mine and I don’t want them to do anything on it that can get me arrested because of a false positive. We must not accept those intrusions. This thing is as serious as if IKEA regularly sent robots in your house to check all the photo albums on your IKEA shelf to check them against some database. Why does it sounds dumb when it’s IKEA but more acceptable when it’s your phone constructor ?