3 ms·

> Anyone who claims this is evidence of danger either doesn’t understand how the system works, or is deliberately misleading people. Anyone who claims this is

> Anyone who claims this is evidence of danger either doesn’t understand how the system works, or is deliberately misleading people. Anyone who claims this is not evidence of danger either doesn’t understand how the system works, or is deliberately misleading people. Fixed it for you :) On device scanning did not exist before apple. Apple is doing it. Now devices don't have your best interest in mind. Bad precedent, easy to be abused in future by others with even worse implementations or motives. Evidence of danger. Their hashing algorithm by nature of being a hashing algorithm will have collisions. That is unavoidable. Unavoidable false positives are not great in the context of your device ratting you out to the authorities.Evidence of danger. QED.

Jtsummers 5y ago
We should be careful with this part: > Unavoidable false positives are not great in the context of your device ratting you out to the authorities. [emphasis added] If the detection were immediately sent to authorities, this would be an issue. And it's possible in the future (or in practice) that the manual review could be removed or will be a rubber stamp. However, they do have a manual review process slated for this. The reason I promote caution with regard to this element of your argument is that if you have an easily rebutted statement, people will disregard the rest. It's why hyperbole in most political arguments should be strongly avoided, it makes it easy for people to dismiss the rest of the statement(s) no matter how valid.
- snowwrestler 5y ago
  Not only does Apple have their own manual review step, if an alert gets through that, then NCMEC (in the U.S) does their own, additional review, and then law enforcement would also do a manual review before seeking a warrant. And unlike Apple, law enforcement can and will review the actual full-resolution file(s) in question.
throw87454 5y ago
> On device scanning did not exist before apple Virus scanners scan on devices. Practically all big brand commercial scanners nowadays scan and check findings on-the-fly online. That said, Microsoft ships Defender since XP or so. Apple has some basic malware scanning since Catalina where it compares hashes of executables. Facebook checks hashes of pictures sent in messages against hashes of abuse picture since years. > Their hashing algorithm by nature of being a hashing algorithm will have collisions. Yes and no. You'll probably not find any collision in a SHA-3 hash. Quite likely you won't even encounter a SHA-1 collision. But this Neural Hash? It's been published since this short amount of time and already such a collision is being presented. As a user I probably don't want Viruses, Ransomware or anything with comparable destructiveness on my computer. But at the same time the implementation should be rock-solid and user-controllable. That's a no-brainer for anything sha-based but for Neural Hash it isn't.
- Jtsummers 5y ago
  Virus scanners do not normally report to some authority. That is an important distinction. For personal (that is, not corporate owned or provisioned) devices this creates a blurring of lines that used to be clearer. If these scans warned you that the content was questionable and quarantined it with an option for you to release it on your own review, it would be truly analogous to antivirus software. However, that's not what's happening. I think a lot of the arguments against are overstating certain elements and often ignoring others (like that there are two manual reviews before anything goes to law enforcement), but we can't go in the other direction to defend the system either. It is not like existing on-device scanning solutions that are meant to protect (by some definition) the user or the device.
zepto 5y ago
> Their hashing algorithm by nature of being a hashing algorithm will have collisions. That is unavoidable. Yes, exactly. They themselves state this, and have a mechanism for preventing collisions from turning into false positive matches. I have to assume you simply didn’t know that.

Jtsummers 5y ago

We should be careful with this part: > Unavoidable false positives are not great in the context of your device ratting you out to the authorities. [emphasis added] If the detection were immediately sent to authorities, this would be an issue. And it's possible in the future (or in practice) that the manual review could be removed or will be a rubber stamp. However, they do have a manual review process slated for this. The reason I promote caution with regard to this element of your argument is that if you have an easily rebutted statement, people will disregard the rest. It's why hyperbole in most political arguments should be strongly avoided, it makes it easy for people to dismiss the rest of the statement(s) no matter how valid.

snowwrestler 5y ago
Not only does Apple have their own manual review step, if an alert gets through that, then NCMEC (in the U.S) does their own, additional review, and then law enforcement would also do a manual review before seeking a warrant. And unlike Apple, law enforcement can and will review the actual full-resolution file(s) in question.

throw87454 5y ago

> On device scanning did not exist before apple Virus scanners scan on devices. Practically all big brand commercial scanners nowadays scan and check findings on-the-fly online. That said, Microsoft ships Defender since XP or so. Apple has some basic malware scanning since Catalina where it compares hashes of executables. Facebook checks hashes of pictures sent in messages against hashes of abuse picture since years. > Their hashing algorithm by nature of being a hashing algorithm will have collisions. Yes and no. You'll probably not find any collision in a SHA-3 hash. Quite likely you won't even encounter a SHA-1 collision. But this Neural Hash? It's been published since this short amount of time and already such a collision is being presented. As a user I probably don't want Viruses, Ransomware or anything with comparable destructiveness on my computer. But at the same time the implementation should be rock-solid and user-controllable. That's a no-brainer for anything sha-based but for Neural Hash it isn't.

Jtsummers 5y ago
Virus scanners do not normally report to some authority. That is an important distinction. For personal (that is, not corporate owned or provisioned) devices this creates a blurring of lines that used to be clearer. If these scans warned you that the content was questionable and quarantined it with an option for you to release it on your own review, it would be truly analogous to antivirus software. However, that's not what's happening. I think a lot of the arguments against are overstating certain elements and often ignoring others (like that there are two manual reviews before anything goes to law enforcement), but we can't go in the other direction to defend the system either. It is not like existing on-device scanning solutions that are meant to protect (by some definition) the user or the device.

zepto 5y ago

> Their hashing algorithm by nature of being a hashing algorithm will have collisions. That is unavoidable. Yes, exactly. They themselves state this, and have a mechanism for preventing collisions from turning into false positive matches. I have to assume you simply didn’t know that.