6 ms·

I am against the idea of scanning for the reason that the author pointed out: It's trivial to repurpose the technology to use it in dystopian ways. I however h

by quitit 3y ago

I am against the idea of scanning for the reason that the author pointed out: It's trivial to repurpose the technology to use it in dystopian ways. I however have precisely zero concerns about impersonating hashes: 1. It's trivial to deal with tainting the database: both secondary hashing and more invasive hashes deal with that problem. 2. It's trivial to deal with impersonated hashes, all positives can be scanned on device in a second round with a different hash method. Those which are still positive can have be checked off device in an automated, yet privacy preserving, way such as by sending a low resolution crop with faces blurred to a system that does a likeness check against the matched CSA image: only the real image will bear likeness. 3. These systems have undisclosed minimums. A person sharing CSAM will be generating positive results at a significant rate. Receiving a few images with faked hashes won't set off alarms, and if a victim is suddenly receiving hundreds of images, that would already be obvious. 4. These articles over sensationalise what occurs when a positive match is found. They vaguely gesture to serious consequences from a single match. In reality even without secondary automated checks, the false positive will be noticed by a human and then discarded. If a person is receiving a lot of false positives that may lead to the individual being informed that they are begin targeted. Overall the idea of scanning chat content seems ineffective, for much of the same reasons why it's ineffective to ban end-to-end encrypted chat: those with criminal intent will just use something else or roll their own.

Hizonner 3y ago
> It's trivial to deal with tainting the database: both secondary hashing and more invasive hashes deal with that problem. Sorry, what does "more invasive hashes" mean? > It's trivial to deal with impersonated hashes, all positives can be scanned on device in a second round with a different hash method. The double hashing would definitely help. I doubt anybody's gotten a close collision for more than one hash at a time. It's not inconceivable that they could do it, though. As far as I know, there are only two hash methods that get used for this, and I think both of them are based on sliding a window over a reduced version of the image and doing a DCT, so they're pretty similar. I wouldn't be surprised if somebody could fool both of them enough to at least force you to tighten up your similarity threhsholds. It would be safer to come up with something that worked on completely different principles. No idea how hard that would be. I don't think that the second-stage backup, where you send the image off somewhere, is "privacy preserving" by any standard I'd be comfortable with. Well, OK, actually even completely foolproof client side scanning isn't privacy preserving enough for me, but I mean that's substantially worse. I also don't think it'd be easy to find anybody to run that server. And it'd be a significant structural change from what's deployed now. It'd be much more disruptive than adding a second hash (which is isomorphic to just switching to a single composite hash). It took a long time to get the present system deployed, so that big a change doesn't seem "trivial". > These systems have undisclosed minimums. A person sharing CSAM will be generating positive results at a significant rate. Receiving a few images with faked hashes won't set off alarms, and if a victim is suddenly receiving hundreds of images, that would already be obvious. Minimum what? If you just say "undisclosed minimums", my mind immediately assumes you mean threshold similarity scores. But you seem to mean hit counts to trigger various actions. Anyway, both of those are set by the service that's using the database. Those services vary in their strictness and clue level, so I'm not sure you can say anything very general about them. > These articles over sensationalise what occurs when a positive match is found. In the threat model of the original article, the people doing the review for false positives are most likely the same people who poisoned the database to begin with. They've corrupted it to give them hits on images they want to suppress, even though the system operators don't want to support them in suppressing those images. So that whole "reviewed by a human and discarded" step doesn't happen. In the broader world, I suspect that the thresholds for "delete the file" tend to be very low... too low for there to be human review. The thresholds for "disable the user's account" probably aren't exactly stratospheric, either. You can have damaging consequences well short of midnight police raids. Also, "framing" people isn't necessarily the only thing you might use it for. For example, you could try to use it to drive the cost of running the system up to unsupportable levels.
- LorenPechtel 3y ago
  I've never played with fingerprinting per se, but long ago I did write a program to detect near-duplicate images. It was based on comparing square of the differences of extremely downscaled images. Quite good at it's intended purpose, exposed the fact that a few images were photoshops and revealed that images that were inherently low-contrast could easily be confused by that approach--beaches looked an awful lot like beaches.
phyzome 3y ago
If you can make one algorithm collide, you can make two collide.
- astrange 3y ago
  You can't because you don't have access to one of the algorithms.
  - logi_sko 3y ago
    We don't actually have access to the first algorithm either, but it's been reverse engineered and a binary published. According to article.
    - astrange 3y ago
      I should have said "algorithm deployment" or something. The main point is that on the server side, you can change the hash (or its seed) anytime you want, if the false positive rate is getting too high. It may be possible to do the same thing to the client actually.
  - quitit 3y ago
    In online people sometimes write things the way they want them to be, not the way they factually are. In small ways it comes across as wishful thinking, in other ways it's just plain old lying. I didn't feel that the comment trivialising multiple hash collisions was worth a direct reply, because the author is clearly not writing with bona fide intentions, or any real knowledge about an image scanning system. It's the fingers in ears, head in the sand kind of denialism that adds nothing to the conversation. My earlier comment already addressed what happens after the hashes match: a visual comparison. This isn't an original concept, it's a standard approach and part of earlier CSAM scanning proposals. One needs to act rather barefaced to pretend that a matched hash alone will have consequences, when we already have established that false positives are the known downside to using hash-based image matching.
- alexfoo 3y ago
  If you can make 46 collide you can mint your own bitcoins. The increase in difficulty is multiplicative not additive.

by quitit 3y ago

Hizonner 3y ago

> It's trivial to deal with tainting the database: both secondary hashing and more invasive hashes deal with that problem. Sorry, what does "more invasive hashes" mean? > It's trivial to deal with impersonated hashes, all positives can be scanned on device in a second round with a different hash method. The double hashing would definitely help. I doubt anybody's gotten a close collision for more than one hash at a time. It's not inconceivable that they could do it, though. As far as I know, there are only two hash methods that get used for this, and I think both of them are based on sliding a window over a reduced version of the image and doing a DCT, so they're pretty similar. I wouldn't be surprised if somebody could fool both of them enough to at least force you to tighten up your similarity threhsholds. It would be safer to come up with something that worked on completely different principles. No idea how hard that would be. I don't think that the second-stage backup, where you send the image off somewhere, is "privacy preserving" by any standard I'd be comfortable with. Well, OK, actually even completely foolproof client side scanning isn't privacy preserving enough for me, but I mean that's substantially worse. I also don't think it'd be easy to find anybody to run that server. And it'd be a significant structural change from what's deployed now. It'd be much more disruptive than adding a second hash (which is isomorphic to just switching to a single composite hash). It took a long time to get the present system deployed, so that big a change doesn't seem "trivial". > These systems have undisclosed minimums. A person sharing CSAM will be generating positive results at a significant rate. Receiving a few images with faked hashes won't set off alarms, and if a victim is suddenly receiving hundreds of images, that would already be obvious. Minimum what? If you just say "undisclosed minimums", my mind immediately assumes you mean threshold similarity scores. But you seem to mean hit counts to trigger various actions. Anyway, both of those are set by the service that's using the database. Those services vary in their strictness and clue level, so I'm not sure you can say anything very general about them. > These articles over sensationalise what occurs when a positive match is found. In the threat model of the original article, the people doing the review for false positives are most likely the same people who poisoned the database to begin with. They've corrupted it to give them hits on images they want to suppress, even though the system operators don't want to support them in suppressing those images. So that whole "reviewed by a human and discarded" step doesn't happen. In the broader world, I suspect that the thresholds for "delete the file" tend to be very low... too low for there to be human review. The thresholds for "disable the user's account" probably aren't exactly stratospheric, either. You can have damaging consequences well short of midnight police raids. Also, "framing" people isn't necessarily the only thing you might use it for. For example, you could try to use it to drive the cost of running the system up to unsupportable levels.

LorenPechtel 3y ago
I've never played with fingerprinting per se, but long ago I did write a program to detect near-duplicate images. It was based on comparing square of the differences of extremely downscaled images. Quite good at it's intended purpose, exposed the fact that a few images were photoshops and revealed that images that were inherently low-contrast could easily be confused by that approach--beaches looked an awful lot like beaches.

phyzome 3y ago

If you can make one algorithm collide, you can make two collide.

astrange 3y ago
You can't because you don't have access to one of the algorithms.
- logi_sko 3y ago
  We don't actually have access to the first algorithm either, but it's been reverse engineered and a binary published. According to article.
  - astrange 3y ago
    I should have said "algorithm deployment" or something. The main point is that on the server side, you can change the hash (or its seed) anytime you want, if the false positive rate is getting too high. It may be possible to do the same thing to the client actually.
- quitit 3y ago
  In online people sometimes write things the way they want them to be, not the way they factually are. In small ways it comes across as wishful thinking, in other ways it's just plain old lying. I didn't feel that the comment trivialising multiple hash collisions was worth a direct reply, because the author is clearly not writing with bona fide intentions, or any real knowledge about an image scanning system. It's the fingers in ears, head in the sand kind of denialism that adds nothing to the conversation. My earlier comment already addressed what happens after the hashes match: a visual comparison. This isn't an original concept, it's a standard approach and part of earlier CSAM scanning proposals. One needs to act rather barefaced to pretend that a matched hash alone will have consequences, when we already have established that false positives are the known downside to using hash-based image matching.
alexfoo 3y ago
If you can make 46 collide you can mint your own bitcoins. The increase in difficulty is multiplicative not additive.