13 ms·

Thinking about this from a more technical perspective. CSAM scanning will only catch the dumbest of abusers. One with even a remote level of sophistication will

Thinking about this from a more technical perspective. CSAM scanning will only catch the dumbest of abusers. One with even a remote level of sophistication will just spin up their own servers, there is plenty of OSS software out there to do such a thing, and thus would completely bypass any measures that are being proposed. With services like Digital Ocean, you can spin up a chat server with pretty much only the most basic of technical knowledge, hell you likely won't even have to touch a command line. Sure the EU could move to require OSS software developers operating in the EU to backdoor their software but that will be moot too as it will get caught on to in a heartbeat and a fork (sans backdoor) will be made available.

rvanlaar 4y ago
It will catch random webmasters. In short, Dutch hoster TransIP had to pause their scanning service. They got hashes that included default images from WordPress. "Waar het misging, is volgens TransIP-woordvoerder Marco Edelman dat per ongeluk hashes van standaardafbeeldingen van WordPress-installaties en plug-ins zijn toegevoegd." [1] https://tweakers.net/nieuws/182766/transip-pauzeert-hashcheckservice-na-onterecht-verwijderen-van-afbeeldingen.html https://tweakers.net/nieuws/182766/transip-pauzeert-hashchec...
- rolandog 4y ago
  With the high-likelihood of false positives, this is just another Child Benefits scandal waiting to happen if the system were to be completely unsupervised.
- withinboredom 4y ago
  It uses the MD5 of the image, no surprise there that there were collisions with ordinary files. I've run into MD5 hash collisions causing mayhem several times in my life (usually between unrelated email addresses, IIRC).
  - readams 4y ago
    You found 2 actual emails with the same md5? That sounds very unlikely. MD5 is weak against attack, but you're not going to be hitting collisions with "normal" data and especially not short strings.
    - withinboredom 4y ago
      Just looked over my notes. It was not email addresses, but a collision of two separate UPC codes concatenated with a timestamp. We never figured out which UPC codes it was or the timestamp (this was picking/sorting software for a warehouse in 2012). I wasn't there when it happened, and only heard about it after the fact (I was on my honeymoon). It crashed the software pretty hard.
      - readams 4y ago
        That would be interesting since that would be likely the shortest known MD5 collisions. The shortest ones known are 512 bits differing by 2 bits. If I had to guess, you might have been hitting collisions in a truncated MD5 hash. It's not uncommon to do something like take only the first 64 bits of the hash. Doing this you are quite vulnerable to birthday problem issues.
        withinboredom 4y ago
        It was in php using the built-in md5 function. I’m actually reaching out to a friend that still has the code to see what the exact string is that got hashed. I have no idea if the database is still somewhere, if so, we can look at the git commit and figure out what the possible UPCs are and narrow it down to a couple of hours worth of time stamps. I have a feeling it would probably be more than 512 bits though. UPCs are can be longish, plus a time stamp with each digit being a byte… I highly doubt the database exists in any meaningful way though, since the product was shut down years ago.
- GekkePrutser 4y ago
  What's so worrying about this is that these stock images were added there in the first place. Clearly the process is completely broken, if there was any kind of human oversight this would have been caught right there. Right now it was clear because it was an often-used package. What if it's something more niche next time?
b112 4y ago
You're right! I think, to protect the children, maybe we should require licensing to run servers? Here's how it works. By law, all machines will require networking, or will refuse to boot. Network stacks will require a key / license. Licence will be verified by an online server, continually, and any OS without this will be fined, and the devs implementing a bypass jailed. Stop complaining/pointing out flaws, or you must hate children! Or... are you one of them?
- SpEd3Y 4y ago
  All machines require networking or refuse to boot? That's quite a solution...
- IntrepidWorm 4y ago
  I know that HN policy is to take the best faith interpretation, but I'm really struggling here. The ramifications of what you suggested are boggling to consider- any developer of any system must implement and require a network at the lowest of levels... That would be impossible to enforce, and a nightmare to implement. Secondly, theres still nothing to prevent dark networks from emerging. Once its out there, its out there. Key/license servers would be even more of a target for corruption, exploits, and circumvention than encryption servers, with all of the same problems we see currently. In no world is this possible, never mind livable. It would be Orwellian to even imagine.
  - markild 4y ago
    I believe it's meant to be read as sarcasm.
- LinuxBender 4y ago
  Surely you jest but truth be told this is the direction I have been watching companies move albeit slowly and carefully. Microsoft are slowly baking in this concept into Windows 10/11 to become more like an XBox at least for the home version. I envision a future version of windows will operate very closely to the method you described. I suspect all of the OS vendors want an app store in front of everyone whenever possible and telemetry to see what needs tuning to keep interaction with the app store as high as attainable. I do not believe it would be a stretch to see a future home version of windows not allow login if their authentication servers can not be reached or if ones cached remote credentials expire. I could even see this eventually happening with Linux. The latest version of the Fedora desktop F36 interacts heavily with admin.fedoraproject.org and they have an app store. AFAIK Redhat/IBM have not yet implemented any centralized authentication servers with corresponding default client configurations to utilize them and I do not know if they intend to.
treis 4y ago
>CSAM scanning will only catch the dumbest of abusers. Why isn't that better than catching none of the abusers?
- deleted 4y ago
  [deleted]
- everforward 4y ago
  It's a balance of issues. These measures are only likely to lead to the arrest of a small portion of abusers, while opening a massive avenue for government overreach across the entire population. I would assume that very few to none of us on here are pedophiles, and we're aware this is going on. It stands to reason that the pedophiles are also aware that this is going on, and the vast majority of them are going to adjust their behavior to avoid this scanning. Doubly so once a few arrests are made. The balance of those issues is subjective, though.
  - GekkePrutser 4y ago
    Exactly. A handful of actual abusers will be caught and the rest will change their methods. The rest of us will have a system looking over our shoulder for the rest of our lives with an ever-expanding scope of uses. Not exactly a great trade-off.
- vorpalhex 4y ago
  The same reason using napalm to cut down on weeds in your backyard is probably unwise.
- Beltiras 4y ago
  I would like law enforcement to use the tools they have now before they beg for more tools to (ab)use.
- okamiueru 4y ago
  If only it caught abusers, it would be great. Just think about what kids do while teenagers, throw in some poor judgement. Kids being kids. Is it a good thing that there should be a system in place to "catch" this? To involve strangers in their most intimate and vulnerable private communique? That's just one argument on "even if everything works exactly as intended, it's still a terrible idea". Then add malicious use of such a system once in place. We know such systems are misused ALL the time. So I would bet good money that someone, somewhere, will instead use it to neatly collect CSAM material. Not to mention other malicious use of surveillance whereby you just add some hashes for files completely unrelated to CSAM. I'm not talking about the slippery-slope of "let's include some gay porn in the mix", but rather stuff related to political opponents, and what not. It's a false dichotomy to suggest the trade-off is "catch some abusers" vs "catch no abusers". It's a "catch some abusers" vs "violate privacy of minors | remove e2e encrypted communication | etc"

rvanlaar 4y ago

It will catch random webmasters. In short, Dutch hoster TransIP had to pause their scanning service. They got hashes that included default images from WordPress. "Waar het misging, is volgens TransIP-woordvoerder Marco Edelman dat per ongeluk hashes van standaardafbeeldingen van WordPress-installaties en plug-ins zijn toegevoegd." [1] https://tweakers.net/nieuws/182766/transip-pauzeert-hashcheckservice-na-onterecht-verwijderen-van-afbeeldingen.html https://tweakers.net/nieuws/182766/transip-pauzeert-hashchec...

rolandog 4y ago
With the high-likelihood of false positives, this is just another Child Benefits scandal waiting to happen if the system were to be completely unsupervised.
withinboredom 4y ago
It uses the MD5 of the image, no surprise there that there were collisions with ordinary files. I've run into MD5 hash collisions causing mayhem several times in my life (usually between unrelated email addresses, IIRC).
- readams 4y ago
  You found 2 actual emails with the same md5? That sounds very unlikely. MD5 is weak against attack, but you're not going to be hitting collisions with "normal" data and especially not short strings.
  - withinboredom 4y ago
    Just looked over my notes. It was not email addresses, but a collision of two separate UPC codes concatenated with a timestamp. We never figured out which UPC codes it was or the timestamp (this was picking/sorting software for a warehouse in 2012). I wasn't there when it happened, and only heard about it after the fact (I was on my honeymoon). It crashed the software pretty hard.
    - readams 4y ago
      That would be interesting since that would be likely the shortest known MD5 collisions. The shortest ones known are 512 bits differing by 2 bits. If I had to guess, you might have been hitting collisions in a truncated MD5 hash. It's not uncommon to do something like take only the first 64 bits of the hash. Doing this you are quite vulnerable to birthday problem issues.
      - withinboredom 4y ago
        It was in php using the built-in md5 function. I’m actually reaching out to a friend that still has the code to see what the exact string is that got hashed. I have no idea if the database is still somewhere, if so, we can look at the git commit and figure out what the possible UPCs are and narrow it down to a couple of hours worth of time stamps. I have a feeling it would probably be more than 512 bits though. UPCs are can be longish, plus a time stamp with each digit being a byte… I highly doubt the database exists in any meaningful way though, since the product was shut down years ago.
GekkePrutser 4y ago
What's so worrying about this is that these stock images were added there in the first place. Clearly the process is completely broken, if there was any kind of human oversight this would have been caught right there. Right now it was clear because it was an often-used package. What if it's something more niche next time?

b112 4y ago

You're right! I think, to protect the children, maybe we should require licensing to run servers? Here's how it works. By law, all machines will require networking, or will refuse to boot. Network stacks will require a key / license. Licence will be verified by an online server, continually, and any OS without this will be fined, and the devs implementing a bypass jailed. Stop complaining/pointing out flaws, or you must hate children! Or... are you one of them?

SpEd3Y 4y ago
All machines require networking or refuse to boot? That's quite a solution...
IntrepidWorm 4y ago
I know that HN policy is to take the best faith interpretation, but I'm really struggling here. The ramifications of what you suggested are boggling to consider- any developer of any system must implement and require a network at the lowest of levels... That would be impossible to enforce, and a nightmare to implement. Secondly, theres still nothing to prevent dark networks from emerging. Once its out there, its out there. Key/license servers would be even more of a target for corruption, exploits, and circumvention than encryption servers, with all of the same problems we see currently. In no world is this possible, never mind livable. It would be Orwellian to even imagine.
- markild 4y ago
  I believe it's meant to be read as sarcasm.
LinuxBender 4y ago
Surely you jest but truth be told this is the direction I have been watching companies move albeit slowly and carefully. Microsoft are slowly baking in this concept into Windows 10/11 to become more like an XBox at least for the home version. I envision a future version of windows will operate very closely to the method you described. I suspect all of the OS vendors want an app store in front of everyone whenever possible and telemetry to see what needs tuning to keep interaction with the app store as high as attainable. I do not believe it would be a stretch to see a future home version of windows not allow login if their authentication servers can not be reached or if ones cached remote credentials expire. I could even see this eventually happening with Linux. The latest version of the Fedora desktop F36 interacts heavily with admin.fedoraproject.org and they have an app store. AFAIK Redhat/IBM have not yet implemented any centralized authentication servers with corresponding default client configurations to utilize them and I do not know if they intend to.

treis 4y ago

>CSAM scanning will only catch the dumbest of abusers. Why isn't that better than catching none of the abusers?

deleted 4y ago
[deleted]
everforward 4y ago
It's a balance of issues. These measures are only likely to lead to the arrest of a small portion of abusers, while opening a massive avenue for government overreach across the entire population. I would assume that very few to none of us on here are pedophiles, and we're aware this is going on. It stands to reason that the pedophiles are also aware that this is going on, and the vast majority of them are going to adjust their behavior to avoid this scanning. Doubly so once a few arrests are made. The balance of those issues is subjective, though.
- GekkePrutser 4y ago
  Exactly. A handful of actual abusers will be caught and the rest will change their methods. The rest of us will have a system looking over our shoulder for the rest of our lives with an ever-expanding scope of uses. Not exactly a great trade-off.
vorpalhex 4y ago
The same reason using napalm to cut down on weeds in your backyard is probably unwise.
Beltiras 4y ago
I would like law enforcement to use the tools they have now before they beg for more tools to (ab)use.
okamiueru 4y ago
If only it caught abusers, it would be great. Just think about what kids do while teenagers, throw in some poor judgement. Kids being kids. Is it a good thing that there should be a system in place to "catch" this? To involve strangers in their most intimate and vulnerable private communique? That's just one argument on "even if everything works exactly as intended, it's still a terrible idea". Then add malicious use of such a system once in place. We know such systems are misused ALL the time. So I would bet good money that someone, somewhere, will instead use it to neatly collect CSAM material. Not to mention other malicious use of surveillance whereby you just add some hashes for files completely unrelated to CSAM. I'm not talking about the slippery-slope of "let's include some gay porn in the mix", but rather stuff related to political opponents, and what not. It's a false dichotomy to suggest the trade-off is "catch some abusers" vs "catch no abusers". It's a "catch some abusers" vs "violate privacy of minors | remove e2e encrypted communication | etc"