5 ms·

> "As mentioned in the article, this does absolutely nothing towards protecting children other than directing all but the biggest idiots towards platforms that

> "As mentioned in the article, this does absolutely nothing towards protecting children other than directing all but the biggest idiots towards platforms that can't be linked to them, which I'd imagine, they already are." I suspect you're more wrong than you think about this. People share large volumes of CSAM through lots of different services - I knew someone who worked on the problem at Linked In(!). HN likes to downplay the actual reality as if it's always some trojan horse, but the issue is real. It's worth talking to people that work on combatting it if you get the chance. I'm not really commenting on Apple's approach here (which I haven't thought enough about), but I know enough that an immediate dismissal based on it 'not helping' is not really appreciating the real tradeoffs you're making. You can be against this kind of thing from Apple, but as a result more CSAM will be undetected. Maybe that's the proper tradeoff, but we shouldn't pretend it's not a tradeoff at all. "Robin Hanson proposed stores where banned products could be sold. There are a number of excellent arguments for such a policy—an inherent right of individual liberty, the career incentive of bureaucrats to prohibit everything, legislators being just as biased as individuals. But even so (I replied), some poor, honest, not overwhelmingly educated mother of five children is going to go into these stores and buy a “Dr. Snakeoil’s Sulfuric Acid Drink” for her arthritis and die, leaving her orphans to weep on national television" [0] [0]: https://www.lesswrong.com/posts/PeSzc9JTBxhaYRp9b/policy-debates-should-not-appear-one-sided https://www.lesswrong.com/posts/PeSzc9JTBxhaYRp9b/policy-deb... Edit: After digging in, HN commentary is missing the most relevant details about this particular implementation. iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate. For iMessage child account ML scanning it’s on device and just generic sexual material restriction - more similar to standard parental controls than anything else (alerts only go to the parent, and only happen on child accounts) My initial impression is this is a good approach. The risk mostly comes from future applications (like the CCP adding hashes of other stuff that isn’t CSAM like tankman or something). The frankly ignorant knee-jerk responses from technical HN readers do a disservice and weaken the ability of technical people to push back when necessary.

White_Wolf 5y ago
I would say most (if not all) know how easily you can change the scope of a database like this(like you said yourself) to check for other inconvenient terms, links, memes, etc that do not "toe the party line"(whether is eastern or western - I don't care) and shut down inconvenient discourse. The point is: This is too easy to abuse.
- fossuser 5y ago
  Being specific with the arguments and addressing the nuance matters imo. Most of the HN responses are dumb, get the details wrong, and don’t take the trade offs seriously. That kind of thing causes me to dismiss them entirely. There are legitimate arguments and risks which I’d concede, but they’re not what most people in the comments are talking about.
  - ctvo 5y ago
    The trade-off here is to get pedophiles off of Apple services and on to something else, we give Apple an entry point into examining our private data. I understand it's hashes now, that does nothing to prevent this from expanding. "People just don't understand!!" has never been a convincing argument.
    - fossuser 5y ago
      This is just a slippery slope argument. The implementation specifically for detecting CSAM can be okay while using that for other purposes can not be okay. The goal is to stop child sexual abuse, FB reports millions of cases a year with a similar hash matching model for messenger. > ""People just don't understand!!" has never been a convincing argument." That's not my argument - I just think most of the HN comments on this issue both miss the relevant details, and are wrong.
      - ErikVandeWater 5y ago
        Slippery slope fallacy is often implied incorrectly when it comes to people. Human nature is subject to the "foot in the door" sales tactic. https://en.wikipedia.org/wiki/Foot-in-the-door_technique https://en.wikipedia.org/wiki/Foot-in-the-door_technique
    - DSingularity 5y ago
      I think HN takes the incorrectly perceived moral high ground and ignore the fact that there is a real duty to act here. There are people who profit from CSAM and in turn this creates a market for abuse. Unless we create structures which disincentivizes these behaviors — right now there are none - they will continue to grow. What Apple is building will basically make any criminal who sells to someone sloppy enough to store in iCloud risk being traced as the origin of the CSAM. Back to the darkest web they go. Anti CSAM is inevitable. You will find similar systems for all major providers eventually.
JoshTko 5y ago
Is CSAM a specific list of images that does not change, or does a government body actively control this list? Is there anything that would technically prevent the addition of a specific image of an enemy of the state? Hypothetical question, how will Apple respond when the FBI asks Apple to scan for images they know are also on the phone of the latest domestic terrorist? And how likely do you think the FBI will try to expand the scope of images that are scanned?
- fossuser 5y ago
  I don't know - I think these are the good questions and the things I would want to know in more detail. I think this kind of thing is where the risk lies. I think this is why getting the details right matter because if people are arguing about unrelated nonsense it's harder to focus on the actual risks represented by the questions you're asking.
2pEXgD0fZ5cF 5y ago
One of the prerequisites to "discuss the tradeoffs" of this ordeal is the trust that the people behind it, and those in the position to push things further are actually interested in keeping the scope limited. There are diminishing returns to using the same old excuse again and again. I'd say most people are just tired of the whole "Just think of the children!" into "Ah we got this system in place why not use it against <a little less evil but still illegal thing> too" followed by "It's the law in China/Turkey/Russia/wherever, <Company> can't just ignore it (and thus not help putting reporters, critics and other people into prison)" combo. What you are saying is basically another rendition of "look the problem of child abuse exists and this could help so it is worth discussing", which is also a variant of the same. > iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate. Actually we recently had news where an Apple tried to help cover up their errors [1], the system was supposed to be safe™, doesn't mean the people having control over it can't make mistakes, or worse. What details are being missed here, exactly? Ultimately it is trivial to expand the hashes to compare to, isn't it? What does it matter that they use CSAM for now? It doesn't remove the involvement of humans in controlling the system, so false positive rate will never be "effectively zero", and it can easily be expanded. We are left with the same two arguments as always: - Think about the children - Just trust the company, they use technology™, no you aren't allowed to check. Yeah they can easily abuse it, but we don't know for sure! I wouldn't say HN is ignoring details, many people are just tired of the same old loop, there is no reason to put trust into these endeavors and it is reasonable to doubt even the motives. [1]: https://news.ycombinator.com/item?id=27878333 https://news.ycombinator.com/item?id=27878333
int_19h 5y ago
It does not imply a false zero positive rate at all, because the hashes used are by necessity fuzzy, and even a completely benign picture can trigger a match. Now they're saying that if the match is a false positive, it'll be screened by the human reviewer. But that means there's some stranger there potentially looking at my private photos (and, by the way, I wonder which country they'll be located in?). That's already completely and utterly unacceptable - you don't have to wait for any "future complications".
ComputerGuru 5y ago
You need to stop assuming everyone is commenting out of ignorance and read up on how perceptual hashing works. It’s about as far as you can get from zero false positives, if configured otherwise it becomes just a poor version of SHA or whatever and can only detect exact matches.
- fossuser 5y ago
  At the time of my comments people weren’t discussing the fuzziness of perceptual hashing - they weren’t discussing implementation details at all. I don’t know enough about the specific implementation or perceptual hashing details and probably pushed back too hard as a result of the other comments at the time (which were comments out of ignorance). The level of downvotes I received is disproportionate to what I wrote anyway - this is clearly a political issue on HN. The irony is I’d probably align more with the risks being too high position, but it needs to be considered after actually understanding what the true risks are first.
  - ComputerGuru 5y ago
    Apple is obviously already scanning for CSAM for whatever non-E2E content hits their servers. With this, they offload some of the cost of computation to the end nodes but that can’t be the only reason for pushing this tech, so it’s only natural to ask why and look for avenues of exploitation. The Trojan horse/foot-in-the-door hypothesis is basically Occam’s razor.
    - deleted 5y ago
      [deleted]
    - fossuser 5y ago
      This Daring Fireball blog post is a better summary than the discussion I’ve found on HN: https://daringfireball.net/2021/08/apple_child_safety_initiatives_slippery_slope https://daringfireball.net/2021/08/apple_child_safety_initia... Occam’s razor is that they’re doing what they say they’re doing and not some more complicated future action.
dcow 5y ago
It’s not just HN folks, go read the “open letter”.
deleted 5y ago
[deleted]
voidnullnil 5y ago
> You can be against this kind of thing from Apple, but as a result more CSAM will be undetected. You cannot claim to be making "the real reasonable analysis" and write this. So much for "you're all geeks stuck on technical details". Quite the contrary: I'm sick of bogus software pretending to solve problems for me, while the quality of tech has exponential degraded over the last 20 years (often due to trying to solve some unsolvable problem in a bad way that backfires). Now imagine you have a 16 year old girlfriend. She sends you a nude photo. Your phone calls the cops on you (it doesn't matter if the phone doesn't quite do this now, it will in the future. They will use their ML crap to detect the age of subjects in photos and explicitness of the photo). You normally wouldn't go to jail for this since 16 is legal in 99% of the civilized world, but thanks to America with their super duper "non-technical" innovations that only big boy white collars can understand, you can go to jail for having a photo of your legal girlfriend.
- skinkestek 5y ago
  Mostly good... > big boy white but why the totally uncalled racism and sexism here? It does nothing to strengthen your argument and it is just dumb.
  - voidnullnil 5y ago
    I am truly sorry for your loss in that your brain is implemented using regex. I meant "white collar big boys", but I did not bother to edit as I'm writing. The guy above is claiming everyone who is against apple's yet-another-bogus-TPM-style-snakeoil is a little geek who does not understand anything outside their little tunnel. Also now that I re-read his comment: > Edit: After digging in, HN commentary is missing the most relevant details about this particular implementation. iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate. False: it's a perceptual hash. Ignoring the fact that if for some reason you choose to let people host stuff in your icloud account (perhaps as a neat hack), which may be out of terms of service, but certainly not worth 20 years of jail: perceptual hashes have false positives, and can confuse images that appear harmless but were crafted to look like $badimg. But you don't have to be technical to understand that having your devices police you is bad, you just have to not be blinded by politics and boogeyman your state has sold you.

White_Wolf 5y ago

I would say most (if not all) know how easily you can change the scope of a database like this(like you said yourself) to check for other inconvenient terms, links, memes, etc that do not "toe the party line"(whether is eastern or western - I don't care) and shut down inconvenient discourse. The point is: This is too easy to abuse.

fossuser 5y ago
Being specific with the arguments and addressing the nuance matters imo. Most of the HN responses are dumb, get the details wrong, and don’t take the trade offs seriously. That kind of thing causes me to dismiss them entirely. There are legitimate arguments and risks which I’d concede, but they’re not what most people in the comments are talking about.
- ctvo 5y ago
  The trade-off here is to get pedophiles off of Apple services and on to something else, we give Apple an entry point into examining our private data. I understand it's hashes now, that does nothing to prevent this from expanding. "People just don't understand!!" has never been a convincing argument.
  - fossuser 5y ago
    This is just a slippery slope argument. The implementation specifically for detecting CSAM can be okay while using that for other purposes can not be okay. The goal is to stop child sexual abuse, FB reports millions of cases a year with a similar hash matching model for messenger. > ""People just don't understand!!" has never been a convincing argument." That's not my argument - I just think most of the HN comments on this issue both miss the relevant details, and are wrong.
    - ErikVandeWater 5y ago
      Slippery slope fallacy is often implied incorrectly when it comes to people. Human nature is subject to the "foot in the door" sales tactic. https://en.wikipedia.org/wiki/Foot-in-the-door_technique https://en.wikipedia.org/wiki/Foot-in-the-door_technique
  - DSingularity 5y ago
    I think HN takes the incorrectly perceived moral high ground and ignore the fact that there is a real duty to act here. There are people who profit from CSAM and in turn this creates a market for abuse. Unless we create structures which disincentivizes these behaviors — right now there are none - they will continue to grow. What Apple is building will basically make any criminal who sells to someone sloppy enough to store in iCloud risk being traced as the origin of the CSAM. Back to the darkest web they go. Anti CSAM is inevitable. You will find similar systems for all major providers eventually.

JoshTko 5y ago

Is CSAM a specific list of images that does not change, or does a government body actively control this list? Is there anything that would technically prevent the addition of a specific image of an enemy of the state? Hypothetical question, how will Apple respond when the FBI asks Apple to scan for images they know are also on the phone of the latest domestic terrorist? And how likely do you think the FBI will try to expand the scope of images that are scanned?

fossuser 5y ago
I don't know - I think these are the good questions and the things I would want to know in more detail. I think this kind of thing is where the risk lies. I think this is why getting the details right matter because if people are arguing about unrelated nonsense it's harder to focus on the actual risks represented by the questions you're asking.

2pEXgD0fZ5cF 5y ago

One of the prerequisites to "discuss the tradeoffs" of this ordeal is the trust that the people behind it, and those in the position to push things further are actually interested in keeping the scope limited. There are diminishing returns to using the same old excuse again and again. I'd say most people are just tired of the whole "Just think of the children!" into "Ah we got this system in place why not use it against <a little less evil but still illegal thing> too" followed by "It's the law in China/Turkey/Russia/wherever, <Company> can't just ignore it (and thus not help putting reporters, critics and other people into prison)" combo. What you are saying is basically another rendition of "look the problem of child abuse exists and this could help so it is worth discussing", which is also a variant of the same. > iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate. Actually we recently had news where an Apple tried to help cover up their errors [1], the system was supposed to be safe™, doesn't mean the people having control over it can't make mistakes, or worse. What details are being missed here, exactly? Ultimately it is trivial to expand the hashes to compare to, isn't it? What does it matter that they use CSAM for now? It doesn't remove the involvement of humans in controlling the system, so false positive rate will never be "effectively zero", and it can easily be expanded. We are left with the same two arguments as always: - Think about the children - Just trust the company, they use technology™, no you aren't allowed to check. Yeah they can easily abuse it, but we don't know for sure! I wouldn't say HN is ignoring details, many people are just tired of the same old loop, there is no reason to put trust into these endeavors and it is reasonable to doubt even the motives. [1]: https://news.ycombinator.com/item?id=27878333 https://news.ycombinator.com/item?id=27878333

int_19h 5y ago

It does not imply a false zero positive rate at all, because the hashes used are by necessity fuzzy, and even a completely benign picture can trigger a match. Now they're saying that if the match is a false positive, it'll be screened by the human reviewer. But that means there's some stranger there potentially looking at my private photos (and, by the way, I wonder which country they'll be located in?). That's already completely and utterly unacceptable - you don't have to wait for any "future complications".

ComputerGuru 5y ago

You need to stop assuming everyone is commenting out of ignorance and read up on how perceptual hashing works. It’s about as far as you can get from zero false positives, if configured otherwise it becomes just a poor version of SHA or whatever and can only detect exact matches.

fossuser 5y ago
At the time of my comments people weren’t discussing the fuzziness of perceptual hashing - they weren’t discussing implementation details at all. I don’t know enough about the specific implementation or perceptual hashing details and probably pushed back too hard as a result of the other comments at the time (which were comments out of ignorance). The level of downvotes I received is disproportionate to what I wrote anyway - this is clearly a political issue on HN. The irony is I’d probably align more with the risks being too high position, but it needs to be considered after actually understanding what the true risks are first.
- ComputerGuru 5y ago
  Apple is obviously already scanning for CSAM for whatever non-E2E content hits their servers. With this, they offload some of the cost of computation to the end nodes but that can’t be the only reason for pushing this tech, so it’s only natural to ask why and look for avenues of exploitation. The Trojan horse/foot-in-the-door hypothesis is basically Occam’s razor.
  - deleted 5y ago
    [deleted]
  - fossuser 5y ago
    This Daring Fireball blog post is a better summary than the discussion I’ve found on HN: https://daringfireball.net/2021/08/apple_child_safety_initiatives_slippery_slope https://daringfireball.net/2021/08/apple_child_safety_initia... Occam’s razor is that they’re doing what they say they’re doing and not some more complicated future action.

dcow 5y ago

It’s not just HN folks, go read the “open letter”.

deleted 5y ago

[deleted]

voidnullnil 5y ago

> You can be against this kind of thing from Apple, but as a result more CSAM will be undetected. You cannot claim to be making "the real reasonable analysis" and write this. So much for "you're all geeks stuck on technical details". Quite the contrary: I'm sick of bogus software pretending to solve problems for me, while the quality of tech has exponential degraded over the last 20 years (often due to trying to solve some unsolvable problem in a bad way that backfires). Now imagine you have a 16 year old girlfriend. She sends you a nude photo. Your phone calls the cops on you (it doesn't matter if the phone doesn't quite do this now, it will in the future. They will use their ML crap to detect the age of subjects in photos and explicitness of the photo). You normally wouldn't go to jail for this since 16 is legal in 99% of the civilized world, but thanks to America with their super duper "non-technical" innovations that only big boy white collars can understand, you can go to jail for having a photo of your legal girlfriend.

skinkestek 5y ago
Mostly good... > big boy white but why the totally uncalled racism and sexism here? It does nothing to strengthen your argument and it is just dumb.
- voidnullnil 5y ago
  I am truly sorry for your loss in that your brain is implemented using regex. I meant "white collar big boys", but I did not bother to edit as I'm writing. The guy above is claiming everyone who is against apple's yet-another-bogus-TPM-style-snakeoil is a little geek who does not understand anything outside their little tunnel. Also now that I re-read his comment: > Edit: After digging in, HN commentary is missing the most relevant details about this particular implementation. iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate. False: it's a perceptual hash. Ignoring the fact that if for some reason you choose to let people host stuff in your icloud account (perhaps as a neat hack), which may be out of terms of service, but certainly not worth 20 years of jail: perceptual hashes have false positives, and can confuse images that appear harmless but were crafted to look like $badimg. But you don't have to be technical to understand that having your devices police you is bad, you just have to not be blinded by politics and boogeyman your state has sold you.