4 ms·

There is in fact no "smart" way to compensate for this particular "bad side effect". Either your communications are spied on to weed out unapproved material, o

There is in fact no "smart" way to compensate for this particular "bad side effect". Either your communications are spied on to weed out unapproved material, or they're not. And there is no way to make the system architecture care about which material is allowed to be "unapproved". The right answer here is just to accept that, beyond a certain point, further reducing the amount of circulating child porn requires unacceptable tradeoffs. Then stop whining and wishing for impossible technical solutions.

benfortuna 2y ago
There is in fact. You could build AI directly into the app to detect inappropriate content and block it. The app has thus not violated end-to-end encryption, nor has that content been exposed to external parties. Platforms could definitely do more.
- shinryuu 2y ago
  And how do you deal with all false positives. That will just be deemed collateral damage?
  - hellojesus 2y ago
    To add to your excellent point, who gets to validate the models efficacy? How do we know the state hadn't trained it to report users talking about maga, or Isreal, or for those with Chinese national lovers?
  - throwaway290 2y ago
    How do you deal with all those UUIDs colliding? You can design a system where false positives are so rare they are insignificant and can be properly handled. The only reason we don't is because we don't think it's that much of a problem.
    - Hizonner 2y ago
      No, you can't. Machine learning doesn't work that way. You can keep the false positive level reasonably low for copies of already known and flagged images. You can't do it for new images.
      - throwaway290 2y ago
        Maybe you underestimate machine learning, if you check HN on any given day it can do anything, probably be the next president. But flagged known images was the point of Apple's algo, for example. Still everyone just went "forget abuse, my privacy is more important for me". Really at this point techbros deserved any dumb law that lets the government read their chats.
        Hizonner 2y ago
        > Maybe you underestimate machine learning, if you check HN on any given day [...] Maybe I actually know something about it. > But flagged known images was the point of Apple's algo, for example. You whined at me a little while ago about how all this was about "abuse, not just porn". Yet you're using that to justify a system that, as you describe it, could only find old, known images that have been circulating around the Internet and made it into a database. Meaning images of past abuse that cannot be prevented, by third parties who would not be caught by this. Pick a threat model, because the measures you defend don't address the threats you claim justify them. ... and if you start talking about "grooming" or "normalization" or other silly bullshit that hypothetically might have a third-order effect, but probably doesn't have any meaningful effect at all in real life, I'm not going to bother to answer it. > Still everyone just went "forget abuse, my privacy is more important for me". Everybody's privacy is important to me. Including the privacy of the children whom you want to have grow up in an ever-expanding panopticon. Because this isn't just about stupid bullshit like your embarrassing disease. It's about people ending up in prison. It's about building infrastructure that can trivially and secretly be repurposed to hurt people, including children, in serious, life-changing, and potentially life-ending ways. The Stasi were not a child-friendly institution.
throwaway290 2y ago
If you put "bad" in quotes, this means you don't think child abuse is actually bad? Cool. > The right answer here is just to accept that, beyond a certain point, further reducing the amount of circulating child porn Not porn, don't delude yourself. Actual abuse. > requires unacceptable tradeoffs And whether tradeoffs are acceptable or not depends on tradeoffs. If you refuse to help find solutions that don't involve spying, don't get all flustered when a law is adopted that does involve spying! > Either your communications are spied on to weed out unapproved material, or they're not. Exactly. Apple's solution did not involve any spying. People have spoken against, now we have solutions that do involve spying. Let's see where this goes yeah?
- Hizonner 2y ago
  > Apple's solution did not involve any spying. Spying by your own device is still spying.
  - throwaway290 2y ago
    Try to understand how the algorithm works. If you do, please share the mental gymnastics routine you use to make it look like spying. It is not spying in any reasonable sense, definitely not the sense of the proposal by EU lawmakers.
    - Hizonner 2y ago
      It would attempt to identify child porn by scanning images (using both hashes and ML if I remember right, but it doesn't matter). This takes place on the local device, but under control of Apple, not the user. Upon detecting a suspect image, it takes action. The action isn't really part of the "algorithm". Depending on the score, Apple's whim of the day, and/or outside compulsion which would probably be applied to Apple, it refuses to allow the image to be sent in messages or to "the cloud", deletes the image, and/or reports the image to Apple (which would presumably report it further). They can change what it does at any time. In other words, it examines data on your device, on behalf of others, in a way that you don't control, and uses the resulting information to your potential detriment, again on behalf of others and again in a way that you don't control. Apple examining your data in a way you don't control, to implement policy that you don't control and may oppose, is spying. It doesn't matter whether they do the spying using you CPU or their CPU.

benfortuna 2y ago

There is in fact. You could build AI directly into the app to detect inappropriate content and block it. The app has thus not violated end-to-end encryption, nor has that content been exposed to external parties. Platforms could definitely do more.

shinryuu 2y ago
And how do you deal with all false positives. That will just be deemed collateral damage?
- hellojesus 2y ago
  To add to your excellent point, who gets to validate the models efficacy? How do we know the state hadn't trained it to report users talking about maga, or Isreal, or for those with Chinese national lovers?
- throwaway290 2y ago
  How do you deal with all those UUIDs colliding? You can design a system where false positives are so rare they are insignificant and can be properly handled. The only reason we don't is because we don't think it's that much of a problem.
  - Hizonner 2y ago
    No, you can't. Machine learning doesn't work that way. You can keep the false positive level reasonably low for copies of already known and flagged images. You can't do it for new images.
    - throwaway290 2y ago
      Maybe you underestimate machine learning, if you check HN on any given day it can do anything, probably be the next president. But flagged known images was the point of Apple's algo, for example. Still everyone just went "forget abuse, my privacy is more important for me". Really at this point techbros deserved any dumb law that lets the government read their chats.
      - Hizonner 2y ago
        > Maybe you underestimate machine learning, if you check HN on any given day [...] Maybe I actually know something about it. > But flagged known images was the point of Apple's algo, for example. You whined at me a little while ago about how all this was about "abuse, not just porn". Yet you're using that to justify a system that, as you describe it, could only find old, known images that have been circulating around the Internet and made it into a database. Meaning images of past abuse that cannot be prevented, by third parties who would not be caught by this. Pick a threat model, because the measures you defend don't address the threats you claim justify them. ... and if you start talking about "grooming" or "normalization" or other silly bullshit that hypothetically might have a third-order effect, but probably doesn't have any meaningful effect at all in real life, I'm not going to bother to answer it. > Still everyone just went "forget abuse, my privacy is more important for me". Everybody's privacy is important to me. Including the privacy of the children whom you want to have grow up in an ever-expanding panopticon. Because this isn't just about stupid bullshit like your embarrassing disease. It's about people ending up in prison. It's about building infrastructure that can trivially and secretly be repurposed to hurt people, including children, in serious, life-changing, and potentially life-ending ways. The Stasi were not a child-friendly institution.

throwaway290 2y ago

If you put "bad" in quotes, this means you don't think child abuse is actually bad? Cool. > The right answer here is just to accept that, beyond a certain point, further reducing the amount of circulating child porn Not porn, don't delude yourself. Actual abuse. > requires unacceptable tradeoffs And whether tradeoffs are acceptable or not depends on tradeoffs. If you refuse to help find solutions that don't involve spying, don't get all flustered when a law is adopted that does involve spying! > Either your communications are spied on to weed out unapproved material, or they're not. Exactly. Apple's solution did not involve any spying. People have spoken against, now we have solutions that do involve spying. Let's see where this goes yeah?

Hizonner 2y ago
> Apple's solution did not involve any spying. Spying by your own device is still spying.
- throwaway290 2y ago
  Try to understand how the algorithm works. If you do, please share the mental gymnastics routine you use to make it look like spying. It is not spying in any reasonable sense, definitely not the sense of the proposal by EU lawmakers.
  - Hizonner 2y ago
    It would attempt to identify child porn by scanning images (using both hashes and ML if I remember right, but it doesn't matter). This takes place on the local device, but under control of Apple, not the user. Upon detecting a suspect image, it takes action. The action isn't really part of the "algorithm". Depending on the score, Apple's whim of the day, and/or outside compulsion which would probably be applied to Apple, it refuses to allow the image to be sent in messages or to "the cloud", deletes the image, and/or reports the image to Apple (which would presumably report it further). They can change what it does at any time. In other words, it examines data on your device, on behalf of others, in a way that you don't control, and uses the resulting information to your potential detriment, again on behalf of others and again in a way that you don't control. Apple examining your data in a way you don't control, to implement policy that you don't control and may oppose, is spying. It doesn't matter whether they do the spying using you CPU or their CPU.