4 ms·

There is in fact. You could build AI directly into the app to detect inappropriate content and block it. The app has thus not violated end-to-end encryption, n

There is in fact. You could build AI directly into the app to detect inappropriate content and block it. The app has thus not violated end-to-end encryption, nor has that content been exposed to external parties. Platforms could definitely do more.

shinryuu 2y ago
And how do you deal with all false positives. That will just be deemed collateral damage?
- hellojesus 2y ago
  To add to your excellent point, who gets to validate the models efficacy? How do we know the state hadn't trained it to report users talking about maga, or Isreal, or for those with Chinese national lovers?
- throwaway290 2y ago
  How do you deal with all those UUIDs colliding? You can design a system where false positives are so rare they are insignificant and can be properly handled. The only reason we don't is because we don't think it's that much of a problem.
  - Hizonner 2y ago
    No, you can't. Machine learning doesn't work that way. You can keep the false positive level reasonably low for copies of already known and flagged images. You can't do it for new images.
    - throwaway290 2y ago
      Maybe you underestimate machine learning, if you check HN on any given day it can do anything, probably be the next president. But flagged known images was the point of Apple's algo, for example. Still everyone just went "forget abuse, my privacy is more important for me". Really at this point techbros deserved any dumb law that lets the government read their chats.
      - Hizonner 2y ago
        > Maybe you underestimate machine learning, if you check HN on any given day [...] Maybe I actually know something about it. > But flagged known images was the point of Apple's algo, for example. You whined at me a little while ago about how all this was about "abuse, not just porn". Yet you're using that to justify a system that, as you describe it, could only find old, known images that have been circulating around the Internet and made it into a database. Meaning images of past abuse that cannot be prevented, by third parties who would not be caught by this. Pick a threat model, because the measures you defend don't address the threats you claim justify them. ... and if you start talking about "grooming" or "normalization" or other silly bullshit that hypothetically might have a third-order effect, but probably doesn't have any meaningful effect at all in real life, I'm not going to bother to answer it. > Still everyone just went "forget abuse, my privacy is more important for me". Everybody's privacy is important to me. Including the privacy of the children whom you want to have grow up in an ever-expanding panopticon. Because this isn't just about stupid bullshit like your embarrassing disease. It's about people ending up in prison. It's about building infrastructure that can trivially and secretly be repurposed to hurt people, including children, in serious, life-changing, and potentially life-ending ways. The Stasi were not a child-friendly institution.
        throwaway290 2y ago
        > Maybe I actually know something about it. If you know then you'd agree that with the right setup ML can do this with a very high precision. We're talking about a highly customized system trained exactly for this one purpose not some chatbot. > You whined at me a little while ago You're the one whining here buddy-- remember this is about a law about to be forced on you that you find inconvenient ;) I find it suboptimal but in some sense it might be better than nothing. > this was about "abuse, not just porn". These are related. If you have this material, you obtained it from somewhere even if you didn't make it yourself. Some police work and it may lead to some dark web exchange marketplace and actual producers. That said yes, there's difference. The EU law being discussed is probably more fit to counter realtime abuse, compared to Apple's algo for example. > Because this isn't just about stupid bullshit like your embarrassing disease. It's about people ending up in prison. I actually agree with these two sentences, but not in the way you probably intended. > The Stasi were not a child-friendly institution. I was waiting until Hitler gets invoked in a discussion about using tech to combat and prevent child abuse facilitated by tech, I was not disappointed.

shinryuu 2y ago

And how do you deal with all false positives. That will just be deemed collateral damage?

hellojesus 2y ago
To add to your excellent point, who gets to validate the models efficacy? How do we know the state hadn't trained it to report users talking about maga, or Isreal, or for those with Chinese national lovers?
throwaway290 2y ago
How do you deal with all those UUIDs colliding? You can design a system where false positives are so rare they are insignificant and can be properly handled. The only reason we don't is because we don't think it's that much of a problem.
- Hizonner 2y ago
  No, you can't. Machine learning doesn't work that way. You can keep the false positive level reasonably low for copies of already known and flagged images. You can't do it for new images.
  - throwaway290 2y ago
    Maybe you underestimate machine learning, if you check HN on any given day it can do anything, probably be the next president. But flagged known images was the point of Apple's algo, for example. Still everyone just went "forget abuse, my privacy is more important for me". Really at this point techbros deserved any dumb law that lets the government read their chats.
    - Hizonner 2y ago
      > Maybe you underestimate machine learning, if you check HN on any given day [...] Maybe I actually know something about it. > But flagged known images was the point of Apple's algo, for example. You whined at me a little while ago about how all this was about "abuse, not just porn". Yet you're using that to justify a system that, as you describe it, could only find old, known images that have been circulating around the Internet and made it into a database. Meaning images of past abuse that cannot be prevented, by third parties who would not be caught by this. Pick a threat model, because the measures you defend don't address the threats you claim justify them. ... and if you start talking about "grooming" or "normalization" or other silly bullshit that hypothetically might have a third-order effect, but probably doesn't have any meaningful effect at all in real life, I'm not going to bother to answer it. > Still everyone just went "forget abuse, my privacy is more important for me". Everybody's privacy is important to me. Including the privacy of the children whom you want to have grow up in an ever-expanding panopticon. Because this isn't just about stupid bullshit like your embarrassing disease. It's about people ending up in prison. It's about building infrastructure that can trivially and secretly be repurposed to hurt people, including children, in serious, life-changing, and potentially life-ending ways. The Stasi were not a child-friendly institution.
      - throwaway290 2y ago
        > Maybe I actually know something about it. If you know then you'd agree that with the right setup ML can do this with a very high precision. We're talking about a highly customized system trained exactly for this one purpose not some chatbot. > You whined at me a little while ago You're the one whining here buddy-- remember this is about a law about to be forced on you that you find inconvenient ;) I find it suboptimal but in some sense it might be better than nothing. > this was about "abuse, not just porn". These are related. If you have this material, you obtained it from somewhere even if you didn't make it yourself. Some police work and it may lead to some dark web exchange marketplace and actual producers. That said yes, there's difference. The EU law being discussed is probably more fit to counter realtime abuse, compared to Apple's algo for example. > Because this isn't just about stupid bullshit like your embarrassing disease. It's about people ending up in prison. I actually agree with these two sentences, but not in the way you probably intended. > The Stasi were not a child-friendly institution. I was waiting until Hitler gets invoked in a discussion about using tech to combat and prevent child abuse facilitated by tech, I was not disappointed.