4 ms·

> Policing DMs seems worse than trying to do hash analysis of private photos. When it comes to DMs between adults and unrelated children, does it really seem w

> Policing DMs seems worse than trying to do hash analysis of private photos. When it comes to DMs between adults and unrelated children, does it really seem worse? Because really what these platforms are enabling is a situation that's never really been common or tolerated before in society, unlimited unsupervised contact and rich (media-wise) communication between unrelated adults and children. These platforms categorically do know when a user is an abuser, potential abuser or has in general been making inappropriate contacts with children and they could easily do something about it.

irjustin 3y ago
> When it comes to DMs between adults and unrelated children, does it really seem worse? I should clarify when I say "worse" i mean "significantly more invasive". Checking if a photo's hash matches another photo's hash, at the very dumbest levels, straight forward, hash == hash or hash is inside larger hash. But for DMs matching words isn't good enough because there are so different ways to say the same things. The system needs to understand when "I really like that top you're wearing" is a cute message between friends vs a predator because who as well as the lines right after really change the conversation. If you don't know the "who" then now we're asking an LLM does this conversation sound predatory!? The machine needs a lot of context to get this right versus photo hashing analysis. THat's why to mean analysis of chats is way "worse" than analysis of photos.
- 15457345234 3y ago
  > If you don't know the "who" then now we're asking an LLM does this conversation sound predatory!? But you always will know the 'who' - so, you (the platform) knows that it's a conversation between an adult and a child unrelated to them and yes, at this stage, simply asking a LLM 'hey is this conversation sus' will give you results that are more than enough to trigger a human level analysis of the interaction. Also you have to consider that adults and unrelated children generally do not interact at all. So, if you wanted to legitimately try and screen off abusers, 'adults that initiated more than 5 conversations with children unrelated to them' then feeding all those interactions into a LLM classifier would get you 99% of the way there.
  - irjustin 3y ago
    Wait how do you know who? Whatsapp you got phone number, Facebook you've got.... an email address[0] and possibly a phone number.... [0] https://www.facebook.com/help/188157731232424 https://www.facebook.com/help/188157731232424
    - 15457345234 3y ago
      > Facebook you've got.... an email address[0] and possibly a phone number.... This doesn't seem like a conversation that's being had in good faith.

irjustin 3y ago

> When it comes to DMs between adults and unrelated children, does it really seem worse? I should clarify when I say "worse" i mean "significantly more invasive". Checking if a photo's hash matches another photo's hash, at the very dumbest levels, straight forward, hash == hash or hash is inside larger hash. But for DMs matching words isn't good enough because there are so different ways to say the same things. The system needs to understand when "I really like that top you're wearing" is a cute message between friends vs a predator because who as well as the lines right after really change the conversation. If you don't know the "who" then now we're asking an LLM does this conversation sound predatory!? The machine needs a lot of context to get this right versus photo hashing analysis. THat's why to mean analysis of chats is way "worse" than analysis of photos.

15457345234 3y ago
> If you don't know the "who" then now we're asking an LLM does this conversation sound predatory!? But you always will know the 'who' - so, you (the platform) knows that it's a conversation between an adult and a child unrelated to them and yes, at this stage, simply asking a LLM 'hey is this conversation sus' will give you results that are more than enough to trigger a human level analysis of the interaction. Also you have to consider that adults and unrelated children generally do not interact at all. So, if you wanted to legitimately try and screen off abusers, 'adults that initiated more than 5 conversations with children unrelated to them' then feeding all those interactions into a LLM classifier would get you 99% of the way there.
- irjustin 3y ago
  Wait how do you know who? Whatsapp you got phone number, Facebook you've got.... an email address[0] and possibly a phone number.... [0] https://www.facebook.com/help/188157731232424 https://www.facebook.com/help/188157731232424
  - 15457345234 3y ago
    > Facebook you've got.... an email address[0] and possibly a phone number.... This doesn't seem like a conversation that's being had in good faith.