5 ms·

I think the issue is that they could have taken steps to lower their rate, but didn't. There are steps even privacy-conscious companies can take. Like it doesn

I think the issue is that they could have taken steps to lower their rate, but didn't. There are steps even privacy-conscious companies can take. Like it doesn't even have to be E2E messaging, they could check the birthday of people, train an LLM to differentiate appropriate vs inappropriate comments on children's Instagram posts, and use that to help surface problematic comments. They could bring up a popup asking any child to report the conversation which would be triggered by client-side javascript based on a hashed word list of words commonly used by groomers. They could warn the kids whenever they upload images to share directly in DMs rather than publicly on their insta or fb. In a perfect world, the rate of grooming would be zero, but at Meta's scale, just creating even a little more friction that makes it harder to groom kids should be seen as worthwhile.

brucethemoose2 3y ago
> They could check the birthday of people, train an LLM to differentiate appropriate vs inappropriate comments on children's Instagram posts, and use that to help surface problematic comments. They could bring up a popup asking any child to report the conversation which would be triggered by client-side javascript based on a hashed word list of words commonly used by groomers. They could warn the kids whenever they upload images to share directly in DMs rather than publicly on their insta or fb. All this reduces engagement. That is the bottom line. Facebook can do more, but they won't unless something makes them, because it potentially costs billions. In fact, they seem to lobby for this kind of thing... But only if someone makes all their competitors do that thing as well.
jimmygrapes 3y ago
for me the issue is: why? followed by a few more "why?" and once the answer becomes "because." it becomes time to solve those intermediary questions. maybe that happens and nobody tells me, idk
irjustin 3y ago
> I think the issue is that they could have taken steps to lower their rate, but didn't. This is where it gets tough really fast. For arguemnt's sake, let's say semantic analysis can properly identify patterns well at 95% level (oooooffff already can see problems). Public/wall comments on any platform are relatively easy to police. But, I argue the vast majority of problematic grooming comes from DMs. Policing DMs seems worse than trying to do hash analysis of private photos. I don't think there's a win scenario here for anyone involved. [edit] > they could check the birthday of people, yeah because this is answered honestly by predators.
- 15457345234 3y ago
  > Policing DMs seems worse than trying to do hash analysis of private photos. When it comes to DMs between adults and unrelated children, does it really seem worse? Because really what these platforms are enabling is a situation that's never really been common or tolerated before in society, unlimited unsupervised contact and rich (media-wise) communication between unrelated adults and children. These platforms categorically do know when a user is an abuser, potential abuser or has in general been making inappropriate contacts with children and they could easily do something about it.
  - irjustin 3y ago
    > When it comes to DMs between adults and unrelated children, does it really seem worse? I should clarify when I say "worse" i mean "significantly more invasive". Checking if a photo's hash matches another photo's hash, at the very dumbest levels, straight forward, hash == hash or hash is inside larger hash. But for DMs matching words isn't good enough because there are so different ways to say the same things. The system needs to understand when "I really like that top you're wearing" is a cute message between friends vs a predator because who as well as the lines right after really change the conversation. If you don't know the "who" then now we're asking an LLM does this conversation sound predatory!? The machine needs a lot of context to get this right versus photo hashing analysis. THat's why to mean analysis of chats is way "worse" than analysis of photos.
    - 15457345234 3y ago
      > If you don't know the "who" then now we're asking an LLM does this conversation sound predatory!? But you always will know the 'who' - so, you (the platform) knows that it's a conversation between an adult and a child unrelated to them and yes, at this stage, simply asking a LLM 'hey is this conversation sus' will give you results that are more than enough to trigger a human level analysis of the interaction. Also you have to consider that adults and unrelated children generally do not interact at all. So, if you wanted to legitimately try and screen off abusers, 'adults that initiated more than 5 conversations with children unrelated to them' then feeding all those interactions into a LLM classifier would get you 99% of the way there.
numpad0 3y ago
> train an LLM to differentiate appropriate vs inappropriate comments > popup asking ... which would be triggered by client-side javascript based on a hashed word Yeah, this is the feature that I think I should build my own social media for. Owning "the algorithm" and using it to indoctrinate users en masse with my own view. Are you sure you want to insist it's in any way or form even remotely ethical thing to do?

brucethemoose2 3y ago

> They could check the birthday of people, train an LLM to differentiate appropriate vs inappropriate comments on children's Instagram posts, and use that to help surface problematic comments. They could bring up a popup asking any child to report the conversation which would be triggered by client-side javascript based on a hashed word list of words commonly used by groomers. They could warn the kids whenever they upload images to share directly in DMs rather than publicly on their insta or fb. All this reduces engagement. That is the bottom line. Facebook can do more, but they won't unless something makes them, because it potentially costs billions. In fact, they seem to lobby for this kind of thing... But only if someone makes all their competitors do that thing as well.

jimmygrapes 3y ago

for me the issue is: why? followed by a few more "why?" and once the answer becomes "because." it becomes time to solve those intermediary questions. maybe that happens and nobody tells me, idk

irjustin 3y ago

> I think the issue is that they could have taken steps to lower their rate, but didn't. This is where it gets tough really fast. For arguemnt's sake, let's say semantic analysis can properly identify patterns well at 95% level (oooooffff already can see problems). Public/wall comments on any platform are relatively easy to police. But, I argue the vast majority of problematic grooming comes from DMs. Policing DMs seems worse than trying to do hash analysis of private photos. I don't think there's a win scenario here for anyone involved. [edit] > they could check the birthday of people, yeah because this is answered honestly by predators.

15457345234 3y ago
> Policing DMs seems worse than trying to do hash analysis of private photos. When it comes to DMs between adults and unrelated children, does it really seem worse? Because really what these platforms are enabling is a situation that's never really been common or tolerated before in society, unlimited unsupervised contact and rich (media-wise) communication between unrelated adults and children. These platforms categorically do know when a user is an abuser, potential abuser or has in general been making inappropriate contacts with children and they could easily do something about it.
- irjustin 3y ago
  > When it comes to DMs between adults and unrelated children, does it really seem worse? I should clarify when I say "worse" i mean "significantly more invasive". Checking if a photo's hash matches another photo's hash, at the very dumbest levels, straight forward, hash == hash or hash is inside larger hash. But for DMs matching words isn't good enough because there are so different ways to say the same things. The system needs to understand when "I really like that top you're wearing" is a cute message between friends vs a predator because who as well as the lines right after really change the conversation. If you don't know the "who" then now we're asking an LLM does this conversation sound predatory!? The machine needs a lot of context to get this right versus photo hashing analysis. THat's why to mean analysis of chats is way "worse" than analysis of photos.
  - 15457345234 3y ago
    > If you don't know the "who" then now we're asking an LLM does this conversation sound predatory!? But you always will know the 'who' - so, you (the platform) knows that it's a conversation between an adult and a child unrelated to them and yes, at this stage, simply asking a LLM 'hey is this conversation sus' will give you results that are more than enough to trigger a human level analysis of the interaction. Also you have to consider that adults and unrelated children generally do not interact at all. So, if you wanted to legitimately try and screen off abusers, 'adults that initiated more than 5 conversations with children unrelated to them' then feeding all those interactions into a LLM classifier would get you 99% of the way there.

numpad0 3y ago

> train an LLM to differentiate appropriate vs inappropriate comments > popup asking ... which would be triggered by client-side javascript based on a hashed word Yeah, this is the feature that I think I should build my own social media for. Owning "the algorithm" and using it to indoctrinate users en masse with my own view. Are you sure you want to insist it's in any way or form even remotely ethical thing to do?