4 ms·

Let me try to explain it again. Suppose an integrity system has a true positive rate of 99.99%. That would be good enough to deploy right? Except that when appl

Let me try to explain it again. Suppose an integrity system has a true positive rate of 99.99%. That would be good enough to deploy right? Except that when applied to millions of accounts, 0.01% is still a massive number of people. This is even worse when those people are unusual in some way. For example they might open conversations with hundreds of strangers a day for good reasons. But their behaviour is so similar to those of abusive accounts that they get penalised. You might say that maybe 99.99% isn’t good enough and the engineers should try for more 9s. Maybe it’s possible but I don’t know how. If you have ideas on this, please share. Your concerns about different treatment for some people is valid. But again, their experience is different. For example, if an account or content is reported by hundreds of people it’ll be taken down. After all, there’s no reason for accounts in good standing to lie right? Except celebrities often are at the receiving end of such campaigns. There needs to be exceptions so such rules aren’t exploited in such a manner.

YossarianFrPrez 5y ago
>"After all, there's no reason for accounts in good standing to lie, right?" No reason is different from no good reason. Also, 99.99% only seems like a high number when we think anecdotally, not statistically. For anything at Facebook's scale, the number of nines should be increased! Because .01% of the actions of three billion people on a single day gives you a city roughly the size of Tampa, Florida (~300,000 people). Given Facebook's financial resources, it should be no problem to increase the size of the team working on the tool. Like any engineering problem, the problem can be broken down into smaller parts, the edge cases can be caught and/or anticipated, creative solutions can be applied, etc. If history teaches us anything, it's that all public facing systems will be exploited. Those who design them should anticipate this. (Thanks, by the way, for posting about your perspective. It looks very different to those of us on the outside.)
- nindalf 5y ago
  > Given Facebook's financial resources, it should be no problem to increase the size of the team working on the tool. Like any engineering problem, the problem can be broken down into smaller parts And I worked for 4 years on one such small part (internally called UFAC), trying to help potential false positives of such a system. As for classifier with a true positive rate of 99.99999%, I don't know much but I don't think it's possible. But if there's someone out there who might know, then they should say so.
  - YossarianFrPrez 5y ago
    No classification system can be perfect; we are in agreement there. From the outside, it doesn't seem like a strictly data science problem. Instead, it seems like a design / product / brand expectation issue as well.
Ansil849 5y ago
> You might say that maybe 99.99% isn’t good enough and the engineers should try for more 9s. Maybe it’s possible but I don’t know how. If you have ideas on this, please share. Hire a large human moderation team. Facebook can afford to. They choose not to.
- nindalf 5y ago
  > a large human moderation team You're not the only person who has suggested this. Let's think about that for a second. Let's say it takes 6 minutes for a moderation team to review an older account. There's 2 billion accounts, so it'd be good to review all of those. It would take about 200 million hours. Presumably you'd want to re-review positive cases so no moderator has too much power. Additional time. Even if Facebook literally doubled the number of employees, and hired 50000 people overnight, they would still take 2 years to complete the review. But in that time it's possible that previously benign accounts turn abusive. And then think about the 20 million odd new accounts that are created every day. How long before each of those are reviewed? And what signals will you use to review them? These are mostly empty accounts, so there's not much to go on. And that's just the problem of aged fake accounts. How about bullying, harassment, nudity, terrorism, CEI and all the other problems? It's interesting talking to people who say "oh that problem is easy to solve, just do X" without realising that the problem is more complicated than it looks.
  - specialist 5y ago
    Facebook shouldn't be expected to clean up the messes they create because it's hard?
    - robocat 5y ago
      They didn’t say that: none of their comments seem to be defending Facebook. They are giving their opinion that human moderation is not a simple solution. I super appreciate nindalf’s comments here. It is a shame that an ex-developer who knows the problem space and is clearly explaining some if the issues is getting flamed by association.
smsm42 5y ago
> if an account or content is reported by hundreds of people it’ll be taken down. This is extremely simplistic view which is extremely prone to obvious abuse, I really hope FB does much better than this. With obsessive surveillance that FB collects on pretty much everything it can lay their paws on and then some, they could do much better than just counting abuse reports. Much, much better if they really had some smart people - like behavioral science PhDs they surely can afford and they now surely use to figure out how to better sell ads - to work on it for a couple of years, I'm sure they could arrive at something better than "if it gets more than N reports, shut it down". If they don't, it means they don't care enough. > After all, there’s no reason for accounts in good standing to lie right? Of course there is. Politicians lie all the time. MSM lie all the time. Why won't regular people lie all the time? Of course they do, for a myriad of reasons. And since there's no punishment for lying, there's zero incentive for them not to. > There needs to be exceptions so such rules No, there need to be better rules. If you rules suck, and you have to do exceptions for the nobles, they still suck for the peasants. Making it easy for friends of Zuckerberg is not fixing it, it's just sweeping it under the carpet and leaving it to rot.
brianpan 5y ago
It's not a problem that there's a different technical solution to high-profile users. There's no problem with FB hiring more (or more skilled) moderators for higher-profile users. The problem is when the rules are not applied evenly, especially when high profile users with greater audience can abuse those rules.

YossarianFrPrez 5y ago

>"After all, there's no reason for accounts in good standing to lie, right?" No reason is different from no good reason. Also, 99.99% only seems like a high number when we think anecdotally, not statistically. For anything at Facebook's scale, the number of nines should be increased! Because .01% of the actions of three billion people on a single day gives you a city roughly the size of Tampa, Florida (~300,000 people). Given Facebook's financial resources, it should be no problem to increase the size of the team working on the tool. Like any engineering problem, the problem can be broken down into smaller parts, the edge cases can be caught and/or anticipated, creative solutions can be applied, etc. If history teaches us anything, it's that all public facing systems will be exploited. Those who design them should anticipate this. (Thanks, by the way, for posting about your perspective. It looks very different to those of us on the outside.)

nindalf 5y ago
> Given Facebook's financial resources, it should be no problem to increase the size of the team working on the tool. Like any engineering problem, the problem can be broken down into smaller parts And I worked for 4 years on one such small part (internally called UFAC), trying to help potential false positives of such a system. As for classifier with a true positive rate of 99.99999%, I don't know much but I don't think it's possible. But if there's someone out there who might know, then they should say so.
- YossarianFrPrez 5y ago
  No classification system can be perfect; we are in agreement there. From the outside, it doesn't seem like a strictly data science problem. Instead, it seems like a design / product / brand expectation issue as well.

Ansil849 5y ago

> You might say that maybe 99.99% isn’t good enough and the engineers should try for more 9s. Maybe it’s possible but I don’t know how. If you have ideas on this, please share. Hire a large human moderation team. Facebook can afford to. They choose not to.

nindalf 5y ago
> a large human moderation team You're not the only person who has suggested this. Let's think about that for a second. Let's say it takes 6 minutes for a moderation team to review an older account. There's 2 billion accounts, so it'd be good to review all of those. It would take about 200 million hours. Presumably you'd want to re-review positive cases so no moderator has too much power. Additional time. Even if Facebook literally doubled the number of employees, and hired 50000 people overnight, they would still take 2 years to complete the review. But in that time it's possible that previously benign accounts turn abusive. And then think about the 20 million odd new accounts that are created every day. How long before each of those are reviewed? And what signals will you use to review them? These are mostly empty accounts, so there's not much to go on. And that's just the problem of aged fake accounts. How about bullying, harassment, nudity, terrorism, CEI and all the other problems? It's interesting talking to people who say "oh that problem is easy to solve, just do X" without realising that the problem is more complicated than it looks.
- specialist 5y ago
  Facebook shouldn't be expected to clean up the messes they create because it's hard?
  - robocat 5y ago
    They didn’t say that: none of their comments seem to be defending Facebook. They are giving their opinion that human moderation is not a simple solution. I super appreciate nindalf’s comments here. It is a shame that an ex-developer who knows the problem space and is clearly explaining some if the issues is getting flamed by association.

smsm42 5y ago

> if an account or content is reported by hundreds of people it’ll be taken down. This is extremely simplistic view which is extremely prone to obvious abuse, I really hope FB does much better than this. With obsessive surveillance that FB collects on pretty much everything it can lay their paws on and then some, they could do much better than just counting abuse reports. Much, much better if they really had some smart people - like behavioral science PhDs they surely can afford and they now surely use to figure out how to better sell ads - to work on it for a couple of years, I'm sure they could arrive at something better than "if it gets more than N reports, shut it down". If they don't, it means they don't care enough. > After all, there’s no reason for accounts in good standing to lie right? Of course there is. Politicians lie all the time. MSM lie all the time. Why won't regular people lie all the time? Of course they do, for a myriad of reasons. And since there's no punishment for lying, there's zero incentive for them not to. > There needs to be exceptions so such rules No, there need to be better rules. If you rules suck, and you have to do exceptions for the nobles, they still suck for the peasants. Making it easy for friends of Zuckerberg is not fixing it, it's just sweeping it under the carpet and leaving it to rot.

brianpan 5y ago

It's not a problem that there's a different technical solution to high-profile users. There's no problem with FB hiring more (or more skilled) moderators for higher-profile users. The problem is when the rules are not applied evenly, especially when high profile users with greater audience can abuse those rules.