3 ms·

> nobody actually wanted to foster such toxic behavior I think no one realized you could make disgusting amounts of money by fostering it. There have been plen

by dvt 6y ago

> nobody actually wanted to foster such toxic behavior I think no one realized you could make disgusting amounts of money by fostering it. There have been plenty of flame wars on BBSes and forums, but the tension between "engagement" and "quality" always favored the latter in small, private communities (HN is a good example). However, when it comes to Facebook and Twitter, the former is always favored (due to market forces, shareholder interests, etc.).

jerf 6y ago
"BBSes and forums" Observe flamewars on BBSs and forums tended to drive people away, thus tending to confine them through natural mechanisms. Our current systems encourage and fan them, and by profiting from them, create a system that sustains them indefinitely instead of naturally isolating them to just the people who actively want to participate. As someone who has a low-grade hobby of kind of keeping track of this sort of thing, I think one of the major challenges of trying to structure communities is all the most powerful drivers of what is going to happen occurs in these low-level decisions that create persistent second-order effects on the community's nature, and this is one of the somewhat unusual cases where the second-order effects utterly dominate the first-order effects. In this case, you can read "first-order effects" as "the things the developers intended to happen", which I think are dominated by the structural effects of decisions that aren't intentional. If you create a community that literally profits from conflict and flamewar, you'll never be able to fix them by any number of systems you try to add to fix that via first-order effects. The underlying structure is too powerful. Facebook can't be fixed with any amount of "moderation"; the entire structure, from the very foundation, is incapable of supporting a friendly community at scale. Until Facebook stops profiting from engagement, they will never be able to fix their problems with toxicity, no matter how many resources they pour into naively fighting it via direct fixups. (Now, I have questions about whether a friendly community of the scale of Facebook is even possible: https://news.ycombinator.com/item?id=20146868 https://news.ycombinator.com/item?id=20146868 . But the fact it may not even be possible does not mean that Facebook is still obviously not that solution.)
- Invictus0 6y ago
  I kinda wonder about just having a no politics policy. Allow user reports... If N users report a post for being political, or it triggers some regex, penalize the post in some way. Facebook doesn't have to be about politics; Instagram wasn't for a long time, but with the onset of the BLM movement, I've seen my timeline filled to the brim with politics, and at one point, people were even saying that no one was even allowed to post non-political content because it shows how privileged you are to be ignoring the movement. I don't use Facebook for the politics... I just want to know what's going on in my friends lives. You can still have engagement without politics.
  - Nextgrid 6y ago
    Ultimately, politics aren't the issue. Not having clear, consistent & enforced rules as well as no consequences for breaking them is the problem. People aren't encouraged to think twice before they post because there's not going to be any significant consequences for breaking the rules. Even if you somehow manage to get permanently banned from a social network, it's very easy to come back; it doesn't cost anything besides spending some time creating a new account. From a business perspective it makes sense - why would you ban an abusive user that makes you money? Just give them a slap on the wrist to pretend that you want to discourage bad behavior and keep collecting their money. Proper enforcement of the rules with significant consequences when broken (losing the account, and new accounts cost $$$ to register) would discourage a lot of bad behavior to begin with. You could then introduce a karma/reputation system to 1) attach even more value to accounts (you wouldn't want to lose an account it took you years to level up and gain access to exclusive privileges) and 2) allow "trusted" users beyond a certain reputation level to participate in moderation, prioritizing reports from those people and automatically hiding content reported by them pending human review (with appropriate sanctions if the report was made in bad faith) to quickly take down offensive content.
  - zzzeek 6y ago
    you can't solve social problems with technology, only policy. companies like Facebook should be broken up and regulated such that the whole model of profiting from social division is removed. this would be highly beneficial to society and is an appropriate role of government.

by dvt 6y ago

jerf 6y ago

"BBSes and forums" Observe flamewars on BBSs and forums tended to drive people away, thus tending to confine them through natural mechanisms. Our current systems encourage and fan them, and by profiting from them, create a system that sustains them indefinitely instead of naturally isolating them to just the people who actively want to participate. As someone who has a low-grade hobby of kind of keeping track of this sort of thing, I think one of the major challenges of trying to structure communities is all the most powerful drivers of what is going to happen occurs in these low-level decisions that create persistent second-order effects on the community's nature, and this is one of the somewhat unusual cases where the second-order effects utterly dominate the first-order effects. In this case, you can read "first-order effects" as "the things the developers intended to happen", which I think are dominated by the structural effects of decisions that aren't intentional. If you create a community that literally profits from conflict and flamewar, you'll never be able to fix them by any number of systems you try to add to fix that via first-order effects. The underlying structure is too powerful. Facebook can't be fixed with any amount of "moderation"; the entire structure, from the very foundation, is incapable of supporting a friendly community at scale. Until Facebook stops profiting from engagement, they will never be able to fix their problems with toxicity, no matter how many resources they pour into naively fighting it via direct fixups. (Now, I have questions about whether a friendly community of the scale of Facebook is even possible: https://news.ycombinator.com/item?id=20146868 https://news.ycombinator.com/item?id=20146868 . But the fact it may not even be possible does not mean that Facebook is still obviously not that solution.)

Invictus0 6y ago
I kinda wonder about just having a no politics policy. Allow user reports... If N users report a post for being political, or it triggers some regex, penalize the post in some way. Facebook doesn't have to be about politics; Instagram wasn't for a long time, but with the onset of the BLM movement, I've seen my timeline filled to the brim with politics, and at one point, people were even saying that no one was even allowed to post non-political content because it shows how privileged you are to be ignoring the movement. I don't use Facebook for the politics... I just want to know what's going on in my friends lives. You can still have engagement without politics.
- Nextgrid 6y ago
  Ultimately, politics aren't the issue. Not having clear, consistent & enforced rules as well as no consequences for breaking them is the problem. People aren't encouraged to think twice before they post because there's not going to be any significant consequences for breaking the rules. Even if you somehow manage to get permanently banned from a social network, it's very easy to come back; it doesn't cost anything besides spending some time creating a new account. From a business perspective it makes sense - why would you ban an abusive user that makes you money? Just give them a slap on the wrist to pretend that you want to discourage bad behavior and keep collecting their money. Proper enforcement of the rules with significant consequences when broken (losing the account, and new accounts cost $$$ to register) would discourage a lot of bad behavior to begin with. You could then introduce a karma/reputation system to 1) attach even more value to accounts (you wouldn't want to lose an account it took you years to level up and gain access to exclusive privileges) and 2) allow "trusted" users beyond a certain reputation level to participate in moderation, prioritizing reports from those people and automatically hiding content reported by them pending human review (with appropriate sanctions if the report was made in bad faith) to quickly take down offensive content.
- zzzeek 6y ago
  you can't solve social problems with technology, only policy. companies like Facebook should be broken up and regulated such that the whole model of profiting from social division is removed. this would be highly beneficial to society and is an appropriate role of government.