3 ms·
The other problem is that the business model of social media is based on generating "engagement" at all costs, so the platforms are built to encourage outrage a
by Nextgrid 6y ago
The other problem is that the business model of social media is based on generating "engagement" at all costs, so the platforms are built to encourage outrage as it generates lots of engagement, among other addictive behaviors (the infinite "algorithmic" feed for example). Social media was supposed to be a tool that serves people but its current business model encourages it to work against people.
There were plenty of other technologies that could've been used to organize large-scale virtual fight clubs (forums, BBSes, chatrooms, maybe even the telephone) but this didn't happen because nobody actually wanted to foster such toxic behavior.
- dvt 6y ago> nobody actually wanted to foster such toxic behavior I think no one realized you could make disgusting amounts of money by fostering it. There have been plenty of flame wars on BBSes and forums, but the tension between "engagement" and "quality" always favored the latter in small, private communities (HN is a good example). However, when it comes to Facebook and Twitter, the former is always favored (due to market forces, shareholder interests, etc.).
- jerf 6y ago"BBSes and forums" Observe flamewars on BBSs and forums tended to drive people away, thus tending to confine them through natural mechanisms. Our current systems encourage and fan them, and by profiting from them, create a system that sustains them indefinitely instead of naturally isolating them to just the people who actively want to participate. As someone who has a low-grade hobby of kind of keeping track of this sort of thing, I think one of the major challenges of trying to structure communities is all the most powerful drivers of what is going to happen occurs in these low-level decisions that create persistent second-order effects on the community's nature, and this is one of the somewhat unusual cases where the second-order effects utterly dominate the first-order effects. In this case, you can read "first-order effects" as "the things the developers intended to happen", which I think are dominated by the structural effects of decisions that aren't intentional. If you create a community that literally profits from conflict and flamewar, you'll never be able to fix them by any number of systems you try to add to fix that via first-order effects. The underlying structure is too powerful. Facebook can't be fixed with any amount of "moderation"; the entire structure, from the very foundation, is incapable of supporting a friendly community at scale. Until Facebook stops profiting from engagement, they will never be able to fix their problems with toxicity, no matter how many resources they pour into naively fighting it via direct fixups. (Now, I have questions about whether a friendly community of the scale of Facebook is even possible: https://news.ycombinator.com/item?id=20146868 https://news.ycombinator.com/item?id=20146868 . But the fact it may not even be possible does not mean that Facebook is still obviously not that solution.)
- Invictus0 6y agoI kinda wonder about just having a no politics policy. Allow user reports... If N users report a post for being political, or it triggers some regex, penalize the post in some way. Facebook doesn't have to be about politics; Instagram wasn't for a long time, but with the onset of the BLM movement, I've seen my timeline filled to the brim with politics, and at one point, people were even saying that no one was even allowed to post non-political content because it shows how privileged you are to be ignoring the movement. I don't use Facebook for the politics... I just want to know what's going on in my friends lives. You can still have engagement without politics.
- Nextgrid 6y agoUltimately, politics aren't the issue. Not having clear, consistent & enforced rules as well as no consequences for breaking them is the problem. People aren't encouraged to think twice before they post because there's not going to be any significant consequences for breaking the rules. Even if you somehow manage to get permanently banned from a social network, it's very easy to come back; it doesn't cost anything besides spending some time creating a new account. From a business perspective it makes sense - why would you ban an abusive user that makes you money? Just give them a slap on the wrist to pretend that you want to discourage bad behavior and keep collecting their money. Proper enforcement of the rules with significant consequences when broken (losing the account, and new accounts cost $$$ to register) would discourage a lot of bad behavior to begin with. You could then introduce a karma/reputation system to 1) attach even more value to accounts (you wouldn't want to lose an account it took you years to level up and gain access to exclusive privileges) and 2) allow "trusted" users beyond a certain reputation level to participate in moderation, prioritizing reports from those people and automatically hiding content reported by them pending human review (with appropriate sanctions if the report was made in bad faith) to quickly take down offensive content.
- zzzeek 6y agoyou can't solve social problems with technology, only policy. companies like Facebook should be broken up and regulated such that the whole model of profiting from social division is removed. this would be highly beneficial to society and is an appropriate role of government.
- montagg 6y agoThey certainly fostered toxic behavior. The social interactions we see on social media—even in HN from time to time—are familiar to anyone who frequented forums in the past. The ad-supported business model industrialized it, though.
- Nextgrid 6y agoMy point is that in most cases it wasn't the desired behavior, just sometimes tolerated if they were high-profile users with connections with the moderation team and/or otherwise provided valuable contributions.
- acdha 6y ago> There were plenty of other technologies that could've been used to organize large-scale virtual fight clubs (forums, BBSes, chatrooms, maybe even the telephone) but this didn't happen because nobody actually wanted to foster such toxic behavior. “Nobody” is leaving out some pretty notorious sources of toxicity (e.g. 4chan). I think a key difference is that these huge platforms dramatically increase the reach of those communities by giving them much better tools, highly-available servers, etc. and in particular mainstreaming them into the same place everyone else is, making it easier to recruit and share outside those communities. In the forum era, people had to learn about a particular site, learn the community, maybe create an account, etc. to know these existed — now it's just one Facebook share away and there's an advanced “engagement” system ensuring that anyone who likes something widely shared will continue to see other content from the same source without needing to seek it out. Brigading was most noticeable with a wave new of new accounts and there wasn't something like an ML system making that activity drive unrelated users to see it.
- Retric 6y ago4Chan is only 4 months older than Facebook and is hardly that profitable. Which is what changed, online dumpster fires where not attractive to advertisers but add a veneer of social networks and some basic location / demographic data and suddenly things change.
- acdha 6y agoFacebook really had two periods because it was limited scale prior to mid-2006, when the number of users grew dramatically. I think the more important point is really the advertising angle: taking a bunch of small communities and giving them high-quality hosting powered by a billion dollar company along with a pre-developed audience.
- shadowgovt 6y agoAnd some moderation. 4chan wasn't ever going to attract ad revenue because nobody at Coca-Cola wants their brand run alongside a goatse link. But massive political fights aren't something that the advertisers (until recently) see as bad imagery for their brands.
- rmrfstar 6y ago> the business model of social media is based on generating "engagement" at all costs I haven't seen evidence that outrage driven engagement is profitable, though the statement is frequently repeated. Is it possible that divisiveness is an intended effect? We know that governments sow discord to achieve selfish ends. Why wouldn't similarly powerful business interests?
- Nextgrid 6y ago> I haven't seen evidence that outrage driven engagement is profitable, though the statement is frequently repeated. Let's assume that you are able to create a social network the scale of Facebook and network effects aren't an issue (let's imagine a magical solution that interoperates with existing social networks in such a way that accounts work on either side and content is visible from both sides) with the caveat that you instantly ban accounts that participate in political flamewars, intentionally spread misinformation/fake news, etc. You're going to end up banning a significant chunk of people that would otherwise make you money if you just turned a blind eye to the problem like current social networks do.
- rmrfstar 6y agoIs it true that failing to ban is the source of the divisiveness? I haven't used Facebook before, but it seems like the mechanism is the feed, which prioritizes divisive content. Even as a relatively senior employee, you may be unable to discern intent. You might just get an incentive scheme that compensates you for engagement rather than profitable engagement.
- Nextgrid 6y agoThe feed brings visibility to offensive content, but ultimately someone has to create that content in the first place. Even if you kept the feed as-is, as long as you had bulletproof moderation that would nuke any offensive content (or other mechanisms that are effective at discouraging people from posting such content), there wouldn't be any bad content for the feed to recommend.
- Lendal 6y agoYep, generates outrage, feeds on it and turns it into money. The added insult is that advertising money used to be used to fund investigative journalism and real community news coverage at local newspapers in every city and town in the US. Anyone know what that money is spent on now?
- Balgair 6y ago> The other problem is that the business model of social media is based on generating "engagement" at all costs (emphasis my own) Well, FB and the like have been bringing the ban hammer out a fair bit. Most recently was the QAnon kiddos. If they really were keeping engagement up at all costs then banning a lot of accounts and dumping all that hand-wringing would be a strange thing to do time and time again.