5 ms·

I've talked about this somewhere else. What you're seeing is just a manifestation of what I call "the fundamental problem of user created content." Said problem

I've talked about this somewhere else. What you're seeing is just a manifestation of what I call "the fundamental problem of user created content." Said problem is that warehousing and distributing content scales insanely well but curation does not. Until we have strong AI, curation is a manual process. Moderators just aren't efficient enough at processing content for their output to pay for a full time salary. You can cut costs by making end users into moderators (the Reddit model) but results may vary. This problem applies to other forms of sites and content as well. The app store gets hundreds of submissions in the game category per day. Hosting and distributing that content is easy. Only a tiny fraction of those games are going to be played and rated enough times to show up in a recommendation engine. The bulk of the incoming content stream isn't being matched to interested people at all (sorting by interesting is the same curation issue as filtering by offensive). Literally every content platform is going to have some problem similar to Facebook. We can blame Facebook, but the reality is no one has a good solution. Not even me.

hashkb 7y ago
> We can blame Facebook, but the reality is no one has a good solution. We can take a step backwards and eliminate Facebook, Insta, etc from our societies and enact legislation that will govern the second generation of social networks.
- Jedi72 7y ago
  What legislation? Specifically? What do you want to make law, that they have to hire moderators (which they already do?) or do you want to make it illegal to post "bad things" on the internet?
  - hashkb 7y ago
    Let me start by saying that I am not running for elected office because it's a really hard job; even though I might believe I would do it with more integrity and rationality than the next person, I'm also smart enough to realize that everyone thinks that way and I'd probably end up within a standard deviation of average corruption. What I'm saying, seriously, is that we need a society-wide blameless post-mortem on social networks. It needs to be a slow, careful discussion, where all the stakeholders have their voices heard, and we decide what is good for us all. I don't know that we need them at all, I'm not sure they provide ANY value to ANYONE, but as a non-user I'd of course be open to persuasion by current victims. Erm, users. In the meantime, it should be illegal to operate a social network. Nobody needs to go to prison yet, even though I think it's horrifyingly clear at this point that the ethics are out the window. One obvious one: advertising and public conversation must always be separated; the same way no public schoolteacher may read scripture in class, it should be illegal for an internet service that hosts public conversation (e.g. Twitter) to allow sponsored content. We should also establish guidelines for addiction. After cigarettes, drugs, sugar, etc; we should as a society be prepared to understand the various forms that addictive products can take and regulate them aggressively before they become serious problems. UX patterns like infinite scroll, pull-to-refresh, and push notifications are particularly suspect. I also think we should close the apparent loophole in COPPA that allows parents to post photos of their children on social networks. > that they have to hire moderators (which they already do?) or do you want to make it illegal to post "bad things" on the internet? I tend to come down on the "free speech" side of these issues as much as possible; I would prefer unregulated public fora that (perhaps by requiring identity verification) encouraged good-faith participation in substantial discussions. I think if you are just fooling around, maybe you should head down to the bar and get drunk with your friends and do your shit talking there.
    - dragonwriter 7y ago
      > What I'm saying, seriously, is that we need a society-wide blameless post-mortem on social networks Social networks aren't dead, so we can't have a post-mortem. Also, you can't have a society-wide blameless analytical conversation about anything, especially if there are significant conflicting interests involved; that's not how people work. (It's perhaps noteworthy that even while suggesting this you offer no objective descriptions of impacts with reasoned analysis of contributing factors, instead jumping straight to blame and policy responses, with some labelled as “obvious”.)
    - closeparen 7y ago
      The point of free speech is that it’s not regulated according to an opinion about “what is good for us all” or what is “needed.”
    - jki275 7y ago
      You claim to be for free speech, yet most of what you propose directly infringes on the concept. You don’t get to determine who gets to say what.
    - Macross8299 7y ago
      >In the meantime, it should be illegal to operate a social network So what about the countless number of people whose livelihoods depend on social media in some capacity? They're supposed to just be fine with being irrevocably fucked over until the moral panic about social media subsides? (or, in your words, "we decide what is good for us all") Where does the small business owner who depends on a social network factor in as a "stakeholder"? Drugs, sugar, cigarettes are measurably and objectively harmful to your health and that's why they are regulated (or should be). There isn't similar comparable scientific evidence that social media is nearly as harmful except for questionable non-reproducible psychology studies, so I don't think it's comparable at all.
      - hashkb 7y ago
        I hear you saying you don't believe social media is addictive. Will you please kindly open the "digital wellbeing" app (or whatever it's called on iOS) on your phone and share the number of hours you've used social networks over the last week?
  - Spooky23 7y ago
    I would start with a few things: - Make social networks mandated reporters for suspected child abuse. Like teachers or little league coaches, reporting would absolve them of liability. - Prohibit generation of profit from illegal activity. - Require transparency in political advertising, with an accountable entity and individual listed and publicly available. - Require reporting of funding source and material individuals for paid political advertising. - Make the platform liable for false product claims in cases where the platform facilitates direct or implied endorsements for a product. (“Jedi72 likes this homeopathic cancer treatment!” - Make the platform responsible for reasonable efforts to block and report on actions against fictitious users posing as people. Share liability for any claims, including libel, resulting from the actions of fictitious people. A regulatory regime like this would either improve the signal/noise ratio or cause the company to abandon or prohibit certain actions or functions.
- 1stcity3rdcoast 7y ago
  Ah, yes, Congress — the most competent subject matter experts regarding the internet and technology.
- Chazprime 7y ago
  Well, to do that you’d need a congress that has the first clue what social media even is, and I’m guessing the US is easily 5-10 years away from that.
- ummwhat 7y ago
  How does that solve the problem that the incoming content stream flows faster than any curation mechanism we can economically build today?
  - joeblubaugh 7y ago
    Well, first you slow the content stream by imposing a rate limit per-user, and users who post things that violate the policy are stopped from posting for a longer time. Second, you do better at automatically detecting duplicate content that’s already been banned.
rhizome 7y ago
>Said problem is that warehousing and distributing content scales insanely well but curation does not I don't think it's a problem any more than weather is a "problem" that required the invention of the roof. As far as user-generated content goes, it's easy to invent the hose, but the faucet is much more complex. It's irresponsible to provide only acceleration ("reach," "virality," etc.) for information without also implementing brakes. Moderation has been a known fix for decades, but people like Zuck -- in disposition and inexperience -- won't launch anything human-powered, and moderation requires a human eye. Fuck us, right?
- ummwhat 7y ago
  I think you're view will change when you start putting numbers on it. I'm going to talk about the games in the app store example because it's the instance of the problem that I first thought about and it is easier to put a monetary value on an app than on a comment. Let's suppose Apple wants to make sure every game in their store gets at least 5 ratings. To do this they are going to pay people to play each game for 5 min and then assign a rating. In an hour a curator can get through 12 games. Over an 8 hour day thats 96 games. Apple would need to hire about 7 (iirc) full time employees just to give every game one rating. We wanted every game to have 5 ratings. Assuming that users on average provide one rating (but sometimes 0), we still need curators to provide the other 4. That means Apple would be employing 28 full time employees just to get their content minimally curated. Let's use the salary from the article and say each curator costs 30k a year. That means this whole department of curation is going to cost about 900k a year. Most of what they are sorting is going to be shovelware anyway. To justify the cost of this department, the curators need to find gems, content that's really good but would have otherwise been looked over. They need to find 900k of gems per year. Considering the average earning of an app, this is just barely plausible. If you want to do anything more advanced with your curation, like have the curators generate key words or figure out which demographic might like it, your budget is blown for sure. IDK how the economics of comment curation works out, but I bet it's even worse. After all, no one is even paying for comments, and making a comment is way less effort than making an app.
  - BRAlNlAC 7y ago
ngold 7y ago
15 billion dollars in pure profit. You could pay a living wage with health benefits, or you could hire twice as many people and have everyone work less while paying more. This is just Facebook being cheap.
- nojvek 7y ago
  Something something. Corporations are hive mind AI optimizing for growth and profit. They don’t give a shit. Amazon doesn’t give a shit unless it impacts the bottom line. Google barely has support and YouTube kids is full of questionable content.
- solstice 7y ago
  The image of the Chernobyl liquidators just popped into my mind, each worker only allowed to shovel radiative shit for 90 seconds before having a mandatory time out. Treating content moderators in a similar way seems like a step into the right direction from a mental health perspective
  - Meskupie 7y ago
    The largest impact events will likely be the smaller percentage of posts that really resonate with the moderator. What would be best is the flexibility to take a break after reviewing those level of posts rather than simply periodically. The most important thing for moderators would be group and individual therapy sessions run at work with hired therapists. Group therapy is great because you can relate to other moderators struggles which helps process, and individual to help process the most traumatic events. This is what Kik has implemented and seems to be effective.
harry8 7y ago
The solution is to legislate it be done properly which is a business expense. Where it is too expensive to be done properly, there is no business case to have a user created content platform. Being a platform is enormously lucrative, shirking the responsibilities to cut costs and make it even more so is evil. Really.
- gradys 7y ago
  Could you describe some potential legislation you'd be happy with? How would you define "properly"? I'm especially interested in how this would work in places like the United States where there are strong free speech laws.
  - harry8 7y ago
    The detail of how best to regulate is a subject matter for experts, not me. In principle precisely the same way we have legislated against pumping sulphur dioxide out of a chimney in the atmosphere. Same rules for everyone. Now go and compete. You get experts on pollution, mitigation and economics to advise on what will work best for the outcomes you want to achieve. Similar domain experts are required here. Now I have no doubt that facebrick and the evil-goog will try for regulatory capture and employ their vast army of Washington lobbyists in rent-seeking activities but that is a separate problem applicable in every industry where market failure means you have to regulate, not just this one. (And yes it does need solving, absolutely urgently). If you don't have to regulate because the market is working, great! Don't regulate! That's not what we have here, this is market failure but we haven't yet regulated. It will come because there are clearly externalities in this market, Well it will come unless we give up on democracy first. Something with which facebrick and evil-goog seem really comfortable as long as they supplant it.
Spooky23 7y ago
It’s fundamental that some members of the public are problems. By prohibiting the content but keeping the people around, they are curating a culture where predators are allowed to roam on the platform as long as they don’t cross a specific line. They do this because acknowledging and dealing with bad users would also mean impacting fake users that make them money with ad fraud, growth metrics and ginning up engagement. End of the day, if you operate a social platform, and you have people uploading child pornography and animal torture video, you should do everything in your power to make the user go away, for good.
- nojvek 7y ago
  I don’t get the point of animal torture videos. The asking of meat consumed in the world is ludicrous. Eating meat is supporting animal torture in a way. Just saying.
  - xodice 7y ago
    Eating meat is part of life, no one is torturing them. Torture is for a reason or for fun. There is a reason some animals are raised and used for food and its not to have fun torturing them. I let you live your way, don't try to force others to live your way by using "scare tactics".
    - c0vfefe 7y ago
      Well, certain practices common within the meat industry are torturous, regardless of whether they're intended for fun. "Food, Inc." & "Earthlings" are good resources.
lone_haxx0r 7y ago
Do you mean a bussiness solution or a solution to the "social problem" of people posting disgusting content? I have no bussiness solution, but the solution to the latter is to simply ignore/block people who post stuff you don't like.
nibbula 7y ago
"Strong" AI is a person, whom you also shouldn't mistreat. Let's hope they think the same about us.

hashkb 7y ago

> We can blame Facebook, but the reality is no one has a good solution. We can take a step backwards and eliminate Facebook, Insta, etc from our societies and enact legislation that will govern the second generation of social networks.

Jedi72 7y ago
What legislation? Specifically? What do you want to make law, that they have to hire moderators (which they already do?) or do you want to make it illegal to post "bad things" on the internet?
- hashkb 7y ago
  Let me start by saying that I am not running for elected office because it's a really hard job; even though I might believe I would do it with more integrity and rationality than the next person, I'm also smart enough to realize that everyone thinks that way and I'd probably end up within a standard deviation of average corruption. What I'm saying, seriously, is that we need a society-wide blameless post-mortem on social networks. It needs to be a slow, careful discussion, where all the stakeholders have their voices heard, and we decide what is good for us all. I don't know that we need them at all, I'm not sure they provide ANY value to ANYONE, but as a non-user I'd of course be open to persuasion by current victims. Erm, users. In the meantime, it should be illegal to operate a social network. Nobody needs to go to prison yet, even though I think it's horrifyingly clear at this point that the ethics are out the window. One obvious one: advertising and public conversation must always be separated; the same way no public schoolteacher may read scripture in class, it should be illegal for an internet service that hosts public conversation (e.g. Twitter) to allow sponsored content. We should also establish guidelines for addiction. After cigarettes, drugs, sugar, etc; we should as a society be prepared to understand the various forms that addictive products can take and regulate them aggressively before they become serious problems. UX patterns like infinite scroll, pull-to-refresh, and push notifications are particularly suspect. I also think we should close the apparent loophole in COPPA that allows parents to post photos of their children on social networks. > that they have to hire moderators (which they already do?) or do you want to make it illegal to post "bad things" on the internet? I tend to come down on the "free speech" side of these issues as much as possible; I would prefer unregulated public fora that (perhaps by requiring identity verification) encouraged good-faith participation in substantial discussions. I think if you are just fooling around, maybe you should head down to the bar and get drunk with your friends and do your shit talking there.
  - dragonwriter 7y ago
    > What I'm saying, seriously, is that we need a society-wide blameless post-mortem on social networks Social networks aren't dead, so we can't have a post-mortem. Also, you can't have a society-wide blameless analytical conversation about anything, especially if there are significant conflicting interests involved; that's not how people work. (It's perhaps noteworthy that even while suggesting this you offer no objective descriptions of impacts with reasoned analysis of contributing factors, instead jumping straight to blame and policy responses, with some labelled as “obvious”.)
  - closeparen 7y ago
    The point of free speech is that it’s not regulated according to an opinion about “what is good for us all” or what is “needed.”
  - jki275 7y ago
    You claim to be for free speech, yet most of what you propose directly infringes on the concept. You don’t get to determine who gets to say what.
  - Macross8299 7y ago
    >In the meantime, it should be illegal to operate a social network So what about the countless number of people whose livelihoods depend on social media in some capacity? They're supposed to just be fine with being irrevocably fucked over until the moral panic about social media subsides? (or, in your words, "we decide what is good for us all") Where does the small business owner who depends on a social network factor in as a "stakeholder"? Drugs, sugar, cigarettes are measurably and objectively harmful to your health and that's why they are regulated (or should be). There isn't similar comparable scientific evidence that social media is nearly as harmful except for questionable non-reproducible psychology studies, so I don't think it's comparable at all.
    - hashkb 7y ago
      I hear you saying you don't believe social media is addictive. Will you please kindly open the "digital wellbeing" app (or whatever it's called on iOS) on your phone and share the number of hours you've used social networks over the last week?
- Spooky23 7y ago
  I would start with a few things: - Make social networks mandated reporters for suspected child abuse. Like teachers or little league coaches, reporting would absolve them of liability. - Prohibit generation of profit from illegal activity. - Require transparency in political advertising, with an accountable entity and individual listed and publicly available. - Require reporting of funding source and material individuals for paid political advertising. - Make the platform liable for false product claims in cases where the platform facilitates direct or implied endorsements for a product. (“Jedi72 likes this homeopathic cancer treatment!” - Make the platform responsible for reasonable efforts to block and report on actions against fictitious users posing as people. Share liability for any claims, including libel, resulting from the actions of fictitious people. A regulatory regime like this would either improve the signal/noise ratio or cause the company to abandon or prohibit certain actions or functions.
1stcity3rdcoast 7y ago
Ah, yes, Congress — the most competent subject matter experts regarding the internet and technology.
Chazprime 7y ago
Well, to do that you’d need a congress that has the first clue what social media even is, and I’m guessing the US is easily 5-10 years away from that.
ummwhat 7y ago
How does that solve the problem that the incoming content stream flows faster than any curation mechanism we can economically build today?
- joeblubaugh 7y ago
  Well, first you slow the content stream by imposing a rate limit per-user, and users who post things that violate the policy are stopped from posting for a longer time. Second, you do better at automatically detecting duplicate content that’s already been banned.

rhizome 7y ago

>Said problem is that warehousing and distributing content scales insanely well but curation does not I don't think it's a problem any more than weather is a "problem" that required the invention of the roof. As far as user-generated content goes, it's easy to invent the hose, but the faucet is much more complex. It's irresponsible to provide only acceleration ("reach," "virality," etc.) for information without also implementing brakes. Moderation has been a known fix for decades, but people like Zuck -- in disposition and inexperience -- won't launch anything human-powered, and moderation requires a human eye. Fuck us, right?

ummwhat 7y ago
I think you're view will change when you start putting numbers on it. I'm going to talk about the games in the app store example because it's the instance of the problem that I first thought about and it is easier to put a monetary value on an app than on a comment. Let's suppose Apple wants to make sure every game in their store gets at least 5 ratings. To do this they are going to pay people to play each game for 5 min and then assign a rating. In an hour a curator can get through 12 games. Over an 8 hour day thats 96 games. Apple would need to hire about 7 (iirc) full time employees just to give every game one rating. We wanted every game to have 5 ratings. Assuming that users on average provide one rating (but sometimes 0), we still need curators to provide the other 4. That means Apple would be employing 28 full time employees just to get their content minimally curated. Let's use the salary from the article and say each curator costs 30k a year. That means this whole department of curation is going to cost about 900k a year. Most of what they are sorting is going to be shovelware anyway. To justify the cost of this department, the curators need to find gems, content that's really good but would have otherwise been looked over. They need to find 900k of gems per year. Considering the average earning of an app, this is just barely plausible. If you want to do anything more advanced with your curation, like have the curators generate key words or figure out which demographic might like it, your budget is blown for sure. IDK how the economics of comment curation works out, but I bet it's even worse. After all, no one is even paying for comments, and making a comment is way less effort than making an app.
- BRAlNlAC 7y ago

ngold 7y ago

15 billion dollars in pure profit. You could pay a living wage with health benefits, or you could hire twice as many people and have everyone work less while paying more. This is just Facebook being cheap.

nojvek 7y ago
Something something. Corporations are hive mind AI optimizing for growth and profit. They don’t give a shit. Amazon doesn’t give a shit unless it impacts the bottom line. Google barely has support and YouTube kids is full of questionable content.
solstice 7y ago
The image of the Chernobyl liquidators just popped into my mind, each worker only allowed to shovel radiative shit for 90 seconds before having a mandatory time out. Treating content moderators in a similar way seems like a step into the right direction from a mental health perspective
- Meskupie 7y ago
  The largest impact events will likely be the smaller percentage of posts that really resonate with the moderator. What would be best is the flexibility to take a break after reviewing those level of posts rather than simply periodically. The most important thing for moderators would be group and individual therapy sessions run at work with hired therapists. Group therapy is great because you can relate to other moderators struggles which helps process, and individual to help process the most traumatic events. This is what Kik has implemented and seems to be effective.

harry8 7y ago

The solution is to legislate it be done properly which is a business expense. Where it is too expensive to be done properly, there is no business case to have a user created content platform. Being a platform is enormously lucrative, shirking the responsibilities to cut costs and make it even more so is evil. Really.

gradys 7y ago
Could you describe some potential legislation you'd be happy with? How would you define "properly"? I'm especially interested in how this would work in places like the United States where there are strong free speech laws.
- harry8 7y ago
  The detail of how best to regulate is a subject matter for experts, not me. In principle precisely the same way we have legislated against pumping sulphur dioxide out of a chimney in the atmosphere. Same rules for everyone. Now go and compete. You get experts on pollution, mitigation and economics to advise on what will work best for the outcomes you want to achieve. Similar domain experts are required here. Now I have no doubt that facebrick and the evil-goog will try for regulatory capture and employ their vast army of Washington lobbyists in rent-seeking activities but that is a separate problem applicable in every industry where market failure means you have to regulate, not just this one. (And yes it does need solving, absolutely urgently). If you don't have to regulate because the market is working, great! Don't regulate! That's not what we have here, this is market failure but we haven't yet regulated. It will come because there are clearly externalities in this market, Well it will come unless we give up on democracy first. Something with which facebrick and evil-goog seem really comfortable as long as they supplant it.

Spooky23 7y ago

It’s fundamental that some members of the public are problems. By prohibiting the content but keeping the people around, they are curating a culture where predators are allowed to roam on the platform as long as they don’t cross a specific line. They do this because acknowledging and dealing with bad users would also mean impacting fake users that make them money with ad fraud, growth metrics and ginning up engagement. End of the day, if you operate a social platform, and you have people uploading child pornography and animal torture video, you should do everything in your power to make the user go away, for good.

nojvek 7y ago
I don’t get the point of animal torture videos. The asking of meat consumed in the world is ludicrous. Eating meat is supporting animal torture in a way. Just saying.
- xodice 7y ago
  Eating meat is part of life, no one is torturing them. Torture is for a reason or for fun. There is a reason some animals are raised and used for food and its not to have fun torturing them. I let you live your way, don't try to force others to live your way by using "scare tactics".
  - c0vfefe 7y ago
    Well, certain practices common within the meat industry are torturous, regardless of whether they're intended for fun. "Food, Inc." & "Earthlings" are good resources.

lone_haxx0r 7y ago

Do you mean a bussiness solution or a solution to the "social problem" of people posting disgusting content? I have no bussiness solution, but the solution to the latter is to simply ignore/block people who post stuff you don't like.

nibbula 7y ago

"Strong" AI is a person, whom you also shouldn't mistreat. Let's hope they think the same about us.