8 ms·

What are your thoughts on incentivizing constructive comments? I've seen publishers (The Guardian, if memory serves) select thoughtful comments and re-print th

What are your thoughts on incentivizing constructive comments? I've seen publishers (The Guardian, if memory serves) select thoughtful comments and re-print them as micro-articles in their own right. This seems to solve part of the problem by setting an established, if not mostly-objective, standard for comment quality: journalistic publication standards. As such, bias and opinion is welcome, provided that it's analytical, verified by fact to a reasonable degree, and respectful of common etiquette. The genius in this approach, as far as I'm concerned, is that it manages to preserve the original purpose of comments: scalable content-generation! Clearly, moderation is a Hard Problem, but one that I think benefits from an economic/incentives analysis. One conclusion I've drawn is that restricting comments to paid-consumers makes banishment and sock-puppetry costly enough that moderators can mop up the rest by hand. To ask a specific question: what, exactly, remains "hard" with this approach? Do you think "free to read / pay to comment" is viable, in principle? Do you think the promise of publication is not a good incentive? Why?

showerst 10y ago
I think that incentive idea is great, and is a smart move to build a community, particularly when you're trying to draw subject matter experts. I like how some of the Ask<X> reddits do it, by flagging people with verified advanced degrees. People think that news sites are afraid of conflicting opinions, but in my experience that's nonsense, it just has to be well thought out and not "DEATH TO <ISRAEL/ARABS/SUNNI/TURKS/AMERICA>", which is the vast majority. It still doesn't solve the problem that for someone to _find_ those great comments, they have to _read_ them, and stop them from getting buried. I'll err on the side of caution with revealing employee counts, but in my experience many of the FP/Atlantic/Mother Jones/Weekly Standard/Pick your midrange site are running on a single digit to low double-digit number of web production staff, many of whom are also trying to make a writing, article layout, or fact-checking quota. The suggestion that these magazines can either get those staffers to moderate tens of thousands of comments per day, or quadruple their web staff just to improve the comments ignores the business reality. User moderation in the normal HN/Reddit way doesn't work well on news sites, it's too easy to game or brigade, and news sites can't or won't give add unpaid moderators to be gatekeepers. That's what's hard; creating comments is scalable, filtering them is not. Leaving them unfiltered doesn't work either.
- omginternets 10y ago
  Thanks for your reply! >I think that incentive [is good], particularly when you're trying to draw subject matter experts. You bring up an excellent point. One of the fundamental problems with comments, I think, is that it creates a space in which ignorance and expertise are equally-weighted. In fact, it's often worse than that for reasons we all know: interesting issues are hard to distill into 300-or-so characters, and short, simple points are often more percussive. Vetting credentials is a very good option IMHO for certain forums but not for others. Reddit's /r/askscience is an example of a forum in which it works well. >It still doesn't solve the problem that for someone to _find_ those great comments, they have to _read_ them, and stop them from getting buried. I wonder if this problem can't be solved through the use of machine-learning to classify comments into high-versus-low quality by grammatical and semantic analysis. This kind of first-pass filtering could, at the very least, help throw out the obvious trash and pre-select candidates for recognition. Such a system can be tuned to minimize false-alarms (shitpost getting flagged as good), which I think represent the most problematic of classification errors. This is a nice problem-space for ML because the increase in misses implied by a bias against false-alarms doesn't degrade the service much: not having one's comment select for re-publication is unexceptional. Thoughts?
  - showerst 10y ago
    RE:Machine learning: I think there are two problems with that approach, one cultural and one technological. The cultural issue is that many news orgs are still run by people for whom the idea that technology could accidentally censor a valid criticism or ban a decent voice is just too risky. I think this is changing, and many newsrooms today a little more fluid than when I really cared about the problem 4 years ago. The tech issue is a little bit of a cop out on my part. An ML approach is super attractive to me as a techie. Google (youtube), facebook, NYT, WaPO, and tons of other billion dollar orgs have this problem, and could loads of money by being seen as better communities. On the more guerrilla side, hundreds of subreddits have automoderaters written by savvy, caring moderators. They have terabytes of training data, already tagged, and world class ML experts on staff. If it was a tractable problem with business value, why wouldn't they have fixed it? I'm guessing it's the sort of thing that looks doable from the surface, but you get buried in the details. Again, cop out answer, so please go prove me wrong!!
bargl 10y ago
A podcast I frequent does this sort of thing. If your comment is read on the podcast (and they read one a day) then you get sent a .NET Rocks coffee mug. Which is kinda neat. The podcast is .NET Rocks and their comments seem to be pretty good overall.
justinlardinois 10y ago
> What are your thoughts on incentivizing constructive comments? I've seen publishers (The Guardian, if memory serves) select thoughtful comments and re-print them as micro-articles in their own right. I don't think you're correctly identifying the problem. In my experience, the problem with comments, especially on news sites, is a glut of bad comments, rather than a lack of good comments. This solution doesn't disincentivize bad comments.
- dageshi 10y ago
  The slashdot system for categorising comments seemed to work really well at making the highest quality comments stand out, I wonder why other sites haven't tried something similar, I don't think I've seen it used elsewhere.
  - basch 10y ago
    it seems sites find it to complex to implement. slashdot nailed moderation, no one has attempted something similar. most systems are simple up/down vote or like/report i am also starting to wonder if the agegroup being hired to implement "social" for websites is now young enough to have missed slashdot in it s prime. the fact that people are still brainstorming from scratch instead of talking about how to improve slashdots model reeks of reinventing the wheel because they never heard of it.
    - edraferi 10y ago
      > i am also starting to wonder if the agegroup being hired to implement "social" for websites is now young enough to have missed slashdot in it s prime. That's me! Can you explain the Slashdot model and why it worked? Or point to a good write up about it somewhere else?
      - flomo 10y ago
        Slashdot's method of scoring comments was overly complicated and probably did not produce any better results than reddit-style voting. However, Slashdot's killer feature was that the reader could filter by comment score and thus only read the 'good' comments, and not have to wade through hundreds of replies.
Theodores 10y ago
When I was delivering newspapers as a small child many aeons ago, the best page of The Guardian was 'letters to the editor'. The rest of the paper was pretty good back then, there was no email, so anything printed in the 'letters to the editor' had to be posted in, to appear some time after the events in question. Needless to say an event happened and was reported the next day, so it could be a whole week between the Trump-of-the-day saying something and comment appearing about it. All of this would be filtered by the 'editor', however you did have frequent letters by the likes of Keith Flett, who somehow got his letters published more often than the other 3-5 million readers (as it was back then, just UK sales with poor distribution in places like Birmingham). There were no 'likes' back then so you had to have something to say to bother writing in. How do we get a digital equivalent? I don't buy the dead-tree paper these days so no idea if 'letters to the editor' still exists, but, back then it was good, very good.
- mholmes680 10y ago
  Its interesting that simply restricting immediate commenting might at least deter useless comments. People who are commenting in order to elicit a response, i suppose, probably have less important things to say. Or maybe they wouldn't say them if they are not granted the immediate satisfaction. I assume it would kill some collaboration/innovation like on HN or a meaningful subreddit, but maybe no one really ever has anything meaningful to say when reacting to general news... I guess it would also produce duplication from many people not knowing something was said already (however, the duplicate reactions could be monetized later down the line maybe...)

showerst 10y ago

I think that incentive idea is great, and is a smart move to build a community, particularly when you're trying to draw subject matter experts. I like how some of the Ask<X> reddits do it, by flagging people with verified advanced degrees. People think that news sites are afraid of conflicting opinions, but in my experience that's nonsense, it just has to be well thought out and not "DEATH TO <ISRAEL/ARABS/SUNNI/TURKS/AMERICA>", which is the vast majority. It still doesn't solve the problem that for someone to _find_ those great comments, they have to _read_ them, and stop them from getting buried. I'll err on the side of caution with revealing employee counts, but in my experience many of the FP/Atlantic/Mother Jones/Weekly Standard/Pick your midrange site are running on a single digit to low double-digit number of web production staff, many of whom are also trying to make a writing, article layout, or fact-checking quota. The suggestion that these magazines can either get those staffers to moderate tens of thousands of comments per day, or quadruple their web staff just to improve the comments ignores the business reality. User moderation in the normal HN/Reddit way doesn't work well on news sites, it's too easy to game or brigade, and news sites can't or won't give add unpaid moderators to be gatekeepers. That's what's hard; creating comments is scalable, filtering them is not. Leaving them unfiltered doesn't work either.

omginternets 10y ago
Thanks for your reply! >I think that incentive [is good], particularly when you're trying to draw subject matter experts. You bring up an excellent point. One of the fundamental problems with comments, I think, is that it creates a space in which ignorance and expertise are equally-weighted. In fact, it's often worse than that for reasons we all know: interesting issues are hard to distill into 300-or-so characters, and short, simple points are often more percussive. Vetting credentials is a very good option IMHO for certain forums but not for others. Reddit's /r/askscience is an example of a forum in which it works well. >It still doesn't solve the problem that for someone to _find_ those great comments, they have to _read_ them, and stop them from getting buried. I wonder if this problem can't be solved through the use of machine-learning to classify comments into high-versus-low quality by grammatical and semantic analysis. This kind of first-pass filtering could, at the very least, help throw out the obvious trash and pre-select candidates for recognition. Such a system can be tuned to minimize false-alarms (shitpost getting flagged as good), which I think represent the most problematic of classification errors. This is a nice problem-space for ML because the increase in misses implied by a bias against false-alarms doesn't degrade the service much: not having one's comment select for re-publication is unexceptional. Thoughts?
- showerst 10y ago
  RE:Machine learning: I think there are two problems with that approach, one cultural and one technological. The cultural issue is that many news orgs are still run by people for whom the idea that technology could accidentally censor a valid criticism or ban a decent voice is just too risky. I think this is changing, and many newsrooms today a little more fluid than when I really cared about the problem 4 years ago. The tech issue is a little bit of a cop out on my part. An ML approach is super attractive to me as a techie. Google (youtube), facebook, NYT, WaPO, and tons of other billion dollar orgs have this problem, and could loads of money by being seen as better communities. On the more guerrilla side, hundreds of subreddits have automoderaters written by savvy, caring moderators. They have terabytes of training data, already tagged, and world class ML experts on staff. If it was a tractable problem with business value, why wouldn't they have fixed it? I'm guessing it's the sort of thing that looks doable from the surface, but you get buried in the details. Again, cop out answer, so please go prove me wrong!!

bargl 10y ago

A podcast I frequent does this sort of thing. If your comment is read on the podcast (and they read one a day) then you get sent a .NET Rocks coffee mug. Which is kinda neat. The podcast is .NET Rocks and their comments seem to be pretty good overall.

justinlardinois 10y ago

> What are your thoughts on incentivizing constructive comments? I've seen publishers (The Guardian, if memory serves) select thoughtful comments and re-print them as micro-articles in their own right. I don't think you're correctly identifying the problem. In my experience, the problem with comments, especially on news sites, is a glut of bad comments, rather than a lack of good comments. This solution doesn't disincentivize bad comments.

dageshi 10y ago
The slashdot system for categorising comments seemed to work really well at making the highest quality comments stand out, I wonder why other sites haven't tried something similar, I don't think I've seen it used elsewhere.
- basch 10y ago
  it seems sites find it to complex to implement. slashdot nailed moderation, no one has attempted something similar. most systems are simple up/down vote or like/report i am also starting to wonder if the agegroup being hired to implement "social" for websites is now young enough to have missed slashdot in it s prime. the fact that people are still brainstorming from scratch instead of talking about how to improve slashdots model reeks of reinventing the wheel because they never heard of it.
  - edraferi 10y ago
    > i am also starting to wonder if the agegroup being hired to implement "social" for websites is now young enough to have missed slashdot in it s prime. That's me! Can you explain the Slashdot model and why it worked? Or point to a good write up about it somewhere else?
    - flomo 10y ago
      Slashdot's method of scoring comments was overly complicated and probably did not produce any better results than reddit-style voting. However, Slashdot's killer feature was that the reader could filter by comment score and thus only read the 'good' comments, and not have to wade through hundreds of replies.

Theodores 10y ago

When I was delivering newspapers as a small child many aeons ago, the best page of The Guardian was 'letters to the editor'. The rest of the paper was pretty good back then, there was no email, so anything printed in the 'letters to the editor' had to be posted in, to appear some time after the events in question. Needless to say an event happened and was reported the next day, so it could be a whole week between the Trump-of-the-day saying something and comment appearing about it. All of this would be filtered by the 'editor', however you did have frequent letters by the likes of Keith Flett, who somehow got his letters published more often than the other 3-5 million readers (as it was back then, just UK sales with poor distribution in places like Birmingham). There were no 'likes' back then so you had to have something to say to bother writing in. How do we get a digital equivalent? I don't buy the dead-tree paper these days so no idea if 'letters to the editor' still exists, but, back then it was good, very good.

mholmes680 10y ago
Its interesting that simply restricting immediate commenting might at least deter useless comments. People who are commenting in order to elicit a response, i suppose, probably have less important things to say. Or maybe they wouldn't say them if they are not granted the immediate satisfaction. I assume it would kill some collaboration/innovation like on HN or a meaningful subreddit, but maybe no one really ever has anything meaningful to say when reacting to general news... I guess it would also produce duplication from many people not knowing something was said already (however, the duplicate reactions could be monetized later down the line maybe...)