3 ms·

Why Must there be one "correct" view of the comments on a document? This insistence seems to be what hobbles us. People will disagree on what is or isn't worth

Why Must there be one "correct" view of the comments on a document? This insistence seems to be what hobbles us. People will disagree on what is or isn't worth reading. That's fine. We should see the problem as sorting comments per viewer, not for all viewers. If 10 of my close friends think a comment is worth reading, then I personally would like to read it. It doesn't matter what the moderator or a quoroum of random internet people think at that point. We should be using the social graph to filter out nonsense, and to surface constant that we are likely to enjoy. If anyone is interested, I've written code to do this here: https://github.com/neyer/respect https://github.com/neyer/respect

paulchristiano 10y ago
I think that the correct behavior in the long run is to give personalized recommendations, analogous to this proposal: https://sideways-view.com/2016/12/01/optimizing-the-news-feed/ https://sideways-view.com/2016/12/01/optimizing-the-news-fee... The main motivation for having a unified view is to decide what to show people who visit but don't rate things / log in / etc. I'm currently finishing my dissertation on collaborative learning algorithms, and I agree that it is not generally possible to have a single global answer. This paper does something similar to your respect matrix: https://arxiv.org/abs/1411.1127 https://arxiv.org/abs/1411.1127 (The same technique can be modified to incorporate information from an external social network.)
Yen 10y ago
I'd argue that there is exactly one "correct" view of comments on a document, per-community. The correct view is the one that best represents the range of opinions of the community, and bests represents that community's cultural norms. To be specific, if I'm reading the comments section on Hacker News, I don't want to see comments only from people who agree with me or think how I do, and I don't only want to see comments from people I disagree with. I want to see what the community of HN readers thinks, in the particular way they express it. Additionally, I want to know that what I'm seeing is also what they're seeing. In a sense, this is still social filtering, except averaged over all members of a community, and with the same results for all members. The community responds to posts with approval or disapproval, through votes or comment responses, and this gets averaged (messily) by the moderators. I think, really, that there's an almost objective set of metrics for what makes a good comment thread. * The thread covers most of the held opinions & responses, with few to no 'duplicate' comments * The posts are well-written and polite * Assertions about the world are based on agreed-on facts, and justifiable (even if not agreed-on) conclusions stemming from those. * The posts respect any other cultural norms particular to that community. These sound subjective, but I think that neutral bystanders would tend to score a post similarly on these different metrics - i.e., there's actually a measurable quality of a post being 'polite', or 'reasonably argued', and it's not entirely relative to an individual reader (or even a single community!) On a polarizing political post, I don't want to read poorly written insults from the opposing position, but I also want to know that the other side won't be reading poorly written insults that misrepresent my side. I'd rather the best, most rational, and most empathetic posts from both sides be displayed as representative of the discussion.
matt4077 10y ago
I'd say it's to counteract the "Eternal September" dynamics. As communities grow, they lose focus. The initial set of moderators may have done an excellent job at curation, but they can't scale to a growing community. The idea is then to have their judgement "flow" to other users.
mncharity 10y ago
I suspect respect also needs to be topic-specific. For example. http://represent.berkeley.edu/umati/ http://represent.berkeley.edu/umati/ is a candy machine that bribes undergraduates to grade exams. It learns both about the exams, and about the graders. But "grader X is a generally B-ish student" is much less valuable than "grader X does well on questions on topic Q, but is unreliably confused about topic R". People's comments within their area of professional expertise are often more valuable than their comments which aren't. Also, my threshold for "insightful enough to spend time on" varies by subject. And by time of day. Just now, I'd love to be able to ask HN "just give me a couple of really nice, deeply insightful comments, on any topic". But yes. Being able to filter by cohort could finally interrupt the dynamic of quality discussion fleeing from site to site, chased by a regression-to-mean eternal September. Which suggests a need for a rich subcommunity selection language. "Show me HN default, but boost the weight of HN old-timers, plus people they highly respect, transitively with a decay factor of R". So rather than social graph, I'd focus on broader machine learning / statistical analysis and computer-human hybrid processes. For example, say a person is sometimes insightful, but sometimes crackpot. An aggregate metric isn't very helpful. And it's hard for automation to make that call. But it's easier for automation to recognize the problem is an issue for some comment, and to select a couple of humans to give moderation judgment.

paulchristiano 10y ago

I think that the correct behavior in the long run is to give personalized recommendations, analogous to this proposal: https://sideways-view.com/2016/12/01/optimizing-the-news-feed/ https://sideways-view.com/2016/12/01/optimizing-the-news-fee... The main motivation for having a unified view is to decide what to show people who visit but don't rate things / log in / etc. I'm currently finishing my dissertation on collaborative learning algorithms, and I agree that it is not generally possible to have a single global answer. This paper does something similar to your respect matrix: https://arxiv.org/abs/1411.1127 https://arxiv.org/abs/1411.1127 (The same technique can be modified to incorporate information from an external social network.)

Yen 10y ago

I'd argue that there is exactly one "correct" view of comments on a document, per-community. The correct view is the one that best represents the range of opinions of the community, and bests represents that community's cultural norms. To be specific, if I'm reading the comments section on Hacker News, I don't want to see comments only from people who agree with me or think how I do, and I don't only want to see comments from people I disagree with. I want to see what the community of HN readers thinks, in the particular way they express it. Additionally, I want to know that what I'm seeing is also what they're seeing. In a sense, this is still social filtering, except averaged over all members of a community, and with the same results for all members. The community responds to posts with approval or disapproval, through votes or comment responses, and this gets averaged (messily) by the moderators. I think, really, that there's an almost objective set of metrics for what makes a good comment thread. * The thread covers most of the held opinions & responses, with few to no 'duplicate' comments * The posts are well-written and polite * Assertions about the world are based on agreed-on facts, and justifiable (even if not agreed-on) conclusions stemming from those. * The posts respect any other cultural norms particular to that community. These sound subjective, but I think that neutral bystanders would tend to score a post similarly on these different metrics - i.e., there's actually a measurable quality of a post being 'polite', or 'reasonably argued', and it's not entirely relative to an individual reader (or even a single community!) On a polarizing political post, I don't want to read poorly written insults from the opposing position, but I also want to know that the other side won't be reading poorly written insults that misrepresent my side. I'd rather the best, most rational, and most empathetic posts from both sides be displayed as representative of the discussion.

matt4077 10y ago

I'd say it's to counteract the "Eternal September" dynamics. As communities grow, they lose focus. The initial set of moderators may have done an excellent job at curation, but they can't scale to a growing community. The idea is then to have their judgement "flow" to other users.

mncharity 10y ago

I suspect respect also needs to be topic-specific. For example. http://represent.berkeley.edu/umati/ http://represent.berkeley.edu/umati/ is a candy machine that bribes undergraduates to grade exams. It learns both about the exams, and about the graders. But "grader X is a generally B-ish student" is much less valuable than "grader X does well on questions on topic Q, but is unreliably confused about topic R". People's comments within their area of professional expertise are often more valuable than their comments which aren't. Also, my threshold for "insightful enough to spend time on" varies by subject. And by time of day. Just now, I'd love to be able to ask HN "just give me a couple of really nice, deeply insightful comments, on any topic". But yes. Being able to filter by cohort could finally interrupt the dynamic of quality discussion fleeing from site to site, chased by a regression-to-mean eternal September. Which suggests a need for a rich subcommunity selection language. "Show me HN default, but boost the weight of HN old-timers, plus people they highly respect, transitively with a decay factor of R". So rather than social graph, I'd focus on broader machine learning / statistical analysis and computer-human hybrid processes. For example, say a person is sometimes insightful, but sometimes crackpot. An aggregate metric isn't very helpful. And it's hard for automation to make that call. But it's easier for automation to recognize the problem is an issue for some comment, and to select a couple of humans to give moderation judgment.