4 ms·

How online ratings affect your judgment

41 points by X4 13y ago

jared314 13y ago
I usually like to read the methodologies on studies that cover online behavior before I consider their results, but I can't in this case because of a paywall. Does anyone have a link to the full text of the study?
- fsckin 13y ago
  I can't find this one, but there are similar papers floating around. Social Bias in Online Product Ratings: A Quasi-Experimental Analysis: http://misrc.umn.edu/wise/papers/4b-2.pdf http://misrc.umn.edu/wise/papers/4b-2.pdf Social Bias in Online Product Ratings: http://www.econ.sdu.edu.cn/advance/uploadfile/2012040908191596.pdf http://www.econ.sdu.edu.cn/advance/uploadfile/20120409081915...
  - enjo 13y ago
    Here's one that draws pretty much the opposite conclusion of the study in OP's article. It focused on stock message boards, which might not be quite the same. http://digital.library.unt.edu/ark:/67531/metadc5130/ http://digital.library.unt.edu/ark:/67531/metadc5130/ I should note that I'm pretty close to the author. She'd be happy to answer questions I imagine.
    - vwinsyee 13y ago
      Is there access to the full dissertation (i.e., not just the description/abstract)? For those interested, the key statement in this study: Using an experiment, message board influence on an investment decision and attitude was tested. The results indicated that individuals that received negative message board postings did have a significantly higher change in investment amount as compared to a control group that did not receive any message postings. The positive message board group and the control group were not significantly different in their amount of investment change. The methodology of the experiment isn't detailed in the description. But it seems that one major difference between this study and the study in the OP article is the action taken by the user: change in monetary investment, and upvote/downvote. People behave differently when their personal money's on the line, so this is an important difference. So I'm very curious as to how this study ran its experiment.
    - X4 13y ago
      Aww, would you? Would you please ask, if the paper could be posted here? That would be really nice!
NhanH 13y ago
The news wasn't clear, and I couldn't find the actual papers. But what does "positive" and "negative" mean? Does "positive" means "similar to one's belief", just positivity in general (in tone, wording of a comment etc)? Or does "positive" = positive amount of karma/ comment ratings (which seems to be the case, by my understanding)?
- vwinsyee 13y ago
  I couldn't find the article without passing through a paywall. But this article [1] summarizes the methodology decently: "We used a site very similar to Reddit," Aral says. Here's how it worked: The researchers convinced an anonymous website–under the promise it would remain nameless–to allow them to test slight manipulations in online voting. Over six months, the research team took a small percentage of more than 100,000 written comments and slightly altered their vote rating as they were submitted. Four percent of the comments were given a positive edge (a single upvote) while another 2 percent were struck with an immediate disadvantage (a lonely downvote). [1] http://www.popularmechanics.com/science/health/nueroscience/upvotes-downvotes-and-the-science-of-the-reddit-hivemind-15784871 http://www.popularmechanics.com/science/health/nueroscience/...
  - vwinsyee 13y ago
    On an unrelated note, I guess someone at Popular Mechanics misspelled the "neuroscience" category label. The correct "neuroscience" spelling redirects to "nueroscience"...
  - NhanH 13y ago
    Thanks. I guess in that case "popular" would be a more accurate description, rather than "positive", isn't it?
  - bobbydavid 13y ago
    my wild theory: what if the site in the study was HN!? disclaimer: I have no idea and tldr
    - 13y ago
mjs 13y ago
The fiddled with the ratings of which "major news-aggregation web site"?
X4 13y ago
Here is more info about the paper's background and ramblings on twitter. http://tjm.org/2013/08/08/proposal-experiment-to-compare-open-vs-paywall-publishing/#more-5516 http://tjm.org/2013/08/08/proposal-experiment-to-compare-ope... DOI:10.1126/science.1240466

41 points by X4 13y ago

I usually like to read the methodologies on studies that cover online behavior before I consider their results, but I can't in this case because of a paywall. Does anyone have a link to the full text of the study?

fsckin 13y ago
I can't find this one, but there are similar papers floating around. Social Bias in Online Product Ratings: A Quasi-Experimental Analysis: http://misrc.umn.edu/wise/papers/4b-2.pdf http://misrc.umn.edu/wise/papers/4b-2.pdf Social Bias in Online Product Ratings: http://www.econ.sdu.edu.cn/advance/uploadfile/2012040908191596.pdf http://www.econ.sdu.edu.cn/advance/uploadfile/20120409081915...
- enjo 13y ago
  Here's one that draws pretty much the opposite conclusion of the study in OP's article. It focused on stock message boards, which might not be quite the same. http://digital.library.unt.edu/ark:/67531/metadc5130/ http://digital.library.unt.edu/ark:/67531/metadc5130/ I should note that I'm pretty close to the author. She'd be happy to answer questions I imagine.
  - vwinsyee 13y ago
    Is there access to the full dissertation (i.e., not just the description/abstract)? For those interested, the key statement in this study: Using an experiment, message board influence on an investment decision and attitude was tested. The results indicated that individuals that received negative message board postings did have a significantly higher change in investment amount as compared to a control group that did not receive any message postings. The positive message board group and the control group were not significantly different in their amount of investment change. The methodology of the experiment isn't detailed in the description. But it seems that one major difference between this study and the study in the OP article is the action taken by the user: change in monetary investment, and upvote/downvote. People behave differently when their personal money's on the line, so this is an important difference. So I'm very curious as to how this study ran its experiment.
  - X4 13y ago
    Aww, would you? Would you please ask, if the paper could be posted here? That would be really nice!

NhanH 13y ago

The news wasn't clear, and I couldn't find the actual papers. But what does "positive" and "negative" mean? Does "positive" means "similar to one's belief", just positivity in general (in tone, wording of a comment etc)? Or does "positive" = positive amount of karma/ comment ratings (which seems to be the case, by my understanding)?

vwinsyee 13y ago
I couldn't find the article without passing through a paywall. But this article [1] summarizes the methodology decently: "We used a site very similar to Reddit," Aral says. Here's how it worked: The researchers convinced an anonymous website–under the promise it would remain nameless–to allow them to test slight manipulations in online voting. Over six months, the research team took a small percentage of more than 100,000 written comments and slightly altered their vote rating as they were submitted. Four percent of the comments were given a positive edge (a single upvote) while another 2 percent were struck with an immediate disadvantage (a lonely downvote). [1] http://www.popularmechanics.com/science/health/nueroscience/upvotes-downvotes-and-the-science-of-the-reddit-hivemind-15784871 http://www.popularmechanics.com/science/health/nueroscience/...
- vwinsyee 13y ago
  On an unrelated note, I guess someone at Popular Mechanics misspelled the "neuroscience" category label. The correct "neuroscience" spelling redirects to "nueroscience"...
- NhanH 13y ago
  Thanks. I guess in that case "popular" would be a more accurate description, rather than "positive", isn't it?
- bobbydavid 13y ago
  my wild theory: what if the site in the study was HN!? disclaimer: I have no idea and tldr
  - 13y ago

mjs 13y ago

The fiddled with the ratings of which "major news-aggregation web site"?

X4 13y ago

Here is more info about the paper's background and ramblings on twitter. http://tjm.org/2013/08/08/proposal-experiment-to-compare-open-vs-paywall-publishing/#more-5516 http://tjm.org/2013/08/08/proposal-experiment-to-compare-ope... DOI:10.1126/science.1240466