7 ms·

The "cumulative advantage" study mentioned in the article is very interesting, and relevant to HN. Basically they built a music sharing site where people could

by kens 13y ago

The "cumulative advantage" study mentioned in the article is very interesting, and relevant to HN. Basically they built a music sharing site where people could download and rate music, and they'd see which songs became popular. The cool part is they randomly split the users into 8 subgroups, each with independent song voting and ranking. The top hits in one subgroup were very different from the top hits in a different subgroup, showing there's a huge random factor due to the feedback loop of people liking what's popular. The obvious connection to HN is that people read and upvote the popular articles, so there's (probably) a lot more randomness and a lot less meritocracy in what makes it to the front page than people suspect. This matches my experience, where one of my blog posts will sink without a trace on HN, and then be hugely popular after someone resubmits it later. It would be very interesting to do a "cumulative advantage" study on HN: split the users into sub-groups, and see if there's any correlation between the popular articles across sites. I suspect there's also a "cumulative advantage" effect on comments, with popular members getting way more upvotes. Some high-karma member could quantify this by posting half their comments from a new account and A/B testing. link to more on the music experiment: http://www.nytimes.com/2007/04/15/magazine/15wwlnidealab.t.html?pagewanted=all&_r=1& http://www.nytimes.com/2007/04/15/magazine/15wwlnidealab.t.h...

jmduke 13y ago
The obvious connection to HN is that people read and upvote the popular articles, so there's (probably) a lot more randomness in what makes it to the front page than people suspect. This matches my experience, where one of my blog posts will sink without a trace on HN, and then get reposted later and shoot to the top. Perhaps a more apt comparison would be how certain blogs (37signals, marco.org, Svbtle, etc) rocket to the front page based off of their TLD alone. I don't think this is a particularly bad thing -- this is how branding works, after all -- but I would definitely be interested to see an experiment that throws everything on Readability and hides the URL to see if things change. I suspect there's also a "cumulative advantage" effect on comments, with popular members getting way more upvotes. Some high-karma member could quantify this by posting half their comments from a new account and A/B testing. I could be incorrect, but I recall reading that part of the ranking algorithm is based on the commenter's average karma.
- jbenz 13y ago
  As I remember it, there was a feature implemented where leaders (veteran accounts with high vote averages) do get something of a boost when they comment, at least within the sequencing of the comments in any one thread. I think this was done to help improve the quality of the conversation, even before the removal of the vote totals next to each comment.
- deleted 13y ago
  [deleted]
- deleted 13y ago
  [deleted]
  - jmduke 13y ago
    Definitely! This is especially a big issues with focus groups (I had to run a bunch for a few semesters for part of my marketing degree, and one of the first things you learn is how to disrupt an echo chamber: if X and Y say two very positive things about a widget, than Z is suddenly very disincentivized to say anything negative and everyone else is more inclined to join the echo chamber). But I don't think that's whats being brought up with the article. Literary critics are more or less the 'first voters' on new books: and yet, the correlation between critical successes and commercial successes are tentative at best (while the first few votes on an article are always an indicator of future votes.) This is mainly, I think, a function of accessibility: you see the rating of a link on HN before you even click on the link, while most (I'd say 90%+) of the people buying books do so unaware of what everyone before them thought about the book.
samolang 13y ago
When you said it was relevant to HN I thought you were going to say because the same phenomenon happens with start-ups. Being good isn't always enough, usually you need a little luck as well.
- kens 13y ago
  Yes, totally. I was going to add that in an edit, but you beat me to it :-)
raverbashing 13y ago
Well, in a sense, the previous rating is an heuristic. Downloading and listening a song is "boring" unless the song is good So it's more or less this: "rate this song" is different from "do you like this song that other people thought it was good as well?" Maybe for HN you can randomize the order on the front page and hide the score
deleted 13y ago
[deleted]
Florin_Andrei 13y ago
Society is full of positive (self-enforcing, runaway) feedback loops. It is not self-stabilizing. It tends to flip-flop between extremes. This applies to pretty much any field. It's the strongest argument against laissez-faire economics, yet it's amazing how many people are blind to it (well, that's exactly part of the mechanism that enables the aforementioned feedback loops).
- liber8 13y ago
  Can you provide one example of such a phenomena operating on anything longer than a very short timescale? I'm genuinely curious. Nearly every example I can think of is long-term negative. edit: I'd venture to say anything longer than 5-10 years could be considered long term for this question. Under that measure, I'd say government spending has exhibited positive-feedback behavior over the last 50+ years. Anything else?
  - n6mac41717 13y ago
    Movies: Rocky, 8 million. Star Trek, Infinitum.
  - mjfl 13y ago
    The population of New York vs. Philadelphia
  - Florin_Andrei 13y ago
    Any decisive battle between two big kingdoms. The "share of the pie" for the upper class in the US in the last few decades.
  - Retric 13y ago
    Religion. Fashion ex Louis chairs. Language ex regional accents and new words / phrases. Manners and other social protocalls. Operating systems, social networks, and computer languages.
    - mc32 13y ago
      In Japan, this is even more pronounced. MFGs seek to ride the wave of teen uptake of new fast-moving products. These products tend to enjoy a short-lived, but intense attention by consumers.
bcgraham 13y ago
Stephen King did this and wrote new novels under a pen name, and the books were also best-sellers.
- n6mac41717 13y ago
  From: http://news.slashdot.org/story/13/07/18/2147231/jk-rowling-should-try-the-voting-algorithm http://news.slashdot.org/story/13/07/18/2147231/jk-rowling-s... "King did say that he thought he was outed too early to obtain any conclusive results from the experiment..."
  - bcgraham 13y ago
    Thanks! I didn't know that.
  - jonnathanson 13y ago
    The other thing to keep in mind is that well over half the battle in becoming a bestseller is even landing a publisher in the first place. There's serious cumulative advantage inherent to being Stephen King, or JK Rowling, in that you can submit your "anonymous" work to your ordinary publisher and at least guarantee some amount of distribution. A truly anonymous play would be if Rowling or King submitted their pseudonymous work to agents and publishers, unsolicited, without any trace whatsoever of their existing relationships. (This might not be possible due to contactual obligations, however).
    - ars 13y ago
      > A truly anonymous play would be if Rowling or King submitted their pseudonymous work to agents and publishers, unsolicited, without any trace whatsoever of their existing relationships. Didn't Rowling do exactly that with this book?
TylerE 13y ago
Plenty of real world examples of that effect. Look at bands that are huge in say, Japan or Germany but struggle to fill a small club in their home country.
- dpeck 13y ago
  A prophet is not without honor except in his own town
tpurves 13y ago
That study has been very influential to my own thinking about how things get popular. Another thing I found fascinating about the study was that 'quality' of the content did matter, but that it accounted for roughly half the correlation with success, the other half being random-chance snowball effects. You could probably extend this theory of what makes things successful to a lot of other domains (music, books, fashion, ideas, memes, startups, people...) However, it's possible the ratio of intrinsic qualities vs extrinsic luck varies or is not 50/50 in all domains. With music (or books) you have to invest a few minutes or a few hours to really evaluate if you like something, so those factors may affect how much extrinsic "cumulative advatage" snowball effects count vs intrinsic quality. I'd really like to see more empirical research of how success by "luck vs skill" actually plays out in other categories. Any one know of follow up work to that 2007 study?
- bostonvaulter2 13y ago
  It seems like this would be a reason to release more high-quality work rather than fewer excellent quality work. Although perhaps the excellent work would stand out more.
anigbrowl 13y ago
Some high-karma member could quantify this by posting half their comments from a new account and A/B testing. As it happens, my wife is a HN member but has not found it easy to get traction for stories she submits. On a couple of occasions I have re-submitted an article that I thought was unfairly overlooked and it has attracted replies and karma in abundance. Anecdata to be sure, but based on this and other observations in HN and other online communities I think your hunch is likely correct. If anyone feels like exploring this in R or somesuch, MetaFilter (where upvotes by other members are public) makes a large amount of its user pattern data available for analysis: http://stuff.metafilter.com/infodump/ http://stuff.metafilter.com/infodump/

by kens 13y ago

jmduke 13y ago

The obvious connection to HN is that people read and upvote the popular articles, so there's (probably) a lot more randomness in what makes it to the front page than people suspect. This matches my experience, where one of my blog posts will sink without a trace on HN, and then get reposted later and shoot to the top. Perhaps a more apt comparison would be how certain blogs (37signals, marco.org, Svbtle, etc) rocket to the front page based off of their TLD alone. I don't think this is a particularly bad thing -- this is how branding works, after all -- but I would definitely be interested to see an experiment that throws everything on Readability and hides the URL to see if things change. I suspect there's also a "cumulative advantage" effect on comments, with popular members getting way more upvotes. Some high-karma member could quantify this by posting half their comments from a new account and A/B testing. I could be incorrect, but I recall reading that part of the ranking algorithm is based on the commenter's average karma.

jbenz 13y ago
As I remember it, there was a feature implemented where leaders (veteran accounts with high vote averages) do get something of a boost when they comment, at least within the sequencing of the comments in any one thread. I think this was done to help improve the quality of the conversation, even before the removal of the vote totals next to each comment.
deleted 13y ago
[deleted]
deleted 13y ago
[deleted]
- jmduke 13y ago
  Definitely! This is especially a big issues with focus groups (I had to run a bunch for a few semesters for part of my marketing degree, and one of the first things you learn is how to disrupt an echo chamber: if X and Y say two very positive things about a widget, than Z is suddenly very disincentivized to say anything negative and everyone else is more inclined to join the echo chamber). But I don't think that's whats being brought up with the article. Literary critics are more or less the 'first voters' on new books: and yet, the correlation between critical successes and commercial successes are tentative at best (while the first few votes on an article are always an indicator of future votes.) This is mainly, I think, a function of accessibility: you see the rating of a link on HN before you even click on the link, while most (I'd say 90%+) of the people buying books do so unaware of what everyone before them thought about the book.

samolang 13y ago

When you said it was relevant to HN I thought you were going to say because the same phenomenon happens with start-ups. Being good isn't always enough, usually you need a little luck as well.

kens 13y ago
Yes, totally. I was going to add that in an edit, but you beat me to it :-)

raverbashing 13y ago

Well, in a sense, the previous rating is an heuristic. Downloading and listening a song is "boring" unless the song is good So it's more or less this: "rate this song" is different from "do you like this song that other people thought it was good as well?" Maybe for HN you can randomize the order on the front page and hide the score

deleted 13y ago

[deleted]

Florin_Andrei 13y ago

Society is full of positive (self-enforcing, runaway) feedback loops. It is not self-stabilizing. It tends to flip-flop between extremes. This applies to pretty much any field. It's the strongest argument against laissez-faire economics, yet it's amazing how many people are blind to it (well, that's exactly part of the mechanism that enables the aforementioned feedback loops).

liber8 13y ago
Can you provide one example of such a phenomena operating on anything longer than a very short timescale? I'm genuinely curious. Nearly every example I can think of is long-term negative. edit: I'd venture to say anything longer than 5-10 years could be considered long term for this question. Under that measure, I'd say government spending has exhibited positive-feedback behavior over the last 50+ years. Anything else?
- n6mac41717 13y ago
  Movies: Rocky, 8 million. Star Trek, Infinitum.
- mjfl 13y ago
  The population of New York vs. Philadelphia
- Florin_Andrei 13y ago
  Any decisive battle between two big kingdoms. The "share of the pie" for the upper class in the US in the last few decades.
- Retric 13y ago
  Religion. Fashion ex Louis chairs. Language ex regional accents and new words / phrases. Manners and other social protocalls. Operating systems, social networks, and computer languages.
  - mc32 13y ago
    In Japan, this is even more pronounced. MFGs seek to ride the wave of teen uptake of new fast-moving products. These products tend to enjoy a short-lived, but intense attention by consumers.

bcgraham 13y ago

Stephen King did this and wrote new novels under a pen name, and the books were also best-sellers.

n6mac41717 13y ago
From: http://news.slashdot.org/story/13/07/18/2147231/jk-rowling-should-try-the-voting-algorithm http://news.slashdot.org/story/13/07/18/2147231/jk-rowling-s... "King did say that he thought he was outed too early to obtain any conclusive results from the experiment..."
- bcgraham 13y ago
  Thanks! I didn't know that.
- jonnathanson 13y ago
  The other thing to keep in mind is that well over half the battle in becoming a bestseller is even landing a publisher in the first place. There's serious cumulative advantage inherent to being Stephen King, or JK Rowling, in that you can submit your "anonymous" work to your ordinary publisher and at least guarantee some amount of distribution. A truly anonymous play would be if Rowling or King submitted their pseudonymous work to agents and publishers, unsolicited, without any trace whatsoever of their existing relationships. (This might not be possible due to contactual obligations, however).
  - ars 13y ago
    > A truly anonymous play would be if Rowling or King submitted their pseudonymous work to agents and publishers, unsolicited, without any trace whatsoever of their existing relationships. Didn't Rowling do exactly that with this book?

TylerE 13y ago

Plenty of real world examples of that effect. Look at bands that are huge in say, Japan or Germany but struggle to fill a small club in their home country.

dpeck 13y ago
A prophet is not without honor except in his own town

tpurves 13y ago

That study has been very influential to my own thinking about how things get popular. Another thing I found fascinating about the study was that 'quality' of the content did matter, but that it accounted for roughly half the correlation with success, the other half being random-chance snowball effects. You could probably extend this theory of what makes things successful to a lot of other domains (music, books, fashion, ideas, memes, startups, people...) However, it's possible the ratio of intrinsic qualities vs extrinsic luck varies or is not 50/50 in all domains. With music (or books) you have to invest a few minutes or a few hours to really evaluate if you like something, so those factors may affect how much extrinsic "cumulative advatage" snowball effects count vs intrinsic quality. I'd really like to see more empirical research of how success by "luck vs skill" actually plays out in other categories. Any one know of follow up work to that 2007 study?

bostonvaulter2 13y ago
It seems like this would be a reason to release more high-quality work rather than fewer excellent quality work. Although perhaps the excellent work would stand out more.

anigbrowl 13y ago

Some high-karma member could quantify this by posting half their comments from a new account and A/B testing. As it happens, my wife is a HN member but has not found it easy to get traction for stories she submits. On a couple of occasions I have re-submitted an article that I thought was unfairly overlooked and it has attracted replies and karma in abundance. Anecdata to be sure, but based on this and other observations in HN and other online communities I think your hunch is likely correct. If anyone feels like exploring this in R or somesuch, MetaFilter (where upvotes by other members are public) makes a large amount of its user pattern data available for analysis: http://stuff.metafilter.com/infodump/ http://stuff.metafilter.com/infodump/