3 ms·

I've been thinking lately about running some analysis of my own, but couldn't find a data dump and didn't feel like scraping. Why has the dump been pulled? I'm

by lt 15y ago

I've been thinking lately about running some analysis of my own, but couldn't find a data dump and didn't feel like scraping. Why has the dump been pulled? I'm thinking of classifying entries as temporal/atemporal. Temporal are industry news, press releases, rumors and alike. Atemporal are reference materials, essays, howtos, analysis, links to libraries/frameworks/products. Essentially, I feel that while temporal entries provide a more immediate reward and larger discussions, atemporal tend to be more technical, more interesting, have more valuable discussions, and overall are what sets HN apart of other places. I'd rather see a front page full of erlang stories than pieces about what Apple has just announced (I'm pretty sure I'll see these in many different places). A good balance is probably ideal: a few most interesting breaking stories, and the most thought-provoking atemporal materials users have ran into lately. The perceived decline of the front page and comment quality could be attributed to a shift in this balance towards temporal stories. As to how to classify, I'd start as marking stories with multiple duplicates in a short period of time as temporal, and the ones with duplicates spread apart over long periods as atemporal, and go from there.

rflrob 15y ago
It's not obvious to me how to automatically classify stories as (a)temporal. My understanding of the way submissions work is that if Bob submits something that Alice also recently[1] submitted, then it just gets counted as an upvote for Alice's story. Then, of course, there are the "this happened recently, which inspired me to think about some broader trend". For example, during the AWS outage, Coding Horror had a story about Netflix's Chaos Monkey[2], which one could easily classify as both temporal and atemporal. [1] For some definition of "recently" that I haven't yet figured out. [2] http://www.codinghorror.com/blog/2011/04/working-with-the-chaos-monkey.html http://www.codinghorror.com/blog/2011/04/working-with-the-ch...
- JoachimSchipper 15y ago
  Anything that has been re-submitted is atemporal. That doesn't catch nearly everything, but...
- lt 15y ago
  I believe your understanding is right, recently being a month, provided it's the exact same URL. Popular news are usually submitted from different sources so there are actual duplicates. I agree that it's certainly not easy to automate the classification. The Coding Horror example is a very good one. Jeff write very atemporal stories, I'm pretty sure that happens intentionally. It's not unusual to see one of his posts from several years ago pop in the front page every now and then. Same with Joel's. It may be inspired by a recent event, but the lesson is meant to stand. I think a good heuristic would be: given a random, highly voted story from the archives, would it get upvoted if reposted today?

by lt 15y ago

rflrob 15y ago

It's not obvious to me how to automatically classify stories as (a)temporal. My understanding of the way submissions work is that if Bob submits something that Alice also recently[1] submitted, then it just gets counted as an upvote for Alice's story. Then, of course, there are the "this happened recently, which inspired me to think about some broader trend". For example, during the AWS outage, Coding Horror had a story about Netflix's Chaos Monkey[2], which one could easily classify as both temporal and atemporal. [1] For some definition of "recently" that I haven't yet figured out. [2] http://www.codinghorror.com/blog/2011/04/working-with-the-chaos-monkey.html http://www.codinghorror.com/blog/2011/04/working-with-the-ch...

JoachimSchipper 15y ago
Anything that has been re-submitted is atemporal. That doesn't catch nearly everything, but...
lt 15y ago
I believe your understanding is right, recently being a month, provided it's the exact same URL. Popular news are usually submitted from different sources so there are actual duplicates. I agree that it's certainly not easy to automate the classification. The Coding Horror example is a very good one. Jeff write very atemporal stories, I'm pretty sure that happens intentionally. It's not unusual to see one of his posts from several years ago pop in the front page every now and then. Same with Joel's. It may be inspired by a recent event, but the lesson is meant to stand. I think a good heuristic would be: given a random, highly voted story from the archives, would it get upvoted if reposted today?