4 ms·

"HACKER NEWS - A dataset that contains all stories and comments from Hacker News since its launch in 2006." I know what I'm doing this weekend.

panarky 11y ago
I'm dying to get comment scores, not just submission scores. There's a lot you could do with that to find the best comments, which is really why HN is so awesome.
- Alex3917 11y ago
  > There's a lot you could do with that to find the best comments. In any given situation, how often does best comment you could write coincide with the one that would get you the most upvotes? I'm guessing not frequently.
  - Estragon 11y ago
    Your best is different from his/her best.
- minimaxir 11y ago
  Blame HN for that. It was removed from public access right after I made a blog post about it, although it was coincidental I think.
  - jrockway 11y ago
    Though the data could be assembled; I can see my comment scores, you can see your comment scores...
pella 11y ago
similar: https://hn.algolia.com/ https://hn.algolia.com/ ( Search Hacker News )
jrockway 11y ago
It's mildly amusing: > SELECT sum(length(text)) FROM [bigquery-public-data:hacker_news.comments] where author="jrockway" 4830955 That's almost 5 MB of comments I've written.
- toomuchtodo 11y ago
  Get outside more :)
  - bcook 11y ago
    Being outside does not preclude internet access... :)
jonesb6 11y ago
Would be interesting to find out the value (upvotes) of sponsored content vs. unsponsered content, as well as comments from green users vs normal users.
- minimaxir 11y ago
  There isn't sponsored content on HN. (if you are referring to the job ads, those do not receive upvotes and follow a steady rank decay.)
  - teddyh 11y ago
    What? The lack of upvotes (or steady decay) does not make it “not content”.
- deleted 11y ago
  [deleted]
fhoffa 11y ago
Some resources to get you started: - How to use the Hacker News dataset https://medium.com/google-cloud/big-data-stories-in-seconds-hacker-news-abe52bc5caad#.pjfjzr76j https://medium.com/google-cloud/big-data-stories-in-seconds-... - Discussion of the HN dataset announcement here https://news.ycombinator.com/item?id=10440502 https://news.ycombinator.com/item?id=10440502 - An iPython notebook: https://github.com/fhoffa/notebooks/blob/master/analyzing%20hacker%20news.ipynb https://github.com/fhoffa/notebooks/blob/master/analyzing%20... - More: http://debarghyadas.com/writes/looking-back-at-9-years-of-hacker-news/ http://debarghyadas.com/writes/looking-back-at-9-years-of-ha... Disclaimer: I'm Felipe Hoffa, and I work at Google. (https://twitter.com/felipehoffa https://twitter.com/felipehoffa)
- bambax 11y ago
  How fresh is the data, and how often is it refreshed? It doesn't seem to be described anywhere...
  - dwmintz 11y ago
    Looks like the data goes up through 2015-10-13. I created a Look (disclosure: work for Looker) that shows story counts by day for the last 365 days here: https://looker.com/publicdata/looks/169?show=viz https://looker.com/publicdata/looks/169?show=viz

panarky 11y ago

I'm dying to get comment scores, not just submission scores. There's a lot you could do with that to find the best comments, which is really why HN is so awesome.

Alex3917 11y ago
> There's a lot you could do with that to find the best comments. In any given situation, how often does best comment you could write coincide with the one that would get you the most upvotes? I'm guessing not frequently.
- Estragon 11y ago
  Your best is different from his/her best.
minimaxir 11y ago
Blame HN for that. It was removed from public access right after I made a blog post about it, although it was coincidental I think.
- jrockway 11y ago
  Though the data could be assembled; I can see my comment scores, you can see your comment scores...

pella 11y ago

similar: https://hn.algolia.com/ https://hn.algolia.com/ ( Search Hacker News )

jrockway 11y ago

It's mildly amusing: > SELECT sum(length(text)) FROM [bigquery-public-data:hacker_news.comments] where author="jrockway" 4830955 That's almost 5 MB of comments I've written.

toomuchtodo 11y ago
Get outside more :)
- bcook 11y ago
  Being outside does not preclude internet access... :)

jonesb6 11y ago

Would be interesting to find out the value (upvotes) of sponsored content vs. unsponsered content, as well as comments from green users vs normal users.

minimaxir 11y ago
There isn't sponsored content on HN. (if you are referring to the job ads, those do not receive upvotes and follow a steady rank decay.)
- teddyh 11y ago
  What? The lack of upvotes (or steady decay) does not make it “not content”.
deleted 11y ago
[deleted]

fhoffa 11y ago

Some resources to get you started: - How to use the Hacker News dataset https://medium.com/google-cloud/big-data-stories-in-seconds-hacker-news-abe52bc5caad#.pjfjzr76j https://medium.com/google-cloud/big-data-stories-in-seconds-... - Discussion of the HN dataset announcement here https://news.ycombinator.com/item?id=10440502 https://news.ycombinator.com/item?id=10440502 - An iPython notebook: https://github.com/fhoffa/notebooks/blob/master/analyzing%20hacker%20news.ipynb https://github.com/fhoffa/notebooks/blob/master/analyzing%20... - More: http://debarghyadas.com/writes/looking-back-at-9-years-of-hacker-news/ http://debarghyadas.com/writes/looking-back-at-9-years-of-ha... Disclaimer: I'm Felipe Hoffa, and I work at Google. (https://twitter.com/felipehoffa https://twitter.com/felipehoffa)

bambax 11y ago
How fresh is the data, and how often is it refreshed? It doesn't seem to be described anywhere...
- dwmintz 11y ago
  Looks like the data goes up through 2015-10-13. I created a Look (disclosure: work for Looker) that shows story counts by day for the last 365 days here: https://looker.com/publicdata/looks/169?show=viz https://looker.com/publicdata/looks/169?show=viz