3 ms·
Ask HN: Any (free) Hacker News datasets around?
As the topic says, I'm looking for datasets containing the data from Hacker News, specifically comments. I've found some on BigQuery that are not very well updated (nor free/downloadable), and via other sources but also not updated.
Mostly looking for data for last three years, but having the data from the beginning > today would obviously be the best, but really hard to find.
- krapp 5y agoIf you have time, you can get everything from the API. Get the latest ID, iterate down and download the comments.
- capableweb 5y agoDoes the API even work anymore? I'm getting prompted to login to Google when trying something like "https://hacker-news.firebaseio.com/v0/item/28065009 https://hacker-news.firebaseio.com/v0/item/28065009"
- krapp 5y agoyou have to add .json at the end
- gabrielsroka 5y agoHere's the current page as seen by the API. https://hacker-news.firebaseio.com/v0/item/28060112.json?print=pretty https://hacker-news.firebaseio.com/v0/item/28060112.json?pri... See also https://github.com/HackerNews/API https://github.com/HackerNews/API
- cpach 5y agoMaybe try Algolia https://hn.algolia.com/api https://hn.algolia.com/api
- karimf 5y agoI believe the one from big query is updated? I just tried using this query SELECT * FROM `bigquery-public-data.hacker_news.full` WHERE type='comment' ORDER BY timestamp desc LIMIT 100 It returns comments as recent as August 2, 2021