16 ms·
Looks like it stopped updating as of February 2nd, but otherwise it's pretty reliable, and as noted in the description, it's free. (you probably won't hit the 1
by minimaxir 8y ago
Looks like it stopped updating as of February 2nd, but otherwise it's pretty reliable, and as noted in the description, it's free. (you probably won't hit the 1TB limit working with this dataset). Here's a few queries I've done recently to answer ad-hoc questions to get an exact answer:
Top posts about bootstrapping (https://news.ycombinator.com/item?id=19258249 https://news.ycombinator.com/item?id=19258249):
#standardSQL
SELECT *
FROM `bigquery-public-data.hacker_news.full`
WHERE REGEXP_CONTAINS(title, '[Bb]ootstrap')
ORDER BY score DESC
LIMIT 100
Count of YC startup posts over time by month (https://news.ycombinator.com/item?id=19185946 https://news.ycombinator.com/item?id=19185946):
#standardSQL
SELECT TIMESTAMP_TRUNC(timestamp, MONTH) as month_posted,
COUNT(*) as num_posts_gte_5
FROM `bigquery-public-data.hacker_news.full`
WHERE REGEXP_CONTAINS(title, 'YC [S|W][0-9]{2}')
AND score >= 5
AND timestamp >= '2015-01-01'
GROUP BY 1
ORDER BY 1
- avian 8y agoOne thing that I was missing last time I checked was comment ranking data. Neither score nor rank was there for comments posted in recent years. I understand that upvote counts are not available in the API, but ranking should be (as in, the order the comments appear on the page).
- shanecglass 8y agoHey all, I manage the BigQuery Public Datasets Program here at Google. You're right, the dataset last updated February 2nd, but we intend to continuing updating it. We had an issue on our end that disrupted our update feed, but we're working to repair it now and get the latest data uploaded to BigQuery.
- minimaxir 8y agoGreat to hear! :)