4 ms·
Wow. I wonder how much a query that searches the content of all of Github costs (if you're not Google). This page says the dataset is 3TB+ https://cloud.google.
by tlrobinson 10y ago
Wow. I wonder how much a query that searches the content of all of Github costs (if you're not Google). This page says the dataset is 3TB+ https://cloud.google.com/bigquery/public-data/github https://cloud.google.com/bigquery/public-data/github and presumably most of that is content.
- zellyn 10y agoYou can do it yourself for free, I believe. https://github.com/blog/2298-github-data-ready-for-you-to-explore-with-bigquery https://github.com/blog/2298-github-data-ready-for-you-to-ex...
- Veratyr 10y agoThe contents table [0] which they ran the query on is 1.8TB. Assuming you only need to do a single pass (seems reasonable given that it's a simple regex), the price should be about $9 [1]. Free quota covers 1TB so the remainder would be $4. [0]: https://bigquery.cloud.google.com/table/bigquery-public-data:github_repos.contents?pli=1&tab=details https://bigquery.cloud.google.com/table/bigquery-public-data... [1]: https://cloud.google.com/bigquery/pricing https://cloud.google.com/bigquery/pricing
- skeletonjelly 10y agoThe data is stored in BigQuery and updated from GitHub weekly. I imagine they do a date range filter when they updated from GitHub