3 ms·
> Since the full dataset is ~285GB, you only get 4 queries per month. That's only true if your 4 queries need to read every single column. One of the big adva
by daave 11y ago
> Since the full dataset is ~285GB, you only get 4 queries per month.
That's only true if your 4 queries need to read every single column.
One of the big advantages of BigQuery's column-oriented storage is that you only pay to read the columns that are actually needed to answer your query.
For example, this query to extract the top 10 authors only cost me 19GB to run (and took 7.0s):
SELECT
author,
COUNT(*) AS COUNT
FROM
TABLE_QUERY([fh-bigquery:reddit_comments], "table_id CONTAINS '20' AND LENGTH(table_id)<8")
GROUP EACH BY
author
ORDER BY
COUNT DESC
LIMIT
10;
author,COUNT
[deleted],228425822
AutoModerator,3677774
conspirobot,575576
ModerationLog,547671
autowikibot,402076
PoliticBot,388395
imgurtranscriber,360248
dogetipbot,358093
qkme_transcriber,301968
TweetPoster,293309
- minimaxir 11y agoGood point. Although if you're doing analysis on the body column itself, it'll use the majority proportion of the data set, of course.