2 ms·
Thanks for the wealth of resources. After reading through, the only big concern I have with BigQuery is pricing. I like the model of pay-per-query, but it seems
by paladin314159 10y ago
Thanks for the wealth of resources. After reading through, the only big concern I have with BigQuery is pricing. I like the model of pay-per-query, but it seems like costs can get out of hand easily.
These two issues particularly worry me:
- No non-date partitioning, which means you'll often end up (in my particular use case) scanning 10-100x as much data as you actually want to.
- Pricing is based on uncompressed data size, which is at least several times more expensive than the alternative (e.g. Athena).
Any thoughts on these? We'll have several hundreds of TB of data, so cost matters.
- vgt 10y agoHappy to help! On partitioning - sounds like a great candidate for sharding. BigQuery standard SQL makes it very easy to run unions on shards (ex. Select * from tables_*) On pricing - BigQuery is a very different product functionally, and I encourage you to compare it via a poc. That said, definitely reach out to me on price.