3 ms·
Can you query parquet from bigquery without loading it into a table from gcs? I've gotten pretty far with jsonl on gcs and bigquery - even some bigquery stream
by mattbillenstein 7y ago
Can you query parquet from bigquery without loading it into a table from gcs?
I've gotten pretty far with jsonl on gcs and bigquery - even some bigquery streaming for more real-time stuff.
- lmkg 7y agoIf the data is in Cloud Storage, BigQuery can query it in-place without loading it. BigQuery calls this an External Data Source. https://cloud.google.com/bigquery/external-data-sources https://cloud.google.com/bigquery/external-data-sources My biggest papercut with using this was having to make sure that all of the locations matched exactly.