2 ms·
ymmv but I kinda like Airflow for automating data ingestion tasks - mainly using its scheduling to run Python scripts that ETL data into bigquery. BigQuery can
by mattbillenstein 7y ago
ymmv but I kinda like Airflow for automating data ingestion tasks - mainly using its scheduling to run Python scripts that ETL data into bigquery.
BigQuery can also directly query data on gcs now I believe, so maybe that's an option if you don't need to do any transforms.