3 ms·
I tend to utilize bigquery's external connections and scheduled queries to periodically clone my "hot" PG data to an BQ table for long term storage and analysis
by spookylettuce 5y ago
I tend to utilize bigquery's external connections and scheduled queries to periodically clone my "hot" PG data to an BQ table for long term storage and analysis.
It seems so much easier to go PG=>BQ than the other way around.
- tlarkworthy 5y agoI second federated table connector (and scheduled queries).
- deleted 5y ago[deleted]
- patwolf 5y agoSame here. We previously used Stitch until they dropped the free tier.
- sa46 5y agoIs the idea something like a scheduled: INSERT INTO bq_table SELECT * FROM EXTERNAL_QUERY(''); I'm guessing you're on the hook for keeping the schema up to date with the Postgres schema.
- nceasy 5y agoI also use the external data connector, the problem is when you have two projects in different locations. Then I go for psycopg2 and uploat it to bigquery using bigquery magic lines