Let's say you're loading data from Postgres into Snowflake. Pretty simple integration ETL. You need to: * Make sure you're only loading incremental updates *

Let's say you're loading data from Postgres into Snowflake. Pretty simple integration ETL. You need to: * Make sure you're only loading incremental updates * Make sure you don't miss any data while doing incremental updates. This includes deleted rows. * Update the Snowflake table schemas as Postgres schemas change. If this is impossible you need to alert someone. * Keep historical metrics so you know if the job is slow or too much data or whatever. Now do this for twenty different data sources every hour.