3 ms·
Disclaimer: I'm the PM for Composer. :) If the cost of Composer is an issue, ping me. Running a static environment _does_ have a cost, but for serious ETL it s
by chimerasaurus 8y ago
Disclaimer: I'm the PM for Composer. :)
If the cost of Composer is an issue, ping me. Running a static environment _does_ have a cost, but for serious ETL it should be pretty inexpensive all things considered. You _should_ be able to use Airflow (in GCP or anywhere else) to call on other services, like S3/Redshift to operate without moving the data through Airflow, keeping network tx low.
If it's network traffic for the actual data moving to and from, that's unfortunately an artifact of how public clouds price.
- occams_chainsaw 8y agoQQ, why is it called Google cloud composer? I totally would've named it Gooflow
- chimerasaurus 8y agoMy key advice for any new PM: Every Dilbert cartoon about naming a product is true.