3 ms·
> A week ago I wanted to learn more about data engineering - got a whole Apache Airflow cluster setup with a single compose file, took a few minutes. Off-topic
by bakuninsbart 3y ago
> A week ago I wanted to learn more about data engineering - got a whole Apache Airflow cluster setup with a single compose file, took a few minutes.
Off-topic, but how did you like it? Tried it out a couple of years ago and felt like it overcomplicates things for probably 99% of use cases, and the overhead is huge.
- zaptheimpaler 3y agoOver-complicated, I agree. "This cluster could have been a Makefile" There probably are good reasons to use it when you have complex distributed DAGs though.