4 ms·
When Max started building airflow Airbnb's engineering team was probably 80 or so people. The data engineering and data infrastructure teams couldn't have been
by bhntr3 6y ago
When Max started building airflow Airbnb's engineering team was probably 80 or so people. The data engineering and data infrastructure teams couldn't have been more than ten combined. I'm not sure that's massive but it is the point where the previous solution (Chronos: https://medium.com/airbnb-engineering/chronos-a-replacement-for-cron-f05d7d986a9d https://medium.com/airbnb-engineering/chronos-a-replacement-...) was starting to strain.
Airflow may not be necessary for a ten person team but it could be if that team has complicated data orchestration needs and doesn't want to incrementally replace their infrastructure every couple years.