4 ms·
Actually that's not true. Have a look at the airflow project by AirBnB. https://github.com/airbnb/airflow https://github.com/airbnb/airflow
by throwaway-anon 10y ago
Actually that's not true. Have a look at the airflow project by AirBnB.
https://github.com/airbnb/airflow https://github.com/airbnb/airflow
- squeaky-clean 10y agoDoesn't seem like it can handle processing on real-time streams of data. They don't really seem comparable. From the readme: "Airflow is not a data streaming solution. Tasks do not move data from one to the other (though tasks can exchange metadata!). Airflow is not in the Spark Streaming or Storm space, it is more comparable to Oozie or Azkaban."
- brianwawok 10y agoThat doesn't really seem the same at all.
- throwaway-anon 10y agoI was replying to a comment not saying airflow is capable of stream processing. > Not for multinode analytics of huge data sets. Airflow does this just fine with celery.