2 ms·
My understanding is each task is supposed to idempotent, so I don't believe this is a valid use for Airflow.
by shmoogy 5y ago
My understanding is each task is supposed to idempotent, so I don't believe this is a valid use for Airflow.
- arthurcolle 5y agoYeah your understanding is correct but if you relax the idempotency constraint you can achieve a lot more with just a little bit extra logic in your interface layer with other internal services or potentially other mechanisms to ensure consistency. YMMV