5 ms·
If you're switching to python operators you should check out the TaskFlow API. You can basically build python operators with just python functions and decorator
by dimberman 6y ago
If you're switching to python operators you should check out the TaskFlow API. You can basically build python operators with just python functions and decorators.
- arthurcolle 6y agoCan you still chain dependent tasks together with the cute ">>" syntax? I like that, makes reasoning about task orderings and co-dependent tasks pretty easy to write and think about.
- dimberman 6y agoUnfortunately that is one feature that we had to take out, but you should check out the TaskFlow API, it's a very worthwhile trade-off! Edit: Sorry I misspoke here The only thing that is no longer allowed is using a bitshift operator between a DAG and a task. task_1 >> task_2 is totally fine my_dag >> task_1 Is no longer allowed. Apologies for the miscommunication.
- shmoogy 6y agoSo if I am on 1.10.6, and upgrade, all of my existing scripts will break as >> is removed?
- dimberman 6y agoWAIT. Sorry I misspoke. The only thing that is no longer allowed is using a bitshift operator between a DAG and a task. task_1 >> task_2 is totally fine my_dag >> task_1 Is no longer allowed. Most of your DAG should be completely fine. Apologies for the miscommunication.
- arthurcolle 6y agoThanks, you almost gave me a heart attack ;)
- shmoogy 6y agoThank you for clarifying, that scared the heck out of me. Is there an official guide of how to update from 1.10.x to 2.0, or do we still need to do the 1.14 update, then move over? I'm interested in updating ASAP, but scared to break my production.
- hn2017 6y agoI posted about that here. They recommend the .14 release. https://stackoverflow.com/q/65345789/220997 https://stackoverflow.com/q/65345789/220997