3 ms·
We have been using Luigi for a larger project and it works fine. Some people have a bit of a hard time understanding what it is about and why at least some soft
by mtrn 11y ago
We have been using Luigi for a larger project and it works fine. Some people have a bit of a hard time understanding what it is about and why at least some software for scheduling is needed. I find the notion of "make for data" useful.
After a presentation on Luigi in a Python User Group, we had a lively discussion about certain features. One issue that came up was the fact, that downstream tasks are not necessarily recomputed, once you change something in the code. For that to happen, you would have to keep track of the source code as well. Similarities with Nix came up, where a change in code leads to a different ID, so all changes can be tracked.
Shameless plug: When I started using Luigi, I missed some auto-generated filename feature for task outputs, so I wrote a utility library for that (and a few other things): https://git.io/vg4D0 https://git.io/vg4D0