I know this thing you want by the name “incremental” processing. GrailBio’s pipeline tech is incremental, as is Lyft’s Flyte I think. Build systems such as Baz

I know this thing you want by the name “incremental” processing. GrailBio’s pipeline tech is incremental, as is Lyft’s Flyte I think. Build systems such as Bazel are build on the same incremental DAG processing foundation. I agree that it’s the most important feature, as it dramatically improves correctness and performance (when at scale). IAM policy trouble is a good example of a bug that crashes a pipeline but has no influence on precomputed outputs of succeeded nodes.