4 ms·

One of the biggest issues I've observed with workflow systems is an inability to get return values from each subprocess and then decide based on those which sub

One of the biggest issues I've observed with workflow systems is an inability to get return values from each subprocess and then decide based on those which subprocesses to instantiate next. It always seems like the workflow execution graph is static in my experience. Is that correct?

brad0 3y ago
That sounds very similar to AWS Step Functions to me. You define a workflow graph in JSON, then run the workflow and hand out subprocesses to lambdas etc.
- 8note 3y ago
  I've definitely seen workflows written as single step graphs, where the output is opening a new workflow with a different single step It was hard to read and understand
sisve 3y ago
Maybe before, but i feel that is not a problem anymore. Windmill.dev solves this with easee. Very good UI on top of the workflow engine.
noselasd 3y ago
The couple workflow engines I've used supports that. It sounds crazy to not support something that obvious
8note 3y ago
Why would you want to rewrite the code of the workflow during execution, vs defining that logic upfront?
- jdeaton 3y ago
  That's like asking why anyone would want to use an if statement. "Why would you want to change which instructions that get executed during the program? Just decide up front!". Because the decision cannot be made ahead of time.
jrop 3y ago
At a prior company we just wrote our own custom workflow engine which was basically a DAG processor where each node called some code. Of course, the code called could modify the DAG in place and accomplish exactly what you're describing. It worked quite well but we didn't polish it as much as we could have. I've been wanting to do a clean-room reimplementation as of late just because I miss certain things about it.
atombender 3y ago
Not sure what you've tried, but Temporal (which I think is the most mature and well-designed open source workflow engine available right now) supports using the result of a "subprocess" to decide the next steps. The only requirement is that your logic is consistent across workflow runs.

brad0 3y ago

That sounds very similar to AWS Step Functions to me. You define a workflow graph in JSON, then run the workflow and hand out subprocesses to lambdas etc.

8note 3y ago
I've definitely seen workflows written as single step graphs, where the output is opening a new workflow with a different single step It was hard to read and understand

sisve 3y ago

Maybe before, but i feel that is not a problem anymore. Windmill.dev solves this with easee. Very good UI on top of the workflow engine.

noselasd 3y ago

The couple workflow engines I've used supports that. It sounds crazy to not support something that obvious

8note 3y ago

Why would you want to rewrite the code of the workflow during execution, vs defining that logic upfront?

jdeaton 3y ago
That's like asking why anyone would want to use an if statement. "Why would you want to change which instructions that get executed during the program? Just decide up front!". Because the decision cannot be made ahead of time.

jrop 3y ago

At a prior company we just wrote our own custom workflow engine which was basically a DAG processor where each node called some code. Of course, the code called could modify the DAG in place and accomplish exactly what you're describing. It worked quite well but we didn't polish it as much as we could have. I've been wanting to do a clean-room reimplementation as of late just because I miss certain things about it.

atombender 3y ago

Not sure what you've tried, but Temporal (which I think is the most mature and well-designed open source workflow engine available right now) supports using the result of a "subprocess" to decide the next steps. The only requirement is that your logic is consistent across workflow runs.