2 ms·
Yeah, that makes sense. I looked at a few workflow orchestrators and I'm building something that I will release soon, but my thinking is that the "workflow engi
by afshinmeh 5mo ago
Yeah, that makes sense. I looked at a few workflow orchestrators and I'm building something that I will release soon, but my thinking is that the "workflow engine" should be an abstraction that takes the input and executes the steps. "What" you use to define that workflow is probably the SDK layer though, but I can certainly see the value in using type safe code to define as opposed to a YAML file.
I'm mainly focusing on the portability aspect of it (e.g. use TS/Python/etc. to define the workflow/steps or just simple a simple YAML file).
- esafak 5mo ago[dead]
- verdverm 5mo agoAre you planning to map those varied definitions onto varied orchestrators?
- afshinmeh 5mo agoSort of. My thinking is that the input to define the workflow should be anything you prefer to use (TS, Go, YAML, etc.) and the orchestrator's job is to model that and execute the job, given your deployment model.
- verdverm 5mo agoThere are a number of widely used orchestrators, it would be nice to deploy to one of those vs a new kid on the block
- afshinmeh 5mo agoI'm mainly looking at Rust based projects and haven't been able to find something to use out of the box, without hacky RPC/Shell execs. Curious if you have any suggestions?
- verdverm 5mo agoThe big data world largely revolves around python, like much of the AI world. Many of the people are more focused on the science than programming, so they aren't interested in the same arguments we often see about rise being a good choice for implementation. They want to use a language they know well to get their job done, hence something like Airflow be asked about in other comments.