3 ms·
> Data pipelines (eg Apache Airflow) where the orchestration is handled by Rust but the high-level business logic is defined using JavaScript That is ... exact
by rubenfiszel 3y ago
> Data pipelines (eg Apache Airflow) where the orchestration is handled by Rust but the high-level business logic is defined using JavaScript
That is ... exactly what windmill is https://github.com/windmill-labs/windmill https://github.com/windmill-labs/windmill
The orchestrator is built from scratch in rust using postgresql to store the state. The steps when in typescript are run in deno: https://github.com/windmill-labs/windmill/blob/main/backend/windmill-worker/src/worker.rs#L1498 https://github.com/windmill-labs/windmill/blob/main/backend/... and the transforms between the steps (piping outputs of any node to the input of any node) are actually javascript expressions run by deno JsRuntimes: https://github.com/windmill-labs/windmill/blob/f3ec9ca09dc372526ec291567f28514e5a7413e5/backend/windmill-worker/src/js_eval.rs#L70 https://github.com/windmill-labs/windmill/blob/f3ec9ca09dc37...