3 ms·
From the README: Reflow comprises: - a functional, lazy, type-safe domain specific language for writing workflow programs; - a runtime for evaluating Reflow
by all2 5y ago
From the README:
Reflow comprises:
- a functional, lazy, type-safe domain specific language for writing workflow programs;
- a runtime for evaluating Reflow programs incrementally, coordinating cluster execution, and transparent memoization;
- a cluster scheduler to dynamically provision and tear down resources from a cloud provider (AWS currently supported).
and
Reflow was designed to support sophisticated, large-scale bioinformatics workflows, but should be widely applicable to scientific and engineering computing workloads. It was built using Go.
Reflow joins a long list of systems designed to tackle bioinformatics workloads, but differ from these in important ways:
- it is a vertically integrated system with a minimal set of external dependencies; this allows Reflow to be "plug-and-play": bring your cloud credentials, and you're off to the races;
- it defines a strict data model which is used for transparent memoization and other optimizations;
- it takes workflow software seriously: the Reflow DSL provides type checking, modularity, and other constructors that are commonplace in general purpose programming languages;
because of its high level data model and use of caching, Reflow computes incrementally: it is always able to compute the smallest set of operations given what has been computed previously.