2 ms·
Ask HN: Make for SQL scripts?
I have been searching for a better way to execute a chain of SQL scripts than a simple bash script, but without the complexity of a full blown workflow manager like Luigi or Airflow - which are really fine solutions for robust ETL processes, but overkill for some interactive analyses.
Ideally, I would like to define a "makefile" in something clean like YAML, and call a routine that reads it as a DAG and executes scripts in order of dependency with appropriate parallelism.
Snakemake [1] is one option, but as its geared towards processes that read and write files, some operations that don't emit data (like calling psql on an SQL script file), require some hacks.
Is there a _simple_ workflow manager that fits this category? We are mostly a Python shop, but languages such as Go are also in use.
[1] https://snakemake.readthedocs.io