3 ms·
Where are you using/advocating crabwalk? It does look interesting, but for the local ETL use case, I am missing the pitch on just having my own collection of S
by 0cf8612b2e1e 2y ago
Where are you using/advocating crabwalk?
It does look interesting, but for the local ETL use case, I am missing the pitch on just having my own collection of SQL scripts. Presumably the all-local case needs less complexity. Unless the idea is that this will eventually support more connectors/backends and work as a full dbt replacement?
- mritchie712 2y agoA few features: * Built-in column level lineage (i.e. dump in 20 .sql files and crabwalk automatically figures out lineage) * Visualize the lineage * Clean handling of input / output (e.g. simply specify @config output and you can export results to parquet, csv, etc.) * Tests are not yet implemented, but crabwalk will have built-in support for tests (e.g. uniqueness, joins, etc.) we're using it in our product (https://www.definite.app/ https://www.definite.app/), but only for lineage right now.