4 ms·
I try to avoid GUI-based tools as they generally don't lend themselves well to version control, code reuse, etc. There are several orchestration tools that allo
by westonsankey 7y ago
I try to avoid GUI-based tools as they generally don't lend themselves well to version control, code reuse, etc. There are several orchestration tools that allow you to create arbitrarily complex ETL pipelines using SQL, Python, Spark, etc. Here are links to a few:
http://airflow.apache.org/ http://airflow.apache.org/
https://www.prefect.io/ https://www.prefect.io/
https://dagster.readthedocs.io/en/0.6.7/ https://dagster.readthedocs.io/en/0.6.7/
https://www.getdbt.com/ https://www.getdbt.com/
- veritas3241 7y agodbt is a fantastic tool. We really on it for basically all of our transformations at GitLab.
- pjot 7y ago+1 for dbt. Great tool with a really active community along with it
- eyeball 7y agoDbt looks awesome. Too bad no oracle support.
- drewbanin 7y agoHi there - I run product for dbt - happy to chat more about Oracle support if you're interested. There are a handful of community-supported dbt plugins out in the wild. Oracle could very well be one of the next ones. Check some of these out to see what's involved in building a database plugin: - https://github.com/fishtown-analytics/dbt-spark https://github.com/fishtown-analytics/dbt-spark - https://github.com/fishtown-analytics/dbt-presto https://github.com/fishtown-analytics/dbt-presto - https://github.com/mikaelene/dbt-sqlserver https://github.com/mikaelene/dbt-sqlserver - https://github.com/jacobm001/dbt-mssql https://github.com/jacobm001/dbt-mssql
- eyeball 7y agocool. would be an interested user, but building a plug in ... beyond my team's ability.