4 ms·
How does it compare to big iron enterprise ETL tools like IBM datastage? I have only dabbled but it looks far more appealing for a variety of reasons.
by rhombocombus 8y ago
How does it compare to big iron enterprise ETL tools like IBM datastage? I have only dabbled but it looks far more appealing for a variety of reasons.
- philosopherlawr 8y agoETL tools just can't compete with a tool that forces code to do anything. It might seem backwards, but we've abandoned all non-code environments and force pure-code for configuration for all of our pipelines.
- caravel 8y agoFor context, I used Datastage, Informatica, Ab Initio and SSIS in previous lives an went on to write the first version of Airflow. I developed a taste for pipelines-as-code while working at Facebook using an internal tool that is not open source. I'd argue that pipelines as code, as opposed to dragndrop GUIs, is a better approach, at least for people who are comfortable writing code. Code is easy to version, test, diff, collaborate and allows for the creation of arbitrary abstractions.