5 ms·
My experience with SSIS 2012, Microsoft's low-code ETL tool, suggests that you're trading code for a giant config file. Unless carefully planned out to be modu
by NortySpock 6y ago
My experience with SSIS 2012, Microsoft's low-code ETL tool, suggests that you're trading code for a giant config file.
Unless carefully planned out to be modular, you end up with a giant pile of technical debt that is
(a) pretty opaque in its error messages
(b) has poor error handling systems
(c) is brittle in its ability to handle changing file formats or business needs
(d) can only be tested end-to-end, with no space for small unit tests to act as guide rails
(e) makes it very easy to hide business logic all throughout the ETL process
Upside to this tool: It seems to be handling a lot of simple parallelization speedups behind the scenes for you, and catches most type-conversion errors.
- commandlinefan 6y ago> a giant config file Which, inevitably, starts to include things like macros, include files, looping constructs and eventually bits of perl or TCL.
- TeMPOraL 6y agoOr more typically, a half-baked Lisp that uses XML or JSON instead of s-expressions. Code is data, but the converse of that is that a sufficiently complex configuration file is just a piece of executable code.
- seabrookmx 6y agoSee also: Pentaho and it's associated tooling, as well as many "workflow management" tools like Camunda. I'd stay as far away from all of these tools as possible.