3 ms·
> I've seen many ETL scripts written where a simple SQL statement would have been better. SQL queries tend to work after a few queries have been verified to be
by toto444 5y ago
> I've seen many ETL scripts written where a simple SQL statement would have been better. SQL queries tend to work after a few queries have been verified to be correct, ETL jobs in languages like java can dump mysterious stack traces referencing many frameworks breaking due to data issues, memory issues, or unhandled cases.
This is so true. I write data pipelines at work. I only use SQL to move data around for this very reason. You never get any bug to fix with SQL. You sometimes need to adjust some queries to some new bad input but never ever will you see : 'index out of bound' or 'wrong argument' or whatever.
- tremon 5y agoYou sometimes need to adjust some queries to some new bad input but never ever will you see : 'index out of bound' or 'wrong argument' or whatever. Just wait until you encounter the "String or binary data would be truncated" error. I guarantee you it will make you long even for Java stack traces.