Two days ago, everyone: “You should use SQL because everyone knows it and modern SQL is pretty good. Other languages are too complicated. :(“ Everyone today:

Two days ago, everyone: “You should use SQL because everyone knows it and modern SQL is pretty good. Other languages are too complicated. :(“ Everyone today: “tries using sql Oh wow, the tooling is quite basic, and you can’t express complex data structures and imperative code. :(“ What did you expect? Look, I spent 4 years in this rabbit hole, and here’s my advice: Don’t try to put the square peg in the round hole. You want easy to write, simple code and pipelines? Just use sql. Have a dev environment and run everything against that to verify it. Do not bother with unit testing your CTEs, it’s hard to do, there are no good tools to do it. If you want Strong Engineering TM, use python and spark and all the python libraries that exist to do all that stuff. It won’t be as quick to write, or make changes to, but it will be easier to write more verifiably robust code. If you treat either as something it is not (eg. Writing complex data structures and frameworks in sql) you’re using the wrong tool for the outcome you’re trying to achieve. It’ll take longer and “feel bad”, not because the tool is bad, but because you’re using it in a bad way.