Yeah, that's exactly the issue. One can do this in Python, but not in SQL, and this leads to boilerplate. I actually think that SparkSQL is a good solution her

Yeah, that's exactly the issue. One can do this in Python, but not in SQL, and this leads to boilerplate. I actually think that SparkSQL is a good solution here, as you can create easily reusable functions.