Spark lets users query via regular SQL or DSL (Scala, PySpark). The DSLs are generally better to work with. Complex queries can be broken up into composable f

Spark lets users query via regular SQL or DSL (Scala, PySpark). The DSLs are generally better to work with. Complex queries can be broken up into composable functions that can be mixed and matched to perform different analyses. Functions are easier to unit test, document, deal with quoting better, can be type safe, etc. Some super-complex queries are still better to write with regular SQL. New query DSLs don't need to reinvent the wheel. They can implement their query capabilities using the Row / Column / DataFrame abstractions that are elegant and familiar in popular projects like Spark. We need an ANSI query engine DSL.