I've tend to use both SQL and a dataframe API library. I guess it is sort of a a best of both worlds. SQL for the high level, large aggregations, dataframe li

I've tend to use both SQL and a dataframe API library. I guess it is sort of a a best of both worlds. SQL for the high level, large aggregations, dataframe library for when the equivalent SQL statement would be very tedious or hard to debug.