Spark 3 was released this year: https://databricks.com/blog/2020/06/18/introducing-apache-spark-3-0-now-available-in-databricks-runtime-7-0.html https://databri

Spark 3 was released this year: https://databricks.com/blog/2020/06/18/introducing-apache-spark-3-0-now-available-in-databricks-runtime-7-0.html https://databricks.com/blog/2020/06/18/introducing-apache-sp... Almost entirely backwards compatible with huge performance gains.