My experience with "Scala at scale" was extremely unsatisfactory and I found Databricks to give very unhelpful responses to basic performance problems arising f

My experience with "Scala at scale" was extremely unsatisfactory and I found Databricks to give very unhelpful responses to basic performance problems arising from the intersection of Scala and Spark, bordering on gaslighting. Memory bloat is one of the most pernicious issues, you have to handle large data very carefully to not end up with massive memory footprints that outstrip any available hardware (and causes hard to debug cascading failures).