3 ms·
Scala Spark - a bit absurd if you don't need the parallelism, though. Most of the development can be done simply in quick compilation iterations or copied from
by lamp_book 2y ago
Scala Spark - a bit absurd if you don't need the parallelism, though. Most of the development can be done simply in quick compilation iterations or copied from the sbt REPL. Python/pandas feels Stone Age in comparison - you absolutely waste a lot of time iterating with run-time testing.
- Centigonal 2y agoWhy scala spark over pyspark?
- smu3l 2y agoScala (and Java) has a typed Dataset api.[0] pyspark only provides the Dataframe API, which is not typed. [0] https://spark.apache.org/docs/latest/sql-programming-guide.html#datasets-and-dataframes https://spark.apache.org/docs/latest/sql-programming-guide.h...
- Centigonal 2y agothanks!