3 ms·
Not only it happened at the worst possible time, but some pain points are entirely self-inflicted, like the braceless syntax hurting tooling and fragmenting the
by hocuspocus 2mo ago
Not only it happened at the worst possible time, but some pain points are entirely self-inflicted, like the braceless syntax hurting tooling and fragmenting the community for very little reason.
But in my opinion Databricks implicitly saying "no thanks" for the foreseeable future did a lot of damage here. Databricks didn't even bother bringing their proprietary runtime to Scala 2.13 until last year. Scala 3 is not entirely out of the question but it's taking forever, and at this point users can seriously ask themselves if Spark will actually stay relevant that long.
- ATMLOTTOBEER 2mo agoNot to mention pyspark stealing mindshare
- hocuspocus 2mo agoBoth sides of the same coin, I'd be surprised if Databricks wasn't looking into getting rid of the JVM entirely. The data processing space is increasingly tangled with ML and AI, Python on top of C++ and Rust libraries makes sense, keeping the JVM to orchestrate tasks and shuffle data around, not so much.