3 ms·
Our backend is primarily in Scala and has been for over a decade. My take is that Scala 2 -> 3 was a massive hit to the language's momentum. It came at a time w
by bearforcenine 2mo ago
Our backend is primarily in Scala and has been for over a decade. My take is that Scala 2 -> 3 was a massive hit to the language's momentum. It came at a time when the language's popularity was waning and it had all the same challenges that other major language version changes tend to have. Migrating's been quite challenging. Tooling, which was already a challenge for the language, got worse because it was split between Scala 2 and 3.
The language was deliberate about providing backwards and forwards compatibility between 2 and 3, but they didn't have a good story for macros. Yes, macros were experimental, but the ecosystem relied on them very heavily. This meant that your 3rd party dependencies likely hadn't migrated to Scala 3. If they had, things got complicated because you had to deal with two macro systems.
After many years the tooling is finally catching up and many 3rd party dependencies are available in Scala 3. That said, it's taken years and years. I think during those years many companies migrated to other languages or put a moratorium on new projects in Scala.
- hocuspocus 2mo agoNot only it happened at the worst possible time, but some pain points are entirely self-inflicted, like the braceless syntax hurting tooling and fragmenting the community for very little reason. But in my opinion Databricks implicitly saying "no thanks" for the foreseeable future did a lot of damage here. Databricks didn't even bother bringing their proprietary runtime to Scala 2.13 until last year. Scala 3 is not entirely out of the question but it's taking forever, and at this point users can seriously ask themselves if Spark will actually stay relevant that long.
- ATMLOTTOBEER 2mo agoNot to mention pyspark stealing mindshare
- hocuspocus 2mo agoBoth sides of the same coin, I'd be surprised if Databricks wasn't looking into getting rid of the JVM entirely. The data processing space is increasingly tangled with ML and AI, Python on top of C++ and Rust libraries makes sense, keeping the JVM to orchestrate tasks and shuffle data around, not so much.