4 ms·
Why did they adopt Scala in the first place? I remember around 2012-2014 it was the hawtness. My guess is that there are languages that give you much of the pow
by ralphc 6y ago
Why did they adopt Scala in the first place? I remember around 2012-2014 it was the hawtness. My guess is that there are languages that give you much of the power of Scala without the complexity - Kotlin, and modern Java, having caught up.
- blandflakes 6y agoModern Java is a language I'd be happy to program in, but doesn't have the language constructs you need if what you're loving FP in Scala (easily chainable monad-things). Kotlin is a nice improvement on modern Java, but feels closer to Java than Scala. Arrow is interesting in this space: https://arrow-kt.io/ https://arrow-kt.io/
- stank345 6y ago> Why did they adopt Scala in the first place? I think in the case of Scala many companies became invested in Spark and Scala came along for the ride. Then as time went on, it turned out you could do advanced FP in Scala, despite its warts, with principled libraries like Cats and the Typelevel stack (see https://typelevel.org/cats/ https://typelevel.org/cats/). Companies could then use Scala as a "better Java" or squint-and-it's-almost-Haskell. Or like you said they could just ditch Scala and use something less powerful but easier to learn and work with.
- denimnerd42 6y agoScala got huge because of Spark. I'm a Scala programmer due to Spark. I love it.
- tiew9Vii 6y agoIndeed. The only recent Scala job's I have seen in Australia have been Spark jobs because Spark uses Scala. Even these are diapering for Python Spark roles. I've worked on a few big data projects for large companies. I have a huge dislike for them as mostly seem about how can we fuck over a customer to better the business or do shady stuff with the data rather than anything meaningful. That and it's mostly just ETL, aggregating various data sources, cleaning, putting the data in some central location then running queries on it and the "If the only tool you have is a hammer, you will start treating all your problems like a nail." problem, tiny data sets going from NiFi / Hadoop / etc when `cat ... | awk ...` will be more than sufficient.
- collyw 6y ago>I've worked on a few big data projects for large companies. I have a huge dislike for them as mostly seem about how can we fuck over a customer to better the business or do shady stuff with the data rather than anything meaningful. Sounds similar to the AI hype these days.
- denimnerd42 6y agoYes for us Spark is being used for just ETL. It's internal data though. It's at the scale where no single machine could ever handle it though. It's replacing large proprietary systems like mainframes and teradata. We wrote a framework to heavily simplify writing these ETL flows. No developer of the ETL flows is using Scala unless something custom is required. I wish we could opensource it.
- icedchai 6y agoI worked at a Scala shop in 2011. It was pure hell. Slow compiles, bad tooling (the Scala plugin for Eclipse was a slow as molasses, IntelliJ's was a bit better though), inexperienced developers. I asked why they chose Scala... It was obviously immature at the time. "Joe thought it was good." (Joe was the architect.) I'd never touch it again.
- bcrosby95 6y agoMy theory is that Scala was adopted because Java stagnated. Java isn't stagnant anymore (even if its behind other languages) and a lot of people I know that advocated Scala are just fine with Java now.
- pjmlp 6y agoKotlin is going down the same path, with JetBrains trying to create an eco-system of its own. I bet in a couple of years, as the Java VM platform keeps improving, it will only matter to those stuck in Android. This is a movie with a script I have seen multiple times.