7 ms·

A summary of what Scala is great for: - Safe and high performant concurrent programming. Scala is miles ahead of most other languages here and pretty much on p

A summary of what Scala is great for: - Safe and high performant concurrent programming. Scala is miles ahead of most other languages here and pretty much on par with the best ones (such as Haskell) - ETL / data transformations. Python is a big player here - but for stable and performant data pipelines, I strongly believe Scala is the better choice. For explorative things, python has the edge though. - Actor systems on the JVM. I believe that most "scalable" systems are over-engineered. A well configured postgres / elasticsearch and a machine with your application on good hardware goes _very_ far. But sometimes it's not enough and then Scala offers a great solution with Akka on the JVM. Erlang/Elixir with their runtime are probably even better, but Scala offers the better typesystem and one can stay in the JVM world if they want. - in-language custom DSLs. No seriously, Scala is the best language I know in terms of building typesafe, customizable DSLs within the language, making it almost look like another language. This is great if not so techy folks are to make changes to actual code in an easy way. The link in this post here shows some examples but it goes beyond. - Writing glue-code. Surprisingly, I think Scala is better than python and many other languages here, because of the sheer power of composability of the language and ways to connect pieces with each other in a typesafe manner. The reason for this lies mostly in the concept of implicits, which both enables great power and reusability but also makes the learning curve much steeper. This list might be a bit subjective and I don't know every language out there, but as the questions comes up almost everytime, I wanted to list the strengths of Scala in the areas where most other languages can't compete with it.

jim-jim-jim 6y ago
Well said. In some of my older comments I've shat on Scala (compared to Haskell) but after being forced to write Python for a day job, I've realized how good I had it with Scala. Luckily I'll be writing some again soon.
threeseed 6y ago
Scala is incredible for web apps. I can write my React/JS front end [1] and highly concurrent, real-time scalable backend [2] using the same language, build tools etc. [1] http://slinky.dev http://slinky.dev [2] http://zio.dev http://zio.dev
wiradikusuma 6y ago
"For explorative things, python has the edge" -- I'm a lazy programmer, how can I get contextual auto-complete like I can with Scala (and other type safe languages)? From my experience, auto-complete for Python shows everything (like JavaScript's), so I end up googling.
- valenterry 6y ago
  From my experience, for exploration of data, you usually use the functionality you already know and you visualize things as much as possible. So there is not a huge advantage for Scala in terms of autocompletion and such, but there is quite a bit of advantage for python for batteries-included, visualisation and speed of execution. But yeah, I get your point, I find it annoying too.
- marcinzm 6y ago
  For exploratory work you're in a notebook so your auto-complete usually has access to the exact object instances in question. There's also some good machine learning based autocomplete systems out there nowadays.
Fiahil 6y ago
I agree with your first point, Scala allows writing incredibly performant concurrent code. However, if -like me- you can't bear the thought of working with Scala anymore, Rust is an excellent competitor. Note, I spent around 5 years working with Scala full-time and grew a profound aversion for the language and its ecosystem. It definitively affects my judgment. I found in Rust all the "good parts" of Scala minus the things I resented.
- aphexairlines 6y ago
  what did you resent?
  - Fiahil 6y ago
    In no particular order: - the JVM - the tooling for writing with VSCode - SBT - NullPointerExceptions bubbling up from Java libs - the split between the functional programming community and the "Scala is just Java without ;" one - everything related to implicits (creation, resolution, etc) and everyone using them - the ability to write the same 0-100 loop with 4 different keywords and 3 different styles - the inability to enforce a coding style properly and consistently with a team of more than 1 engineer - too much people writing DSLs for no particular reason - ++, +:, :+, :++, ++=, _+_, ::, :::, _, `, <%, %, %%, %%%, <-, =>, :>, >:, (╯°□°)╯︵ ┻━┻, ., .., ... - compiler errors
    - maattdd 6y ago
      - compilation time
      - zug_zug 6y ago
        Seriously. I modify a single line, in a single file, and the scala system takes 2-3 minutes to recompile at work. Apparently it spends over a minute inferring types! Hello! The types are EXACTLY THE SAME AS ALWAYS. Please cache the whole typing phase as a build artifact based on file/directory hashes. Maybe even make this cache source-control-safe.
marcinzm 6y ago
>ETL / data transformations. Python is a big player here - but for stable and performant data pipelines, I strongly believe Scala is the better choice. For explorative things, python has the edge though. I disagree. And I've done a lot of Scala ETL work. The issue is that in Scala ETL work your data structures are either untyped (Spark Dataframe), case classes or typed tuples. If they're untyped then there's little to gain from Scala vs. Python in terms of type safety and a lot of overhead. If they're case classes then you're moving around a lot of unnecessary data fields and you've got the overhead of making fifty intermediate case classes yourself. If it's typed tuples then it's on you to remember which field is what as nothing is named. Some sort of intelligent compile time case class subset generation would have helped the situation immensely. I think frameless (https://github.com/typelevel/frameless https://github.com/typelevel/frameless) gives you that power but it's a lot of shapeless black magic overhead and I've never seen it used in production (and probably various maximum field limitations and much slower compile times). edit: Case classes also have (or had) a steeply increasing compile time memory requirement as you add more fields which is fun when your data is 200+ fields long. Almost as fun as having to make a 200 field case class to read in a single file of which you only need 50 fields but want to be type safe. edit2: Scala is popular in ETL because the Hadoop ecosystem is JVM and Java used to be atrocious. So Spark (which wanted Hadoop compatibility) picked Scala because it was better than the alternatives. However nowadays Databricks is investing a lot more into their Python support from what I can tell and there's Python native competitors out there (Dask, Ray, etc.).
- codygman 6y ago
  > they're case classes then you're moving around a lot of unnecessary data fields and you've got the overhead of making fifty intermediate case classes yourself. Haskell developer here... What about case classes and lenses? Do they solve this?
  - marcinzm 6y ago
    As I understand it lenses don't change the underlying data structure. For ETL you need a way to basically say "the code only uses fields X, Y and Z so we will only load X, Y and Z during runtime." Automatically based on usage without having to keep updating your lens definition. Modern on disk file formats are columnar so they can very efficiently read subsets of the data. If your data has 200 columns than reading the 199 unnecessary ones can be very slow. They could help with the intermediate data structure but some of them aren't subsets or trivial derivatives. So you really need an inline way to create single use case classes. I think frameless in Scala can do some of this for standard transformations but that requires the black magic of shapeless. Spark in Python (and the untyped Dataframe API in Scala) compiles everything internally before running it to achieve the above. So it's trivial to have unit tests on empty data structures which "type check" your Spark code.

jim-jim-jim 6y ago

Well said. In some of my older comments I've shat on Scala (compared to Haskell) but after being forced to write Python for a day job, I've realized how good I had it with Scala. Luckily I'll be writing some again soon.

threeseed 6y ago

Scala is incredible for web apps. I can write my React/JS front end [1] and highly concurrent, real-time scalable backend [2] using the same language, build tools etc. [1] http://slinky.dev http://slinky.dev [2] http://zio.dev http://zio.dev

wiradikusuma 6y ago

"For explorative things, python has the edge" -- I'm a lazy programmer, how can I get contextual auto-complete like I can with Scala (and other type safe languages)? From my experience, auto-complete for Python shows everything (like JavaScript's), so I end up googling.

valenterry 6y ago
From my experience, for exploration of data, you usually use the functionality you already know and you visualize things as much as possible. So there is not a huge advantage for Scala in terms of autocompletion and such, but there is quite a bit of advantage for python for batteries-included, visualisation and speed of execution. But yeah, I get your point, I find it annoying too.
marcinzm 6y ago
For exploratory work you're in a notebook so your auto-complete usually has access to the exact object instances in question. There's also some good machine learning based autocomplete systems out there nowadays.

Fiahil 6y ago

I agree with your first point, Scala allows writing incredibly performant concurrent code. However, if -like me- you can't bear the thought of working with Scala anymore, Rust is an excellent competitor. Note, I spent around 5 years working with Scala full-time and grew a profound aversion for the language and its ecosystem. It definitively affects my judgment. I found in Rust all the "good parts" of Scala minus the things I resented.

aphexairlines 6y ago
what did you resent?
- Fiahil 6y ago
  In no particular order: - the JVM - the tooling for writing with VSCode - SBT - NullPointerExceptions bubbling up from Java libs - the split between the functional programming community and the "Scala is just Java without ;" one - everything related to implicits (creation, resolution, etc) and everyone using them - the ability to write the same 0-100 loop with 4 different keywords and 3 different styles - the inability to enforce a coding style properly and consistently with a team of more than 1 engineer - too much people writing DSLs for no particular reason - ++, +:, :+, :++, ++=, _+_, ::, :::, _, `, <%, %, %%, %%%, <-, =>, :>, >:, (╯°□°)╯︵ ┻━┻, ., .., ... - compiler errors
  - maattdd 6y ago
    - compilation time
    - zug_zug 6y ago
      Seriously. I modify a single line, in a single file, and the scala system takes 2-3 minutes to recompile at work. Apparently it spends over a minute inferring types! Hello! The types are EXACTLY THE SAME AS ALWAYS. Please cache the whole typing phase as a build artifact based on file/directory hashes. Maybe even make this cache source-control-safe.

marcinzm 6y ago

>ETL / data transformations. Python is a big player here - but for stable and performant data pipelines, I strongly believe Scala is the better choice. For explorative things, python has the edge though. I disagree. And I've done a lot of Scala ETL work. The issue is that in Scala ETL work your data structures are either untyped (Spark Dataframe), case classes or typed tuples. If they're untyped then there's little to gain from Scala vs. Python in terms of type safety and a lot of overhead. If they're case classes then you're moving around a lot of unnecessary data fields and you've got the overhead of making fifty intermediate case classes yourself. If it's typed tuples then it's on you to remember which field is what as nothing is named. Some sort of intelligent compile time case class subset generation would have helped the situation immensely. I think frameless (https://github.com/typelevel/frameless https://github.com/typelevel/frameless) gives you that power but it's a lot of shapeless black magic overhead and I've never seen it used in production (and probably various maximum field limitations and much slower compile times). edit: Case classes also have (or had) a steeply increasing compile time memory requirement as you add more fields which is fun when your data is 200+ fields long. Almost as fun as having to make a 200 field case class to read in a single file of which you only need 50 fields but want to be type safe. edit2: Scala is popular in ETL because the Hadoop ecosystem is JVM and Java used to be atrocious. So Spark (which wanted Hadoop compatibility) picked Scala because it was better than the alternatives. However nowadays Databricks is investing a lot more into their Python support from what I can tell and there's Python native competitors out there (Dask, Ray, etc.).

codygman 6y ago
> they're case classes then you're moving around a lot of unnecessary data fields and you've got the overhead of making fifty intermediate case classes yourself. Haskell developer here... What about case classes and lenses? Do they solve this?
- marcinzm 6y ago
  As I understand it lenses don't change the underlying data structure. For ETL you need a way to basically say "the code only uses fields X, Y and Z so we will only load X, Y and Z during runtime." Automatically based on usage without having to keep updating your lens definition. Modern on disk file formats are columnar so they can very efficiently read subsets of the data. If your data has 200 columns than reading the 199 unnecessary ones can be very slow. They could help with the intermediate data structure but some of them aren't subsets or trivial derivatives. So you really need an inline way to create single use case classes. I think frameless in Scala can do some of this for standard transformations but that requires the black magic of shapeless. Spark in Python (and the untyped Dataframe API in Scala) compiles everything internally before running it to achieve the above. So it's trivial to have unit tests on empty data structures which "type check" your Spark code.