3 ms·

I work at a major online fashion aggregator and my team uses Scala extensively for our search API (with Twitter's Finatra) and also in several of our data pipel

by amzans 7y ago

I work at a major online fashion aggregator and my team uses Scala extensively for our search API (with Twitter's Finatra) and also in several of our data pipeline systems (Akka, Spark). When I joined a few years ago, I was honestly not most excited about working with Scala, but the more I got into it, the more I realised that Scala got a lot of things just right. In particular, I really miss working with the collections and futures API whenever I switch to another language. And it's overall a pretty good package: very predictable behaviour, easy to debug (IntelliJ Idea helps a lot here), fast runtime and high quality libraries available for pretty much anything you might need.

skrebbel 7y ago
I used Scala five-ish years ago, and I loved it, but I had two qualms: 1. The community was terribly split between FP purists who wanted to write Haskell on the JVM and pragmatists who leaned more towards the "better Java" side of things. Plus the occasional ex-Rubyists who came up with method names consisting solely of interpunction ("embedded DSL!"). Has any side won? It would be nice to have anything resembling "idiomatic Scala" out there. 2. The compilation times were horrid, really blocking a fast edit-compile-run(-tests) loop, often making my attention wander away etc. Has this improved sufficiently much that you can edit code, switch to the browser (or wherever) and pretty much immediately see the result?
- aphexairlines 7y ago
  Compilation times have improved substantially: https://scala-ci.typesafe.com/grafana/dashboard/db/scala-benchmark?orgId=1 https://scala-ci.typesafe.com/grafana/dashboard/db/scala-ben...
  - cutler 7y ago
    2.13 seems to be a very marginal improvement over 2.12 so it seems to be business as usual re compilation.
    - skrebbel 7y ago
      Yes but I asked since 5 years ago, not since last release. Thanks aphexairlines, that's great to hear!
    - kelnos 7y ago
      By that graph, 2.13 is 17% faster than 2.12. I'd call that more than marginal. And as always with benchmarks, your workload may give you different results. 2.13 will especially help with code bases that heavily use the collections library.
    - pas 7y ago
      Both IDEA/IntelliJ and sbt (probably both utilizing the same compiler library/features underneath) can do incremental compilation, which is very fast. (You get autocompletion suggestions in IDEA as you type, the reload-incrementalcompile cycle using Play is now spent mostly in spinning up the app after reload not in compilation, etc.) Yes, compiling a ~200 file project from scratch takes ~5 minutes, but it has scalaz, shapeless and a ton of other libraries, but this is basically only required when you import/open the project for the first time.
    - merb 7y ago
      actually all graphs are using projects that compile in under a minute. even the scala compiler is relativly small compared to average size scala projects. the gain is way bigger in projects that have clean compile times above 1 minute.
- pas 7y ago
  I think there's a sensible middle ground. Using types for side effects (eg Slick does it for DB IO) and error handling (see also Rust's Result type) is great. Scala's for-comprehension (which is Haskell's "do" notation) is a bit awkward (you have to understand that it's a syntactic sugar, and you have to learn about monad transformers, because you have to align the types, so if you started with an Option you have to return an Option, so you have to wrap your Futures and Lists, but it's pretty automatic). There are a few things that seems harder if you are used to less pure languages, let's say you can't just manage a request context "somewhere" and rely on the dark arts of whatever Java (or Python's Flask) frameworks do. You have to pass around a context. Use implicits. For async Scala already wants you to pass around the execution context. And that's a good thing. Less black magic is always nice in the long term. Going full FP can be ... painful. When all you see is functions, functions getting passed function, that return functions, that can be called with other functions that ... Yes, sure, in the end the data is passed/transformed/filtered/mapped/reduced the same, but usually reading FP code is a lot less straightforward than it should be. (For example nice chained functions / data-pipelines are great, easy to read, etc. But going overboard with currying and function passing is not.)
  - gridlockd 7y ago
    > Going full FP can be ... painful. When all you see is functions, functions getting passed function, that return functions, that can be called with other functions that ... Exactly. "Functional purists" write code that caters to their aesthetics. It's completely unreadable and a pain in the ass to debug, but it sure looks really nice and clean. If pure functional programming really is a major benefit, it should show in the number of successful software projects. I don't see it.
- tormeh 7y ago
  I don't think any side has won, but from what I gather Odersky thinks pragmatic functional is the way to go, i.e. only mutate when it brings substantial benefits to code clarity over a pure approach. That is, the default should always be functional, but use an imperative style where it makes sense.
  - rockostrich 7y ago
    This is my engineering org's philosophy as well. New engineers who have a java background tend to have some difficulty getting used to it, but all it takes is a light switch in their head. For example, the other day there was a PR for creating a simple csv report and they wrote something like val buffer = new StringBuffer() buffer.add(header) buffer.add("\n") someList.forEach { item => val values: List[String] = someTransformation(item) buffer.add(values.mkString(",")) buffer.add("\n") } buffer.toString which is fine and readable, but it doesn't really seem like having side-effects (even though the scope was contained in the function and there weren't any worries about thread safety) aren't worth it over doing something like val rows: List[String] = someList.map { item => val values: List[String] = someTransformation(item) values.mkString(",") } (header :: rows).mkString("\n") + "\n" // I forgot whether they appended the string with a line break or not...
    - Matthias247 7y ago
      The above version might be more efficient (less allocations, less GC), which certainly matters for some applications. I actually also find it more readable, but that’s a personal preference
    - gridlockd 7y ago
      I like that version A 1) is more readable (fight me) 2) doesn't rely on internal optimizations for string concatenation, in order to be as efficient 3) doesn't create extra garbage (linked lists) In fact, the only thing I would change is make it a for loop instead.
    - rbonvall 7y ago
      You can have the line break appended at the end by using the three-arg version of mkString: (header :: rows).mkString("", "\n", "\n") My two cents: I prefer the second solution. Maybe it's because I have never programmed in Java.
- mruts 7y ago
  I would disagree that there isn’t idiomatic scala. Probably the most dominant style of programming scala is flatish class hierarchies with pure functions inherited and overrided as necessary. In idiomatic example of this style is the ByteString library that is used by akka.

by amzans 7y ago

skrebbel 7y ago

I used Scala five-ish years ago, and I loved it, but I had two qualms: 1. The community was terribly split between FP purists who wanted to write Haskell on the JVM and pragmatists who leaned more towards the "better Java" side of things. Plus the occasional ex-Rubyists who came up with method names consisting solely of interpunction ("embedded DSL!"). Has any side won? It would be nice to have anything resembling "idiomatic Scala" out there. 2. The compilation times were horrid, really blocking a fast edit-compile-run(-tests) loop, often making my attention wander away etc. Has this improved sufficiently much that you can edit code, switch to the browser (or wherever) and pretty much immediately see the result?

aphexairlines 7y ago
Compilation times have improved substantially: https://scala-ci.typesafe.com/grafana/dashboard/db/scala-benchmark?orgId=1 https://scala-ci.typesafe.com/grafana/dashboard/db/scala-ben...
- cutler 7y ago
  2.13 seems to be a very marginal improvement over 2.12 so it seems to be business as usual re compilation.
  - skrebbel 7y ago
    Yes but I asked since 5 years ago, not since last release. Thanks aphexairlines, that's great to hear!
  - kelnos 7y ago
    By that graph, 2.13 is 17% faster than 2.12. I'd call that more than marginal. And as always with benchmarks, your workload may give you different results. 2.13 will especially help with code bases that heavily use the collections library.
  - pas 7y ago
    Both IDEA/IntelliJ and sbt (probably both utilizing the same compiler library/features underneath) can do incremental compilation, which is very fast. (You get autocompletion suggestions in IDEA as you type, the reload-incrementalcompile cycle using Play is now spent mostly in spinning up the app after reload not in compilation, etc.) Yes, compiling a ~200 file project from scratch takes ~5 minutes, but it has scalaz, shapeless and a ton of other libraries, but this is basically only required when you import/open the project for the first time.
  - merb 7y ago
    actually all graphs are using projects that compile in under a minute. even the scala compiler is relativly small compared to average size scala projects. the gain is way bigger in projects that have clean compile times above 1 minute.
pas 7y ago
I think there's a sensible middle ground. Using types for side effects (eg Slick does it for DB IO) and error handling (see also Rust's Result type) is great. Scala's for-comprehension (which is Haskell's "do" notation) is a bit awkward (you have to understand that it's a syntactic sugar, and you have to learn about monad transformers, because you have to align the types, so if you started with an Option you have to return an Option, so you have to wrap your Futures and Lists, but it's pretty automatic). There are a few things that seems harder if you are used to less pure languages, let's say you can't just manage a request context "somewhere" and rely on the dark arts of whatever Java (or Python's Flask) frameworks do. You have to pass around a context. Use implicits. For async Scala already wants you to pass around the execution context. And that's a good thing. Less black magic is always nice in the long term. Going full FP can be ... painful. When all you see is functions, functions getting passed function, that return functions, that can be called with other functions that ... Yes, sure, in the end the data is passed/transformed/filtered/mapped/reduced the same, but usually reading FP code is a lot less straightforward than it should be. (For example nice chained functions / data-pipelines are great, easy to read, etc. But going overboard with currying and function passing is not.)
- gridlockd 7y ago
  > Going full FP can be ... painful. When all you see is functions, functions getting passed function, that return functions, that can be called with other functions that ... Exactly. "Functional purists" write code that caters to their aesthetics. It's completely unreadable and a pain in the ass to debug, but it sure looks really nice and clean. If pure functional programming really is a major benefit, it should show in the number of successful software projects. I don't see it.
tormeh 7y ago
I don't think any side has won, but from what I gather Odersky thinks pragmatic functional is the way to go, i.e. only mutate when it brings substantial benefits to code clarity over a pure approach. That is, the default should always be functional, but use an imperative style where it makes sense.
- rockostrich 7y ago
  This is my engineering org's philosophy as well. New engineers who have a java background tend to have some difficulty getting used to it, but all it takes is a light switch in their head. For example, the other day there was a PR for creating a simple csv report and they wrote something like val buffer = new StringBuffer() buffer.add(header) buffer.add("\n") someList.forEach { item => val values: List[String] = someTransformation(item) buffer.add(values.mkString(",")) buffer.add("\n") } buffer.toString which is fine and readable, but it doesn't really seem like having side-effects (even though the scope was contained in the function and there weren't any worries about thread safety) aren't worth it over doing something like val rows: List[String] = someList.map { item => val values: List[String] = someTransformation(item) values.mkString(",") } (header :: rows).mkString("\n") + "\n" // I forgot whether they appended the string with a line break or not...
  - Matthias247 7y ago
    The above version might be more efficient (less allocations, less GC), which certainly matters for some applications. I actually also find it more readable, but that’s a personal preference
  - gridlockd 7y ago
    I like that version A 1) is more readable (fight me) 2) doesn't rely on internal optimizations for string concatenation, in order to be as efficient 3) doesn't create extra garbage (linked lists) In fact, the only thing I would change is make it a for loop instead.
  - rbonvall 7y ago
    You can have the line break appended at the end by using the three-arg version of mkString: (header :: rows).mkString("", "\n", "\n") My two cents: I prefer the second solution. Maybe it's because I have never programmed in Java.
mruts 7y ago
I would disagree that there isn’t idiomatic scala. Probably the most dominant style of programming scala is flatish class hierarchies with pure functions inherited and overrided as necessary. In idiomatic example of this style is the ByteString library that is used by akka.