8 ms·

Yammer moving from Scala to Java

187 points by Jd 15y ago

squidsoup 15y ago
There seem to be some questions around how accurate this is; can anyone from Yammer comment?
- guywithabike 15y ago
  https://twitter.com/coda/status/141641259969286145 https://twitter.com/coda/status/141641259969286145
  - RossDM 15y ago
    It appears that this was an authentic correspondence that was not meant to be seen by the public. http://news.ycombinator.com/item?id=3291980 http://news.ycombinator.com/item?id=3291980
    - kingkilr 15y ago
      How does a private email accidentally end up on the internet?
      - foobarbazetc 15y ago
        Because the guy publishing it is a dick.
        throwaway21222 15y ago
        Be that as it may, I'm quite grateful to whoever did the original writeup. He did a great job of doing it respectfully and providing a ton of good data for people considering Scala, or really any new language.
        foobarbazetc 15y ago
        Yeah. Coda's a good guy, and I'm not saying otherwise. What I'm saying is that Stephen Colebourne is being a dick, because he used some leaked email as some sort of "justification" for his poorly argued "attacks" on Scala. His whole post is a "I told you so! Look!" even though his original argument had no technical merit what so ever, and pales in comparison to the sort of constructive criticism that Coda is providing to Typesafe in the leaked email. I mean, half this email is about performance/micro-opts. These things aren't really unknowns to anyone using Scala in a production setting. FWIW, we're using Scala at a bigger scale than Yammer, and yeah, there's some things that aren't great, but on the whole, using Scala is a joy compared to straight Java. We also have no issues with sbt or using the latest Scala (we're on 2.9.1, it was a fairly easy upgrade from 2.8. Our 2.7 -> 2.8 upgrade was more a pain in the ass. :) Our only major issue is insanely long compile times. This whole thing is amazingly pointless, because no one is saying Scala is magical and without room for improvement (no one I know, anyway). Constructive criticism is great, but there's nothing here that hasn't been said on the Scala mailing lists. The real shame about this whole thing is how much respect Stephen has lost in a lot of people's eyes. Joda Time is the best library for date/time manipulation on any platform/language. I love it. But now I have this irrational compulsion to rewrite the whole thing because I want nothing to do with Stephen. Not because I disagree with him, but because I think he's gone about everything entirely the wrong way.
    - sixbrx 15y ago
      This episode illustrates the reason to keep important communications polite and well argued, you just never know who the eventual audience might be. I think the author expressed himself well enough that this shouldn't be too embarrassing.
- spaznode 15y ago
  When's the last time a development team making an internal technical decision became something a company chose to "officially announce"? "ÜberCorp announced to shareholders today that it decided once and for all curly braces belong on a new line. And that only fumbling retards would choose vi over emacs.."
- fizx 15y ago
  As a Scala developer at one of the major Scala-using companies, this feels on-the-money.
  - runT1ME 15y ago
    Can you elaborate? Do you mean that the criticism is valid (which I agree is), or that you agree with the conclusion of the detriments outweighing the benefits?
    - daniel_solano 15y ago
      While I haven't used Scala on an "enterprise" scale, I can certainly agree with many of the criticisms. However, in my experience, Scala is still worth it. I don't deal with training a team of developers, nor do I have to integrate with enterprise-scale infrastructure, so those things just don't apply to me. What are the benefits of Scala? I can think of a few, off-hand: 1. Traits: I really like traits and the way that you can use them to encapsulate cohesive and reusable functionality. In some ways, traits allow you to do a form of aspect-oriented programming. 2. Case classes are a nice, simple way of creating basic data structures. In Java, creating a data structure that should have a proper equality implementation and a nice toString is a tedious chore. Granted, I think Clojures record's are better since they give you immutability, too. 3. Closures. Other languages also have them, but Java does not. Being able to use closures can really help deal with things such as handling events. In Java, you have to resort to creating inner classes, some of which end up instantiated as members of the class. 4. Judicious use of implicits can give you some half-decent DSLs. 5. 3+4: A nice implicit conversion of a one- or two- line block into an appropriate instance of a ManagerAdapterEventListener can really simplifiy the code. 6. The mixed paradigm approach can be a blessing sometimes. I do prefer the functional approach. However, if you have integrate with a heavily object-oriented framework, being able to be object-oriented can be helpful. Overall, I would generally prefer Scala over Java for my projects. However, I do prefer Clojure over Scala.
      - bad_user 15y ago
        I can certainly agree with many of the criticisms. However, in my experience, Scala is still worth it Considering the alternatives, C++ was and is still worth it. However because of its popularity, it prevented and still prevents other languages from gaining critical mass, making it the only game in town, with the sole exception being Obj-C, but that's only because Cocoa was designed for it and Apple doesn't mind a little technology lockin either.
guard-of-terra 15y ago
They should consider Bolts as treatment: https://bitbucket.org/stepancheg/bolts/wiki/Home https://bitbucket.org/stepancheg/bolts/wiki/Home https://bitbucket.org/stepancheg/bolts/src/tip/src/main/java/ru/yandex/bolts/collection/example/ListFExample.java#cl-22 https://bitbucket.org/stepancheg/bolts/src/tip/src/main/java... (a small Java library with collections wrappers having native performance and all the FP methods; composable function classes; other niceties like tuples)
johnbender 15y ago
I've been playing around inside the closure-compiler in my free time recently and the thing that struck me most after years of working in other languages (Ruby, JavaScript, C#) is that Java is relatively simple. While it's obviously a very small sample set, I have yet to find myself spending any significant amount of time wondering at what a given snippet of code does. I don't have any Scala experience but in general simplicity has major and obvious benefits (as noted many times in the letter).
- 6ren 15y ago
  Isn't the counter-argument that Java code is harder to understand at a slightly higher level, of how classes interact? And that in more expressive languages, that complexity is pushed down to the code level, so the total complexity is the same, just in different places. Of course, there's an advantage to being able to understand a snippet locally, in isolation from the rest of the project, if that's all you need to understand (e.g. when making a local change).
  - johnbender 15y ago
    Do you mean that, because code is more expressive/terse, less project level (class, factory, etc) juggling is required? What you're saying seems to make sense but I'm trying to conjure an example to help me understand it.
    - 6ren 15y ago
      Yes. There's a family of examples with design patterns, where what is a design pattern in a less expressive language like Java requiring a standard arrangement of classes, is absorbed into the language in a more expressive language. The "design pattern" is a kind of redundant overhead. Describing examples is a bit involved, because you have to describe both the pattern and the expressive alternative. One concrete example is the visitor pattern, which is more simply written with multi-methods (a multi-method is one that is dynamically selected by the type of its arguments - though Java can have methods with the same name but different arguments, they are selected statically, based on compile-time types rather than runtime classes). I actually can't think of any other examples off the top of my head, but "design patterns are language features in more expressive languages" is a common idea, so I'm sure others can suggest other examples (or google). EDIT here we go (includes a list of examples) http://c2.com/cgi/wiki?AreDesignPatternsMissingLanguageFeatures http://c2.com/cgi/wiki?AreDesignPatternsMissingLanguageFeatu...
va_coder 15y ago
Are Clojure shops experiencing this? One of the things that makes Clojure appealing to me is the lack of types and simplicity (IMO) of the language.
- throwaway21222 15y ago
  So I'm not a big lisp guy, but from what I understand and what I know of CL, lisp is in a much better place on the expressiveness:simplicity relationship than Scala. That said, some of these issues (slow collections, talent availability, tools) have nothing to do with language complexity and might affect almost any relatively new programming language if the implementors didn't consciously work to counteract it.
  - spaznode 15y ago
    "That's just conpletely false. I don't know where your assertions are coming from but language design absolutely has a great deal to do with what kind of tools / developers you can expect to find to support the language. I'm all a-huff with the sheer gall and ignorance of your assertion that I'm just not even sure what to do. Bad developer. >:( bad" Revised: fine ok maybe a bit harsh. My opinion is that language design would be extremely influencial over how people implementing collections/tools/compilers go about doing their work and what the end result of their efforts would look like. In fact they probably are more telling of a design than anything else. That last part was sort of bs guesswork but it sounds right. Happy now?
  - Peaker 15y ago
    Lisp is in a very different domain, being dynamically/uni-typed, whereas Scala is statically typed. Statically typed languages don't aim only for simplicity/power ratio, but also for various compile-time guarantees.
- jshen 15y ago
  I wrote a medium sized project in clojure at my last job and it was a good experience. The team was small (under 5) so the hiring and training wasn't an issue for us. I never had a problem reading clojure code, and the only major headache I can remember, which was our fault, was that clojure keywords (think ruby symbols) used to be turned into interned strings which are stored in permgen and we were running out of permgen space. I believe this has been fixed in recent versions of clojure, but I'm not 100% sure about that. http://eigenjoy.com/2011/03/02/clojures-keyword-can-fill-up-your-permgen-space/ http://eigenjoy.com/2011/03/02/clojures-keyword-can-fill-up-...
frio 15y ago
The big problem I had with Scala, and the reason I stopped actively developing in it months back, was its similarity to Perl's philosophy: there are too many ways to skin the same apple. Problems can be resolved functionally, imperatively, or in some (inevitably, when you come back to work on it, hard to follow) hybrid of the two; unless you set down clear restrictions and code guidelines, one man's Scala is entirely different to the next's. I initially liked the expressivity the language afforded (and some of the supporting ecosystem - sbt, specs - I found extremely impressive), but came to abhor it over time. Unfortunately, after maintaining legacy Perl applications I've developed an aversion to anything that doesn't follow the Zen of Python: "There should be one-- and preferably only one --obvious way to do it."
- chromatic 15y ago
  unless you set down clear restrictions and code guidelines What happened when you tried?
  - frio 15y ago
    I was using Scala on some lone projects as prototypes, working with at most one other developer. In the end, we decided against it; there was never a chance to set down such guidelines (which I'm sure would be effective). The spectre of our Perl codebase was too much, sadly.
    - rjbond3rd 15y ago
      > The spectre of our Perl codebase was too much, sadly. Just curious, though -- was that Perl codebase written and factored correctly, or was it already a big mess when you got it?
- thurn 15y ago
  This is a fair complaint, but doesn't Python let you mix functional and imperative styles just as much?
  - 15y ago
swah 15y ago
In a way, just like that Yegge's post, I'm glad I could get access to this rant.
sreque 15y ago
@codahale, was this email from you or someone else in your company? Judging from your blog post a year ago at http://eng.yammer.com/blog/tag/scala http://eng.yammer.com/blog/tag/scala, at one point you thought Scala was great. As someone who still thinks Scala is great, I would like to know how long it took you to change your mind and what triggered you to reach this realization. When it comes down to it, I've never developed Scala as part of a team or worked on a project more than a few thousand lines of code, so I definitely lack the experience you probably have with the language and runtime. As to the email itself, I find the complaints about performance most uninsightful, and I say this as someone who has written CPU-intensive NLP code in Scala. Anyone who writes performant code to run on the JVM knows that the only way to get C-like performance in Java is to, it turns out, write C-like code in Java! That means operating only on primitives and arrays of primitives when in tight loops. It means avoiding allocations as much as possible. It even means avoiding generics like the plague, including java.util.HashMap. In my NLP code, for instance, I got at least a 100x speed up by switching from HashMaps with string keys to arrays by manually interning each string into an incremented number. One library I used, called Mallet, relied on GNU trove to have hash maps without boxing of primitives, because for high performance, even java.util.HashMap is too slow! This means that, if you want amazing performance in Scala, you have to write C-like code in Scala as well. I'd expect anyone who needs top-notch performance to know this kind of thing already. Writing performant code in any high-level language in our day and age, even C++, still requires you to understand how your abstractions are actually getting implemented underneath the hood. Scala doesn't work miracles here or everyone would probably be using it already. Just ask the people who run JGit about performant JVM code. (http://marc.info/?l=git&m=124111702609723&w=2 http://marc.info/?l=git&m=124111702609723&w=2) They ran into a ton of cases where they couldn't write the equivalent C-code in Java and performance suffered as a consequence. However, the points about the build toolchain, backwards binary compatibility, and to some extent, initial learning complexity are the ones that I think are most damning against Scala. I hope Typesafe has a good response to this or other companies step up to share how they worked around these deficiencies successfully. Otherwise, I feel like this could be potentially very damaging to Scala's reputation, and maybe even deservedly so.
- arete 15y ago
  Well said! I've found Scala to be a fabulous language for writing > 90% of my app(s). The overhead of Scala's built-in collections, for loops, property accessors/mutators, closures, etc is simply unimportant outside of critical hotspots and provides so much in terms of brevity and expressiveness. Yes, Scala's lack of looping constructs (break, continue, a real indexed for loop) means that you're better off writing performance-critical things like crypto or protocol parsing code in Java (or if it really matters, C called via JNI). But once you're concerned about performance at that level, every language requires you to carefully consider what your code is really generating, what your allocation patterns are, etc. Scala is a much more expressive language compared to Java, and the Scala standard library is excellent. Completely dropping Scala for Java when you're building large, complex systems seems like chopping your foot off because your small toe hurts. Use Java or C for the % of code that needs it and Scala for the rest!
jebblue 15y ago
At the end of the day, Java still rocks because it makes the most difficult aspect of programming more manageable...herding cats.
protomyth 15y ago
I look at the talent availability problem as an HR / Mangement issue. It feels like they want to do checklist hiring instead of hiring great developers who can learn. I would rather take the time needed in hiring, but at some point it becomes gears in the machine.
PaulHoule 15y ago
There's something primal about Java that invites people to write it in styles that ape other languages. In fact, whenever I write Java I find myself trying to write FORTRAN or Lisp or something else. I did a stint of work in Scala that now has me writing Scala in Java. I'm starting to feel that Java is a better Scala than Scala! It all started when I had to write Java classes that consumed results from Scala. There are all these Scala classes like Some that, if you look at the docs, you realize you can use in Java without any trouble at all. Many of the patterns that Scala uses can be implemented easily in Java... If there's something you like about Scala, you can usually do it in Java with a little thinking. A good example is all of that Actor BS in Scala. You can screw around with that and come to realize that you're completely out of control, you don't know how many threads are involved and exactly how. Java's got ExecutorService, which for many of the tasks I do, can keep an 8-CPU machine busy with 100% utilization. I remember having a discussion with a guy in Germany about his system that used Scala Actors that revealed that neither of us had any idea of what was really going on at all. Today I build pure Java systems using Java idioms that are so fast it's almost scary.
- spaznode 15y ago
  "You can screw around with that and come to realize that you're completely out of control, you don't know how many threads are involved and exactly how" I think this was the turning point for me, the day I realized someone had created this large concurrent system with absolutely no idea or care as to what was going on. (the not uncoincidentally same day qa discovered some serious problems making the app become almost useless with extremely high CPU loads)
pbreit 15y ago
"The Rest of the Story": http://codahale.com/the-rest-of-the-story/ http://codahale.com/the-rest-of-the-story/
nirav 15y ago
It is interesting to see how start-ups (twitter, forsquare and now yammer) adopting or dropping programming languages rains hailstorm on twitter, HN and blogosphere. As a serious PL designer, start-ups seem to be the prime market to target your language.
- trimbo 15y ago
  Same goes for the nouveau databases. It's going to be hard to get traction in an enterprise that can already afford and/or has purchased Oracle.
rayiner 15y ago
Interesting read. Honestly, Scala looks like gibberish to me. I strongly distrust a language where I can't hold its conceptual model in my head all at once. This is true for C, true for JS, true for ML, and hell even true for Common Lisp (once you realize it's all macros over a simple core). It is not true for Scala or Haskell, at least for me.
- Peaker 15y ago
  From http://www.paulgraham.com/avg.html http://www.paulgraham.com/avg.html: > As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.
  - rayiner 15y ago
    The power continuum isn't the same as the conceptual simplicity continuum. C++, for example, is less powerful than Common Lisp while being more complex as well. At the same time, is Haskell more powerful? Common Lisp, relative to C++, buys me the ability to run arbitrary functions at compile-time and generate code. Haskell takes away that power, and as far as I can tell doesn't add anything to replace it. Indeed, it lets your code do strictly less thanks to the type system.
    - Peaker 15y ago
      There's also the reliability/error-proneness continuum. Haskell is probably less powerful than Common Lisp (Template Haskell technically yields the power, but it is less practical). The type system may also rule out some useful programs. However, Haskell yields more reliable, less error-prone programs. The number of runtime bugs that get through the Haskell compiler is substantially lower than that of Common Lisp (I venture this is still true if you use CL's optional typing). Haskell also has forms of power that as far as I know, Common Lisp does not have. Namely, type-classes allow expressing various forms of static-type-based polymorphism that's not expressible via Common Lisp (e.g: Return-type polymorphism). In practice, having a smaller emphasis on macros pushed Haskell to abstract various repeated patterns using means other than macros. Some cases are covered by laziness. Others are covered by passing around simple functions. Some are covered by special wrapper types and combinators. My personal opinion is that the extra composability of these alternatives is more beneficial than their weaker power is detrimental. The cases where alternatives to macros cannot be found are relatively rare, so the extra cost of Haskell's macro system (TH) rarely needs to be paid.
SWengineer 15y ago
I found Neal Gafter's response to Stephen's blog rather telling: blog.joda.org/2011/11/real-life-scala-feedback-from-yammer.html?showComment=1322613312501&m=1#c6062812671177626424
Uchikoma 15y ago
One year ago I wrote on Scala: "In 15 years of Java I never had those problems. I could focus on writing production code." http://codemonkeyism.com/scala-unfit-development/ http://codemonkeyism.com/scala-unfit-development/ Promises have been made in the comments, things did either not get better or even got worse since then (Collection API, the nightmare that is SBT 0.1X, binary compatibility).

187 points by Jd 15y ago

squidsoup 15y ago

There seem to be some questions around how accurate this is; can anyone from Yammer comment?

guywithabike 15y ago
https://twitter.com/coda/status/141641259969286145 https://twitter.com/coda/status/141641259969286145
- RossDM 15y ago
  It appears that this was an authentic correspondence that was not meant to be seen by the public. http://news.ycombinator.com/item?id=3291980 http://news.ycombinator.com/item?id=3291980
  - kingkilr 15y ago
    How does a private email accidentally end up on the internet?
    - foobarbazetc 15y ago
      Because the guy publishing it is a dick.
      - throwaway21222 15y ago
        Be that as it may, I'm quite grateful to whoever did the original writeup. He did a great job of doing it respectfully and providing a ton of good data for people considering Scala, or really any new language.
        foobarbazetc 15y ago
        Yeah. Coda's a good guy, and I'm not saying otherwise. What I'm saying is that Stephen Colebourne is being a dick, because he used some leaked email as some sort of "justification" for his poorly argued "attacks" on Scala. His whole post is a "I told you so! Look!" even though his original argument had no technical merit what so ever, and pales in comparison to the sort of constructive criticism that Coda is providing to Typesafe in the leaked email. I mean, half this email is about performance/micro-opts. These things aren't really unknowns to anyone using Scala in a production setting. FWIW, we're using Scala at a bigger scale than Yammer, and yeah, there's some things that aren't great, but on the whole, using Scala is a joy compared to straight Java. We also have no issues with sbt or using the latest Scala (we're on 2.9.1, it was a fairly easy upgrade from 2.8. Our 2.7 -> 2.8 upgrade was more a pain in the ass. :) Our only major issue is insanely long compile times. This whole thing is amazingly pointless, because no one is saying Scala is magical and without room for improvement (no one I know, anyway). Constructive criticism is great, but there's nothing here that hasn't been said on the Scala mailing lists. The real shame about this whole thing is how much respect Stephen has lost in a lot of people's eyes. Joda Time is the best library for date/time manipulation on any platform/language. I love it. But now I have this irrational compulsion to rewrite the whole thing because I want nothing to do with Stephen. Not because I disagree with him, but because I think he's gone about everything entirely the wrong way.
  - sixbrx 15y ago
    This episode illustrates the reason to keep important communications polite and well argued, you just never know who the eventual audience might be. I think the author expressed himself well enough that this shouldn't be too embarrassing.
spaznode 15y ago
When's the last time a development team making an internal technical decision became something a company chose to "officially announce"? "ÜberCorp announced to shareholders today that it decided once and for all curly braces belong on a new line. And that only fumbling retards would choose vi over emacs.."
fizx 15y ago
As a Scala developer at one of the major Scala-using companies, this feels on-the-money.
- runT1ME 15y ago
  Can you elaborate? Do you mean that the criticism is valid (which I agree is), or that you agree with the conclusion of the detriments outweighing the benefits?
  - daniel_solano 15y ago
    While I haven't used Scala on an "enterprise" scale, I can certainly agree with many of the criticisms. However, in my experience, Scala is still worth it. I don't deal with training a team of developers, nor do I have to integrate with enterprise-scale infrastructure, so those things just don't apply to me. What are the benefits of Scala? I can think of a few, off-hand: 1. Traits: I really like traits and the way that you can use them to encapsulate cohesive and reusable functionality. In some ways, traits allow you to do a form of aspect-oriented programming. 2. Case classes are a nice, simple way of creating basic data structures. In Java, creating a data structure that should have a proper equality implementation and a nice toString is a tedious chore. Granted, I think Clojures record's are better since they give you immutability, too. 3. Closures. Other languages also have them, but Java does not. Being able to use closures can really help deal with things such as handling events. In Java, you have to resort to creating inner classes, some of which end up instantiated as members of the class. 4. Judicious use of implicits can give you some half-decent DSLs. 5. 3+4: A nice implicit conversion of a one- or two- line block into an appropriate instance of a ManagerAdapterEventListener can really simplifiy the code. 6. The mixed paradigm approach can be a blessing sometimes. I do prefer the functional approach. However, if you have integrate with a heavily object-oriented framework, being able to be object-oriented can be helpful. Overall, I would generally prefer Scala over Java for my projects. However, I do prefer Clojure over Scala.
    - bad_user 15y ago
      I can certainly agree with many of the criticisms. However, in my experience, Scala is still worth it Considering the alternatives, C++ was and is still worth it. However because of its popularity, it prevented and still prevents other languages from gaining critical mass, making it the only game in town, with the sole exception being Obj-C, but that's only because Cocoa was designed for it and Apple doesn't mind a little technology lockin either.

guard-of-terra 15y ago

They should consider Bolts as treatment: https://bitbucket.org/stepancheg/bolts/wiki/Home https://bitbucket.org/stepancheg/bolts/wiki/Home https://bitbucket.org/stepancheg/bolts/src/tip/src/main/java/ru/yandex/bolts/collection/example/ListFExample.java#cl-22 https://bitbucket.org/stepancheg/bolts/src/tip/src/main/java... (a small Java library with collections wrappers having native performance and all the FP methods; composable function classes; other niceties like tuples)

johnbender 15y ago

I've been playing around inside the closure-compiler in my free time recently and the thing that struck me most after years of working in other languages (Ruby, JavaScript, C#) is that Java is relatively simple. While it's obviously a very small sample set, I have yet to find myself spending any significant amount of time wondering at what a given snippet of code does. I don't have any Scala experience but in general simplicity has major and obvious benefits (as noted many times in the letter).

6ren 15y ago
Isn't the counter-argument that Java code is harder to understand at a slightly higher level, of how classes interact? And that in more expressive languages, that complexity is pushed down to the code level, so the total complexity is the same, just in different places. Of course, there's an advantage to being able to understand a snippet locally, in isolation from the rest of the project, if that's all you need to understand (e.g. when making a local change).
- johnbender 15y ago
  Do you mean that, because code is more expressive/terse, less project level (class, factory, etc) juggling is required? What you're saying seems to make sense but I'm trying to conjure an example to help me understand it.
  - 6ren 15y ago
    Yes. There's a family of examples with design patterns, where what is a design pattern in a less expressive language like Java requiring a standard arrangement of classes, is absorbed into the language in a more expressive language. The "design pattern" is a kind of redundant overhead. Describing examples is a bit involved, because you have to describe both the pattern and the expressive alternative. One concrete example is the visitor pattern, which is more simply written with multi-methods (a multi-method is one that is dynamically selected by the type of its arguments - though Java can have methods with the same name but different arguments, they are selected statically, based on compile-time types rather than runtime classes). I actually can't think of any other examples off the top of my head, but "design patterns are language features in more expressive languages" is a common idea, so I'm sure others can suggest other examples (or google). EDIT here we go (includes a list of examples) http://c2.com/cgi/wiki?AreDesignPatternsMissingLanguageFeatures http://c2.com/cgi/wiki?AreDesignPatternsMissingLanguageFeatu...

va_coder 15y ago

Are Clojure shops experiencing this? One of the things that makes Clojure appealing to me is the lack of types and simplicity (IMO) of the language.

throwaway21222 15y ago
So I'm not a big lisp guy, but from what I understand and what I know of CL, lisp is in a much better place on the expressiveness:simplicity relationship than Scala. That said, some of these issues (slow collections, talent availability, tools) have nothing to do with language complexity and might affect almost any relatively new programming language if the implementors didn't consciously work to counteract it.
- spaznode 15y ago
  "That's just conpletely false. I don't know where your assertions are coming from but language design absolutely has a great deal to do with what kind of tools / developers you can expect to find to support the language. I'm all a-huff with the sheer gall and ignorance of your assertion that I'm just not even sure what to do. Bad developer. >:( bad" Revised: fine ok maybe a bit harsh. My opinion is that language design would be extremely influencial over how people implementing collections/tools/compilers go about doing their work and what the end result of their efforts would look like. In fact they probably are more telling of a design than anything else. That last part was sort of bs guesswork but it sounds right. Happy now?
- Peaker 15y ago
  Lisp is in a very different domain, being dynamically/uni-typed, whereas Scala is statically typed. Statically typed languages don't aim only for simplicity/power ratio, but also for various compile-time guarantees.
jshen 15y ago
I wrote a medium sized project in clojure at my last job and it was a good experience. The team was small (under 5) so the hiring and training wasn't an issue for us. I never had a problem reading clojure code, and the only major headache I can remember, which was our fault, was that clojure keywords (think ruby symbols) used to be turned into interned strings which are stored in permgen and we were running out of permgen space. I believe this has been fixed in recent versions of clojure, but I'm not 100% sure about that. http://eigenjoy.com/2011/03/02/clojures-keyword-can-fill-up-your-permgen-space/ http://eigenjoy.com/2011/03/02/clojures-keyword-can-fill-up-...

frio 15y ago

The big problem I had with Scala, and the reason I stopped actively developing in it months back, was its similarity to Perl's philosophy: there are too many ways to skin the same apple. Problems can be resolved functionally, imperatively, or in some (inevitably, when you come back to work on it, hard to follow) hybrid of the two; unless you set down clear restrictions and code guidelines, one man's Scala is entirely different to the next's. I initially liked the expressivity the language afforded (and some of the supporting ecosystem - sbt, specs - I found extremely impressive), but came to abhor it over time. Unfortunately, after maintaining legacy Perl applications I've developed an aversion to anything that doesn't follow the Zen of Python: "There should be one-- and preferably only one --obvious way to do it."

chromatic 15y ago
unless you set down clear restrictions and code guidelines What happened when you tried?
- frio 15y ago
  I was using Scala on some lone projects as prototypes, working with at most one other developer. In the end, we decided against it; there was never a chance to set down such guidelines (which I'm sure would be effective). The spectre of our Perl codebase was too much, sadly.
  - rjbond3rd 15y ago
    > The spectre of our Perl codebase was too much, sadly. Just curious, though -- was that Perl codebase written and factored correctly, or was it already a big mess when you got it?
thurn 15y ago
This is a fair complaint, but doesn't Python let you mix functional and imperative styles just as much?
- 15y ago

swah 15y ago

In a way, just like that Yegge's post, I'm glad I could get access to this rant.

sreque 15y ago

@codahale, was this email from you or someone else in your company? Judging from your blog post a year ago at http://eng.yammer.com/blog/tag/scala http://eng.yammer.com/blog/tag/scala, at one point you thought Scala was great. As someone who still thinks Scala is great, I would like to know how long it took you to change your mind and what triggered you to reach this realization. When it comes down to it, I've never developed Scala as part of a team or worked on a project more than a few thousand lines of code, so I definitely lack the experience you probably have with the language and runtime. As to the email itself, I find the complaints about performance most uninsightful, and I say this as someone who has written CPU-intensive NLP code in Scala. Anyone who writes performant code to run on the JVM knows that the only way to get C-like performance in Java is to, it turns out, write C-like code in Java! That means operating only on primitives and arrays of primitives when in tight loops. It means avoiding allocations as much as possible. It even means avoiding generics like the plague, including java.util.HashMap. In my NLP code, for instance, I got at least a 100x speed up by switching from HashMaps with string keys to arrays by manually interning each string into an incremented number. One library I used, called Mallet, relied on GNU trove to have hash maps without boxing of primitives, because for high performance, even java.util.HashMap is too slow! This means that, if you want amazing performance in Scala, you have to write C-like code in Scala as well. I'd expect anyone who needs top-notch performance to know this kind of thing already. Writing performant code in any high-level language in our day and age, even C++, still requires you to understand how your abstractions are actually getting implemented underneath the hood. Scala doesn't work miracles here or everyone would probably be using it already. Just ask the people who run JGit about performant JVM code. (http://marc.info/?l=git&m=124111702609723&w=2 http://marc.info/?l=git&m=124111702609723&w=2) They ran into a ton of cases where they couldn't write the equivalent C-code in Java and performance suffered as a consequence. However, the points about the build toolchain, backwards binary compatibility, and to some extent, initial learning complexity are the ones that I think are most damning against Scala. I hope Typesafe has a good response to this or other companies step up to share how they worked around these deficiencies successfully. Otherwise, I feel like this could be potentially very damaging to Scala's reputation, and maybe even deservedly so.

arete 15y ago
Well said! I've found Scala to be a fabulous language for writing > 90% of my app(s). The overhead of Scala's built-in collections, for loops, property accessors/mutators, closures, etc is simply unimportant outside of critical hotspots and provides so much in terms of brevity and expressiveness. Yes, Scala's lack of looping constructs (break, continue, a real indexed for loop) means that you're better off writing performance-critical things like crypto or protocol parsing code in Java (or if it really matters, C called via JNI). But once you're concerned about performance at that level, every language requires you to carefully consider what your code is really generating, what your allocation patterns are, etc. Scala is a much more expressive language compared to Java, and the Scala standard library is excellent. Completely dropping Scala for Java when you're building large, complex systems seems like chopping your foot off because your small toe hurts. Use Java or C for the % of code that needs it and Scala for the rest!

jebblue 15y ago

At the end of the day, Java still rocks because it makes the most difficult aspect of programming more manageable...herding cats.

protomyth 15y ago

I look at the talent availability problem as an HR / Mangement issue. It feels like they want to do checklist hiring instead of hiring great developers who can learn. I would rather take the time needed in hiring, but at some point it becomes gears in the machine.

PaulHoule 15y ago

There's something primal about Java that invites people to write it in styles that ape other languages. In fact, whenever I write Java I find myself trying to write FORTRAN or Lisp or something else. I did a stint of work in Scala that now has me writing Scala in Java. I'm starting to feel that Java is a better Scala than Scala! It all started when I had to write Java classes that consumed results from Scala. There are all these Scala classes like Some that, if you look at the docs, you realize you can use in Java without any trouble at all. Many of the patterns that Scala uses can be implemented easily in Java... If there's something you like about Scala, you can usually do it in Java with a little thinking. A good example is all of that Actor BS in Scala. You can screw around with that and come to realize that you're completely out of control, you don't know how many threads are involved and exactly how. Java's got ExecutorService, which for many of the tasks I do, can keep an 8-CPU machine busy with 100% utilization. I remember having a discussion with a guy in Germany about his system that used Scala Actors that revealed that neither of us had any idea of what was really going on at all. Today I build pure Java systems using Java idioms that are so fast it's almost scary.

spaznode 15y ago
"You can screw around with that and come to realize that you're completely out of control, you don't know how many threads are involved and exactly how" I think this was the turning point for me, the day I realized someone had created this large concurrent system with absolutely no idea or care as to what was going on. (the not uncoincidentally same day qa discovered some serious problems making the app become almost useless with extremely high CPU loads)

pbreit 15y ago

"The Rest of the Story": http://codahale.com/the-rest-of-the-story/ http://codahale.com/the-rest-of-the-story/

nirav 15y ago

It is interesting to see how start-ups (twitter, forsquare and now yammer) adopting or dropping programming languages rains hailstorm on twitter, HN and blogosphere. As a serious PL designer, start-ups seem to be the prime market to target your language.

trimbo 15y ago
Same goes for the nouveau databases. It's going to be hard to get traction in an enterprise that can already afford and/or has purchased Oracle.

rayiner 15y ago

Interesting read. Honestly, Scala looks like gibberish to me. I strongly distrust a language where I can't hold its conceptual model in my head all at once. This is true for C, true for JS, true for ML, and hell even true for Common Lisp (once you realize it's all macros over a simple core). It is not true for Scala or Haskell, at least for me.

Peaker 15y ago
From http://www.paulgraham.com/avg.html http://www.paulgraham.com/avg.html: > As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.
- rayiner 15y ago
  The power continuum isn't the same as the conceptual simplicity continuum. C++, for example, is less powerful than Common Lisp while being more complex as well. At the same time, is Haskell more powerful? Common Lisp, relative to C++, buys me the ability to run arbitrary functions at compile-time and generate code. Haskell takes away that power, and as far as I can tell doesn't add anything to replace it. Indeed, it lets your code do strictly less thanks to the type system.
  - Peaker 15y ago
    There's also the reliability/error-proneness continuum. Haskell is probably less powerful than Common Lisp (Template Haskell technically yields the power, but it is less practical). The type system may also rule out some useful programs. However, Haskell yields more reliable, less error-prone programs. The number of runtime bugs that get through the Haskell compiler is substantially lower than that of Common Lisp (I venture this is still true if you use CL's optional typing). Haskell also has forms of power that as far as I know, Common Lisp does not have. Namely, type-classes allow expressing various forms of static-type-based polymorphism that's not expressible via Common Lisp (e.g: Return-type polymorphism). In practice, having a smaller emphasis on macros pushed Haskell to abstract various repeated patterns using means other than macros. Some cases are covered by laziness. Others are covered by passing around simple functions. Some are covered by special wrapper types and combinators. My personal opinion is that the extra composability of these alternatives is more beneficial than their weaker power is detrimental. The cases where alternatives to macros cannot be found are relatively rare, so the extra cost of Haskell's macro system (TH) rarely needs to be paid.

SWengineer 15y ago

I found Neal Gafter's response to Stephen's blog rather telling: blog.joda.org/2011/11/real-life-scala-feedback-from-yammer.html?showComment=1322613312501&m=1#c6062812671177626424

Uchikoma 15y ago

One year ago I wrote on Scala: "In 15 years of Java I never had those problems. I could focus on writing production code." http://codemonkeyism.com/scala-unfit-development/ http://codemonkeyism.com/scala-unfit-development/ Promises have been made in the comments, things did either not get better or even got worse since then (Collection API, the nightmare that is SBT 0.1X, binary compatibility).