15 ms·

True Scala complexity

182 points by plinkplonk 15y ago

hp 15y ago
One source of Scala's design is Java interoperability, much as C++ has to live with its C legacy. This compromise may also be why people are able to use Scala in practice, though; they need to talk to Java libraries or need the performance of a language that maps straightforwardly to the JVM. Scala is a functional/object-oriented hybrid, making it more complex than a purist language in either mold. It always lets you do things in a Java-like way - you can use it as "Java with less boilerplate" and touch almost nothing Scala-specific. Then it adds functional programming alongside. To me this feels very natural; I like objects for the big-picture structure, but I like to write algorithms and manipulate data in functional style. If you need raw performance in some hotspot, write a Java-like while loop with mutable state; otherwise, write something nice and high-level (and the JVM will still be much faster than a "scripting language" however you define it, e.g. http://blog.j15r.com/2011/12/for-those-unfamiliar-with-it-box2d-is.html http://blog.j15r.com/2011/12/for-those-unfamiliar-with-it-bo...). Scala does fix some Java warts that are legitimately complex or confusing in their own right. For example, primitive and boxed types are less strongly separated; there's no "static", just nice syntax for singleton objects; collections are 100x nicer with far less noise; a nice multiple inheritance design; covariance eliminates a bunch of nasty hacks; better ways to specify access controls; there's a decent way to factor out exception handling; case matching is _awesome_; and _so_ much less boilerplate in general. I'm not sure the static vs. dynamic religious war can ever be resolved, but static types feel less broken and less verbose in Scala than in Java. Java makes you lie to the type system, or do something unnatural, much too often. Scala hasn't cured every such situation, but it's cured a lot of the most common ones, and greatly reduced the need for manual type annotations. Scala gives you the conciseness of Ruby, but with static type checking, higher runtime performance, and interoperability with existing Java code. Some tradeoffs of static type checking remain, such as compilation times. People do go on wild goose chases trying to push the language farther than it's ready to go. I've done it myself. I agree with the article that there are lots of areas to improve and appreciate the constructive write-up. But on the other hand, the perfect shouldn't be the enemy of the good. I certainly would not choose to go back to Java, even as I'd love to keep seeing Scala get even better.
j_baker 15y ago
I have a confession to make. I voted this article up simply based on the title. Even if it turned out to be incredible link bait (which it didn't), I'm just happy that someone out there is at least recognizing the desire to have a reasonable discussion of this issue.
jashkenas 15y ago
Fantastic post. The most salient excerpt for me: def filterMap[B,D](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D]): D def filterMap[B,D <: GenTraversableOnce[B]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D]): D def filterMap[B,D <% GenTraversableOnce[B]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D]): D def filterMap[B,D[B]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D[B]]): D[B] def filterMap[B,D[B] <: GenTraversableOnce[B]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D[B]]): D[B] def filterMap[B,D[B] <% GenTraversableOnce[B]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D[B]]): D[B] def filterMap[B,D[_]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D[B]]): D[B] def filterMap[B,D[_]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D[B]], ev: D[B] <:< GenTraversableOnce[B]): D[B] def filterMap[B,D[_]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D[B]], ev: D[B] => GenTraversableOnce[B]): D[B] ... > The answer to our original question? It turns out none > of these are correct. In fact, *it is impossible to insert > a new method that behaves like a normal collection method.* > This, despite the heavy advertising of enrich my library. Stuff like this makes think about how, despite all of the problems with using it in libraries, it's lovely that many dynamic languages can be extended in your application with little fuss. # Ruby. class Array def filter_map ... end end // JavaScript. Array.prototype.filterMap = function() { ... };
- lisper 15y ago
  Common Lisp: (defmethod filter-map (f (a array)) ...)
  - soc88 15y ago
    That was not the question. It is trivial to do that in Scala for the use case mentioned. The problem is adding it as an "instance" method to a collection and expecting that it is usable by something not being a collection at all.
    - lisper 15y ago
      I have no idea what you mean by "adding it (which "it"?) as an "instance" method to a collection". You can do this: (defmethod filter-map (f (c (eql some-particular-collection))) ...) but I suspect that's not what you meant. I also can't make heads or tails out of "expecting that it (which "it"?) is usable by something not being a collection at all." I'm not even sure that's proper English, let alone semantically meaningful.
adriaanm 15y ago
I think it would be great if we could replace implicits by: 1) a simple mechanism for "enrichment" (aka. retro-active extension, virtual classes) 2) functional type-level computation (as opposed to the mini-prolog engine that is implicit search + type inference). Reducing complexity is complicated, unfortunately. We [have been|are] thinking about both of these alternative features, though. --- ps: The following part of the article is inaccurate: "Turns out that Scala will search up to one level up the type hierarchy for a matching shape for the implicit." Check out the "implicits without the import tax" part of http://eed3si9n.com/implicit-parameter-precedence-again http://eed3si9n.com/implicit-parameter-precedence-again. The implicit scope includes all the superclasses (and their companion objects) of all the parts of the type of the implicit value that's being resolved.
dkhenry 15y ago
After reading a bunch of these articles and really walking away unimpressed this one is fantastic. It does a great job of showing how cryptic you can make Scala code. It also shows that the same topic on which half the article is spent explaining can be boiled down to four lines of less "Scala Like" code. As a pragmatic user of Scala I don't see why the fact that you _can_ do some amazing but cryptic things with the language is a detriment. Its like pointers in C or templates and multiple inheritance in C++. Some people say that the developer shouldn't be given such capabilities since it can lead to cryptic and possible dangerous code. I am of the opinion that the additional flexibility the features provides is nothing but a benefit and that its up to the developer/organization to make sane limits on what can be used.
- nickik 15y ago
  Some languages show that you can do these things in a simple way. For example I impmentet a Persisten Vector for Dylan (very common lisp like language) in about 3 sessions. It was the first time I wrote dylan code befor that I only read on dylan. Its working and fiels like a "native" collection allready. I guess it would take me the same amount of time fully understand this article. Other people in this thread have shown how easy it is in other languages too.
  - soc88 15y ago
    I still have not found a single comment showing "how easy it is in other languages". Do you have a link?
    - a_strange_guy 15y ago
      I'm not Parent, but I think that this is his implementation: https://github.com/nickik/Persistent-Vector-in-Dylan/blob/master/PersistentVector.dylan#L222 https://github.com/nickik/Persistent-Vector-in-Dylan/blob/ma... The last part (line 222+) is what defines it as a sequence in Dylan. Dylan (and other multi-dispatch languages) don't have the problem of "adding methods to objects/classes", because methods are standalone entities (first-class, whereas in Scala the are not) that exist independently of the data they operate on. so you can just "add" a method to something by defining it: define method upcase(s :: String) => (ret :: String) // code here end method upcase; "abc".upcase // => "ABC" // is just sugar for upcase("abc") // => "ABC" so you could define 'forward-iteration-protocol (a method that returns 8 values) on anything in Dylan (built-in or not) to make it a "Sequence".
harryh 15y ago
A lot of the items the post author describes as complex you can't do at all in other languages. "Simple things should be simple, complex things should be possible." - Alan Kay I don't think the blog author gives particularly great examples of simple things that are made complex by the language. He simply gives examples of things that are inherently complex that scala at least makes possible.
- xyzzyrz 15y ago
  "Notice how several of the problems I pinpoint above are of the form, “How do I accomplish this expression that isn’t even possible in many other languages?” And these are mostly static build issues, which are far from the worst fates imaginable. Certainly, if you don’t care at all about clumsy code or resorting to escape hatches or “writing Java in Scala,” it’s frequently possible to bang out mostly-type-safe Scala while side-stepping battles against the compiler straitjacket. Plus, let’s not lose sight of all the things Scala brings to the table, none of which I’ve described. If there’s one thing more time-consuming than illustrating the complexities of the Scala language, it’s illustrating all the advantages, and I’ve already spent more time on this post than I’d care to admit."
  - harryh 15y ago
    Yes, the post author says much the same thing as I just did. I just thought it was worth specifically mentioning in the discussion thread (in case some people didn't make it all the way through the blog post).
- akeefer 15y ago
  I have to respectfully disagree. First of all, the things he shows aren't inherently complex: adding an additional function to the existing collections library is something that's possible in other languages in less confusing ways (see other examples in this post). The various concepts required to solve the problem in Scala like implicits and higher kinds might be inherently complex, but the problem he's trying to solve isn't inherently complex. And while the Scala solution might not be 100% identical to, say, just adding the method directly to a class in Ruby (since in Scala it's type-safe, etc. etc.) that's still missing the point. The point is that if you sit down and say "I want to add a method to all my collections that works like the existing methods," in Scala it requires an understanding of a huge number of complicated intersecting concepts, whereas in other languages it doesn't. Secondly, excusing complexity by saying "it makes things possible that wouldn't otherwise be possible" isn't really enough of a justification. The question is: do those exact things need to be possible? Or is there some way that gets me 90% of the way there without the complexity, and that's good enough? More power isn't always better, which was a big part of the point of the article. Just dismissing it by saying "well, the complexity lets you do powerful things" doesn't really refute the point of the article, it totally misses it. There's a tradeoff to be made. You might make those tradeoffs differently than I would, which is totally understandable, but we should at least be able to have a conversation about the fact that there is such a tradeoff without people dimissing statements like "Scala is complex" out-of-hand.
jberryman 15y ago
Site is unresponsive for me: http://webcache.googleusercontent.com/search?q=cache:http://yz.mit.edu/wp/true-scala-complexity/&hl=en&strip=1 http://webcache.googleusercontent.com/search?q=cache:http://...
oacgnol 15y ago
I wonder how much of an impact it would be if Scala simply had better documentation. I love the terseness of the code and the complex things you can do, but I'll be damned if I could ever glean ideas from looking at method definitions rather than code examples.
- hp 15y ago
  I found it essential to read Martin Odersky's Programming in Scala book - if you haven't then I recommend it. It makes many things clear that can be hard to pick up using only the free online resources. There's also a new docs site and it's getting better all the time: http://docs.scala-lang.org/ http://docs.scala-lang.org/
  - gtani 15y ago
    There's a lot of good books on the language, this is 1st edition of staircase book, which covers 2.7, so probably the majority was written for 2.8 http://www.artima.com/pins1ed/ http://www.artima.com/pins1ed/ There's also a book manuscript (downloadable from typesafe.com, and Manning.com has 3 books in the pipeline, tho the 3rd, on FP by Tony Morris doesn't show there.
deleted 15y ago
[deleted]
reinhardt 15y ago
Just wow. My eyes started glazing over at "views don’t chain. Instead, use view bounds" and he completely lost me on the next slide with generics. I had Scala in my mental TODO/Maybe list but after skimming over this post I think I'll pass.
- Confusion 15y ago
  Yeah, the post went steeply uphill there. But you can actually understand the gist of most of what he's saying without actually understanding what it means that "views don’t chain. Instead, use view bounds". And somewhat further on, it goes downhill again. In the simplest form, you can use Scala as a better Java. It's worth exploring it just for that comparison. And then there's much, much more you can do, if you want to, but it's also a very nice language if you don't try to add methods to the collections library and don't use dependent types.
- SomeOtherGuy 15y ago
  So, a guy tries to solve a hard problem by using very powerful but very complex features that 99.9999999% of scala devs would never touch, and you take that as a reason to avoid scala? And then you come here to tell everyone how absurd your decision making process is?
- hythloday 15y ago
  I'd have the opposite reaction, I think: I'd want to learn a language because it introduced new concepts (and thus, terminology). It seems to me that it's a bit pointless to learn a language if you already intimately understand its theoretical underpinnings.
dxbydt 15y ago
Regarding some of the "brainteasers" posted in the article, I am assuming that most of them are simply bugs that will get ironed out in version 2.9.2 or thereabouts ? eg. The following fails: Set(1,2,3).toIndexedSeq sortBy (-_) But doing the same in 2 steps, ie. after assignment, works val xs = Set(1,2,3).toIndexedSeq; xs sortBy (-_) I have been bitten by this several times now, so I don't unnecessarily chain > 2 functions even if it does compile ( which also solves one other brainteaser posed in that article ) Have also seen the problem with the add function, when I wrote a matrix manipulation library. def add(x: Int, y: Int) = x + y add(1,_) fails, but add(_,_) works. Even though the error message " missing parameter type for expanded function" seems reasonable, and providing the parameter type ie. add(1,_:Int), does compile, the bahavior is hard to explain to newbies. His point about hanging your hat on asInstanceOf[...] and the resulting code being clumsy...ok, guilty as charged, but then you only have so many hours in a day, and mgmt is paying $$ to solve boring business problems ( compute the asset quality of five million loans in thirty lines of business over twelve quarters using scala ) and not mucking around ( add a filterMap to the scala collection library to get an idiomatic Scala implementation ) I enjoyed the programming snippets in the article very much, but I still don't get what the point of the rant was. He goes on and on about "lack of acknowledgement of complexity", but what does that really mean ? Does he want like a gold star ? Even if everybody acknowledges that scala is complex, what then ? Eventually some of it will get fixed & the rest will not & life will go on. Why be a downer at such an early phase of growth of the language ? All this talk of excess complexity will simply scare off the early adopters. Java has been around for 15 years and we still don't have decent generics. Scala is so far ahead in such a short time. Patience, etc.
- eropple 15y ago
  > Even if everybody acknowledges that scala is complex, what then ? Eventually some of it will get fixed & the rest will not & life will go on. Why be a downer at such an early phase of growth of the language ? Because things don't get fixed if people won't talk about them. Irrational defenses (like calling someone who points out legitimate problems a "downer") are a hindrance toward making Scala better than it is right now.
modersky 15y ago
First, I think a blog post ending with "bring on the flames" should have comments enabled, otherwise it's not really fair. It comes across as whining without wanting to listen to advice or a response. Second, I think the blog post is useful because it shows what's wrong with some (fortunately very small) part of the Scala ecosystem, and because it points to a way to fix it. Here's a quick recap: The author tries to add arbitrary operations to Scala's Seq abstraction without changing its source code and wants them to work also on arrays (which are plain old Java arrays), without any extra work. Arrays in Java support: length, index, and update; that's it. There is as far as I know no language in existence that allows the precise thing the author wants to achieve. And there are many variations, such as adding only to Seq or only to Array that would be really easy in Scala but still impossible in most other mainstream languages. The author then throws all the machinery he can think of at the problem to still achieve the same non-result. Well, tough luck. He might have hit a thing that's simply impossible to do in a generic way, given the tools we currently have. In fact, I have not checked whether there would be a way to achieve the result that he wants because that's beside the point. There are always limits to a generic formulation that will force you at some point to treat things on a case by case basis. The problem is that, in trying to achieve his impossible goal, the author (mis-)uses a lot of the most powerful features of Scala, and concludes that Scala is simply too complex for helping him achieve the result. I believe he wrote this blog post to prompt the maintainers of Scala to add even more power to the language so that he can achieve his goal (the only other motivation I can think of is that he's trying to actively damage the ecosystem he writes code in, but that would make no sense to me). My response will probably not please him. I think that we need to take away sharp knifes from people who have a tendency to cut themselves. I was always proud that in Scala you could do in a library where in other languages you had to change the compiler. Inevitably, some of this is cutting edge stuff. We have tried many times to clarify the boundaries, for instance when I defined the Scala levels http://www.scala-lang.org/node/8610 http://www.scala-lang.org/node/8610 But we can't prevent a developer who prides himself to "stroll right through level L3" to get hurt. So, I believe here is what we need do: Truly advanced, and dangerously powerful, features such as implicit conversions and higher-kinded types will in the future be enabled only under a special compiler flag. The flag will come with documentation that if you enable it, you take the responsibility. Even with the flag disabled, Scala will be a more powerful language than any of the alternatives I can think of. And enabling the flag to do advanced stuff is in any case much easier than hacking your own compiler. I would be interested to read your comments on this proposal.
jonshea 15y ago
If anyone is able, would you please explain to me these two questions from the quiz? I’m stumped. Why does `toSeq` compile, but not `toIndexedSeq`? Set(1,2,3).toIndexedSeq sortBy (-_) Set(1,2,3).toSeq sortBy (-_) Why does `h` compile, but `f` does not? def add(x: Int, y: Int) = x + y val f = add(1,_) val h = add(_,_)
- extempore 15y ago
  toIndexedSeq takes a type parameter (I'm not sure why - it appears unncessary) which means the result of toIndexedSeq is not known when it is attempting to infer the Ordering. toSeq doesn't take one, it's known to be Seq[Int]. The other looks like some quirk of partial application. It's unlikely there's any fundamental reason, only an implementation imperfection.
  - jonshea 15y ago
    Good to know. Thanks Paul!
OlegYch 15y ago
Very good post. One point with which i cannot agree is "In fact, it is impossible to insert a new method that behaves like a normal collection method. " Please see http://ideone.com/ePUHG http://ideone.com/ePUHG An excerpt: import MyEnhancements._ println("qwe".quickSort) println(Array(2,0).quickSort) println(Seq(2,0).quickSort)
- etorreborre 15y ago
  Miles Sabin's solution is also worth a look: https://gist.github.com/f83892f65f63b14a1f75 https://gist.github.com/f83892f65f63b14a1f75 It uses dependent types which will be included by default in Scala 2.10. I don't consider this as "simple" but my point of view is that Computer Science is not "simple" :-). And having a language supporting that level of genericity is really helpful for type-safety and code reuse.
  - milessabin 15y ago
    @OlegYch's solution also represents a good trade off of complexity vs. convenience: his solution is simpler but doesn't get along so nicely with type inference; mine gets on just fine with type inference, but is more complex and depends (no pun intended) on dependent method types. As ever YMMV.
pron 15y ago
I think that the source of the problem is that Scala is simply not an opinionated language. You can go functional, you can go OO; you can go immutable, you can go with locks etc. And because it tries to do everything it falls on its face, but worse than that - its users don't really know HOW you're supposed to use it. I always like to contrast Scala with Clojure and Erlang. Both are very opinionated: they have a philosophy of tackling problems, not just a set of tools, and that's why they are so elegant and beautiful (well, at least Clojure is. Erlang is showing its age at times). Even languages more similar to Scala like Groovy or Kotlin are more opinionated and more focused (and thus more elegant and simpler) than Scala. They are trying to be a more modern Java, a blue-collar OO language with some modern fun. But it seems like Scala is trying to push not only tools but concepts, only it hasn't decided which concepts are best so it's pushing all of them at the same time. The result is not only frustration but harmful education as to the best way go forward.
- soc88 15y ago
  Both "more opinionated" languages are considerably slower than Scala, so sometimes it is necessary to fall back to "ugly, imperative" code and those languages make sure you will hate that experience. The difference imho is that Scala doesn't punish you for trying to be fast where necessary. Apart from that I would really like to where Groovy or Kotlin are "more elegant or simpler". I would have probably looked into the specification but something like that doesn't even exist for Groovy. From my last journey into Groovy I learned that this language is substantially underdocumented and buggy as hell. I prefer not touching it anymore.
  - pron 15y ago
    Actually falling back to ugly imperative and super-fast code in Clojure is extremely easy, and its integration with Java is a pleasure. But I don't want to pit a specific language against Scala. My point is that it isn't clear (to anyone possibly), what Scala IS, other than a kitchen-sink language for programming concepts. Some things in Scala are easy and some things are hard, but it's not always clear why. I'd expect the "good" things (whatever is considered good by the language designers, their expert opinion, that is) to be easy and the "bad" things hard. But in Scala it seems that whether something is easy or not depends on how well it fits with the language's algebraic model, and not how well it fits with recommended practice. Immutable is easy (that must be good, no?), but mutable is just as easy (so, wait, I'm supposed to... what exactly?); functional is easy, but so is imperative; implicits are easy but extension methods, as the blog post shows, can be really hard; type inference is easy except when it's not. See the problem?

182 points by plinkplonk 15y ago

hp 15y ago

One source of Scala's design is Java interoperability, much as C++ has to live with its C legacy. This compromise may also be why people are able to use Scala in practice, though; they need to talk to Java libraries or need the performance of a language that maps straightforwardly to the JVM. Scala is a functional/object-oriented hybrid, making it more complex than a purist language in either mold. It always lets you do things in a Java-like way - you can use it as "Java with less boilerplate" and touch almost nothing Scala-specific. Then it adds functional programming alongside. To me this feels very natural; I like objects for the big-picture structure, but I like to write algorithms and manipulate data in functional style. If you need raw performance in some hotspot, write a Java-like while loop with mutable state; otherwise, write something nice and high-level (and the JVM will still be much faster than a "scripting language" however you define it, e.g. http://blog.j15r.com/2011/12/for-those-unfamiliar-with-it-box2d-is.html http://blog.j15r.com/2011/12/for-those-unfamiliar-with-it-bo...). Scala does fix some Java warts that are legitimately complex or confusing in their own right. For example, primitive and boxed types are less strongly separated; there's no "static", just nice syntax for singleton objects; collections are 100x nicer with far less noise; a nice multiple inheritance design; covariance eliminates a bunch of nasty hacks; better ways to specify access controls; there's a decent way to factor out exception handling; case matching is _awesome_; and _so_ much less boilerplate in general. I'm not sure the static vs. dynamic religious war can ever be resolved, but static types feel less broken and less verbose in Scala than in Java. Java makes you lie to the type system, or do something unnatural, much too often. Scala hasn't cured every such situation, but it's cured a lot of the most common ones, and greatly reduced the need for manual type annotations. Scala gives you the conciseness of Ruby, but with static type checking, higher runtime performance, and interoperability with existing Java code. Some tradeoffs of static type checking remain, such as compilation times. People do go on wild goose chases trying to push the language farther than it's ready to go. I've done it myself. I agree with the article that there are lots of areas to improve and appreciate the constructive write-up. But on the other hand, the perfect shouldn't be the enemy of the good. I certainly would not choose to go back to Java, even as I'd love to keep seeing Scala get even better.

j_baker 15y ago

I have a confession to make. I voted this article up simply based on the title. Even if it turned out to be incredible link bait (which it didn't), I'm just happy that someone out there is at least recognizing the desire to have a reasonable discussion of this issue.

jashkenas 15y ago

Fantastic post. The most salient excerpt for me: def filterMap[B,D](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D]): D def filterMap[B,D <: GenTraversableOnce[B]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D]): D def filterMap[B,D <% GenTraversableOnce[B]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D]): D def filterMap[B,D[B]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D[B]]): D[B] def filterMap[B,D[B] <: GenTraversableOnce[B]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D[B]]): D[B] def filterMap[B,D[B] <% GenTraversableOnce[B]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D[B]]): D[B] def filterMap[B,D[_]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D[B]]): D[B] def filterMap[B,D[_]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D[B]], ev: D[B] <:< GenTraversableOnce[B]): D[B] def filterMap[B,D[_]](f: A => Option[B])(implicit b: CanBuildFrom[?,B,D[B]], ev: D[B] => GenTraversableOnce[B]): D[B] ... > The answer to our original question? It turns out none > of these are correct. In fact, *it is impossible to insert > a new method that behaves like a normal collection method.* > This, despite the heavy advertising of enrich my library. Stuff like this makes think about how, despite all of the problems with using it in libraries, it's lovely that many dynamic languages can be extended in your application with little fuss. # Ruby. class Array def filter_map ... end end // JavaScript. Array.prototype.filterMap = function() { ... };

lisper 15y ago
Common Lisp: (defmethod filter-map (f (a array)) ...)
- soc88 15y ago
  That was not the question. It is trivial to do that in Scala for the use case mentioned. The problem is adding it as an "instance" method to a collection and expecting that it is usable by something not being a collection at all.
  - lisper 15y ago
    I have no idea what you mean by "adding it (which "it"?) as an "instance" method to a collection". You can do this: (defmethod filter-map (f (c (eql some-particular-collection))) ...) but I suspect that's not what you meant. I also can't make heads or tails out of "expecting that it (which "it"?) is usable by something not being a collection at all." I'm not even sure that's proper English, let alone semantically meaningful.

adriaanm 15y ago

I think it would be great if we could replace implicits by: 1) a simple mechanism for "enrichment" (aka. retro-active extension, virtual classes) 2) functional type-level computation (as opposed to the mini-prolog engine that is implicit search + type inference). Reducing complexity is complicated, unfortunately. We [have been|are] thinking about both of these alternative features, though. --- ps: The following part of the article is inaccurate: "Turns out that Scala will search up to one level up the type hierarchy for a matching shape for the implicit." Check out the "implicits without the import tax" part of http://eed3si9n.com/implicit-parameter-precedence-again http://eed3si9n.com/implicit-parameter-precedence-again. The implicit scope includes all the superclasses (and their companion objects) of all the parts of the type of the implicit value that's being resolved.

dkhenry 15y ago

After reading a bunch of these articles and really walking away unimpressed this one is fantastic. It does a great job of showing how cryptic you can make Scala code. It also shows that the same topic on which half the article is spent explaining can be boiled down to four lines of less "Scala Like" code. As a pragmatic user of Scala I don't see why the fact that you _can_ do some amazing but cryptic things with the language is a detriment. Its like pointers in C or templates and multiple inheritance in C++. Some people say that the developer shouldn't be given such capabilities since it can lead to cryptic and possible dangerous code. I am of the opinion that the additional flexibility the features provides is nothing but a benefit and that its up to the developer/organization to make sane limits on what can be used.

nickik 15y ago
Some languages show that you can do these things in a simple way. For example I impmentet a Persisten Vector for Dylan (very common lisp like language) in about 3 sessions. It was the first time I wrote dylan code befor that I only read on dylan. Its working and fiels like a "native" collection allready. I guess it would take me the same amount of time fully understand this article. Other people in this thread have shown how easy it is in other languages too.
- soc88 15y ago
  I still have not found a single comment showing "how easy it is in other languages". Do you have a link?
  - a_strange_guy 15y ago
    I'm not Parent, but I think that this is his implementation: https://github.com/nickik/Persistent-Vector-in-Dylan/blob/master/PersistentVector.dylan#L222 https://github.com/nickik/Persistent-Vector-in-Dylan/blob/ma... The last part (line 222+) is what defines it as a sequence in Dylan. Dylan (and other multi-dispatch languages) don't have the problem of "adding methods to objects/classes", because methods are standalone entities (first-class, whereas in Scala the are not) that exist independently of the data they operate on. so you can just "add" a method to something by defining it: define method upcase(s :: String) => (ret :: String) // code here end method upcase; "abc".upcase // => "ABC" // is just sugar for upcase("abc") // => "ABC" so you could define 'forward-iteration-protocol (a method that returns 8 values) on anything in Dylan (built-in or not) to make it a "Sequence".

harryh 15y ago

A lot of the items the post author describes as complex you can't do at all in other languages. "Simple things should be simple, complex things should be possible." - Alan Kay I don't think the blog author gives particularly great examples of simple things that are made complex by the language. He simply gives examples of things that are inherently complex that scala at least makes possible.

xyzzyrz 15y ago
"Notice how several of the problems I pinpoint above are of the form, “How do I accomplish this expression that isn’t even possible in many other languages?” And these are mostly static build issues, which are far from the worst fates imaginable. Certainly, if you don’t care at all about clumsy code or resorting to escape hatches or “writing Java in Scala,” it’s frequently possible to bang out mostly-type-safe Scala while side-stepping battles against the compiler straitjacket. Plus, let’s not lose sight of all the things Scala brings to the table, none of which I’ve described. If there’s one thing more time-consuming than illustrating the complexities of the Scala language, it’s illustrating all the advantages, and I’ve already spent more time on this post than I’d care to admit."
- harryh 15y ago
  Yes, the post author says much the same thing as I just did. I just thought it was worth specifically mentioning in the discussion thread (in case some people didn't make it all the way through the blog post).
akeefer 15y ago
I have to respectfully disagree. First of all, the things he shows aren't inherently complex: adding an additional function to the existing collections library is something that's possible in other languages in less confusing ways (see other examples in this post). The various concepts required to solve the problem in Scala like implicits and higher kinds might be inherently complex, but the problem he's trying to solve isn't inherently complex. And while the Scala solution might not be 100% identical to, say, just adding the method directly to a class in Ruby (since in Scala it's type-safe, etc. etc.) that's still missing the point. The point is that if you sit down and say "I want to add a method to all my collections that works like the existing methods," in Scala it requires an understanding of a huge number of complicated intersecting concepts, whereas in other languages it doesn't. Secondly, excusing complexity by saying "it makes things possible that wouldn't otherwise be possible" isn't really enough of a justification. The question is: do those exact things need to be possible? Or is there some way that gets me 90% of the way there without the complexity, and that's good enough? More power isn't always better, which was a big part of the point of the article. Just dismissing it by saying "well, the complexity lets you do powerful things" doesn't really refute the point of the article, it totally misses it. There's a tradeoff to be made. You might make those tradeoffs differently than I would, which is totally understandable, but we should at least be able to have a conversation about the fact that there is such a tradeoff without people dimissing statements like "Scala is complex" out-of-hand.

jberryman 15y ago

Site is unresponsive for me: http://webcache.googleusercontent.com/search?q=cache:http://yz.mit.edu/wp/true-scala-complexity/&hl=en&strip=1 http://webcache.googleusercontent.com/search?q=cache:http://...

oacgnol 15y ago

I wonder how much of an impact it would be if Scala simply had better documentation. I love the terseness of the code and the complex things you can do, but I'll be damned if I could ever glean ideas from looking at method definitions rather than code examples.

hp 15y ago
I found it essential to read Martin Odersky's Programming in Scala book - if you haven't then I recommend it. It makes many things clear that can be hard to pick up using only the free online resources. There's also a new docs site and it's getting better all the time: http://docs.scala-lang.org/ http://docs.scala-lang.org/
- gtani 15y ago
  There's a lot of good books on the language, this is 1st edition of staircase book, which covers 2.7, so probably the majority was written for 2.8 http://www.artima.com/pins1ed/ http://www.artima.com/pins1ed/ There's also a book manuscript (downloadable from typesafe.com, and Manning.com has 3 books in the pipeline, tho the 3rd, on FP by Tony Morris doesn't show there.

deleted 15y ago

[deleted]

reinhardt 15y ago

Just wow. My eyes started glazing over at "views don’t chain. Instead, use view bounds" and he completely lost me on the next slide with generics. I had Scala in my mental TODO/Maybe list but after skimming over this post I think I'll pass.

Confusion 15y ago
Yeah, the post went steeply uphill there. But you can actually understand the gist of most of what he's saying without actually understanding what it means that "views don’t chain. Instead, use view bounds". And somewhat further on, it goes downhill again. In the simplest form, you can use Scala as a better Java. It's worth exploring it just for that comparison. And then there's much, much more you can do, if you want to, but it's also a very nice language if you don't try to add methods to the collections library and don't use dependent types.
SomeOtherGuy 15y ago
So, a guy tries to solve a hard problem by using very powerful but very complex features that 99.9999999% of scala devs would never touch, and you take that as a reason to avoid scala? And then you come here to tell everyone how absurd your decision making process is?
hythloday 15y ago
I'd have the opposite reaction, I think: I'd want to learn a language because it introduced new concepts (and thus, terminology). It seems to me that it's a bit pointless to learn a language if you already intimately understand its theoretical underpinnings.

dxbydt 15y ago

Regarding some of the "brainteasers" posted in the article, I am assuming that most of them are simply bugs that will get ironed out in version 2.9.2 or thereabouts ? eg. The following fails: Set(1,2,3).toIndexedSeq sortBy (-_) But doing the same in 2 steps, ie. after assignment, works val xs = Set(1,2,3).toIndexedSeq; xs sortBy (-_) I have been bitten by this several times now, so I don't unnecessarily chain > 2 functions even if it does compile ( which also solves one other brainteaser posed in that article ) Have also seen the problem with the add function, when I wrote a matrix manipulation library. def add(x: Int, y: Int) = x + y add(1,_) fails, but add(_,_) works. Even though the error message " missing parameter type for expanded function" seems reasonable, and providing the parameter type ie. add(1,_:Int), does compile, the bahavior is hard to explain to newbies. His point about hanging your hat on asInstanceOf[...] and the resulting code being clumsy...ok, guilty as charged, but then you only have so many hours in a day, and mgmt is paying $$ to solve boring business problems ( compute the asset quality of five million loans in thirty lines of business over twelve quarters using scala ) and not mucking around ( add a filterMap to the scala collection library to get an idiomatic Scala implementation ) I enjoyed the programming snippets in the article very much, but I still don't get what the point of the rant was. He goes on and on about "lack of acknowledgement of complexity", but what does that really mean ? Does he want like a gold star ? Even if everybody acknowledges that scala is complex, what then ? Eventually some of it will get fixed & the rest will not & life will go on. Why be a downer at such an early phase of growth of the language ? All this talk of excess complexity will simply scare off the early adopters. Java has been around for 15 years and we still don't have decent generics. Scala is so far ahead in such a short time. Patience, etc.

eropple 15y ago
> Even if everybody acknowledges that scala is complex, what then ? Eventually some of it will get fixed & the rest will not & life will go on. Why be a downer at such an early phase of growth of the language ? Because things don't get fixed if people won't talk about them. Irrational defenses (like calling someone who points out legitimate problems a "downer") are a hindrance toward making Scala better than it is right now.

modersky 15y ago

First, I think a blog post ending with "bring on the flames" should have comments enabled, otherwise it's not really fair. It comes across as whining without wanting to listen to advice or a response. Second, I think the blog post is useful because it shows what's wrong with some (fortunately very small) part of the Scala ecosystem, and because it points to a way to fix it. Here's a quick recap: The author tries to add arbitrary operations to Scala's Seq abstraction without changing its source code and wants them to work also on arrays (which are plain old Java arrays), without any extra work. Arrays in Java support: length, index, and update; that's it. There is as far as I know no language in existence that allows the precise thing the author wants to achieve. And there are many variations, such as adding only to Seq or only to Array that would be really easy in Scala but still impossible in most other mainstream languages. The author then throws all the machinery he can think of at the problem to still achieve the same non-result. Well, tough luck. He might have hit a thing that's simply impossible to do in a generic way, given the tools we currently have. In fact, I have not checked whether there would be a way to achieve the result that he wants because that's beside the point. There are always limits to a generic formulation that will force you at some point to treat things on a case by case basis. The problem is that, in trying to achieve his impossible goal, the author (mis-)uses a lot of the most powerful features of Scala, and concludes that Scala is simply too complex for helping him achieve the result. I believe he wrote this blog post to prompt the maintainers of Scala to add even more power to the language so that he can achieve his goal (the only other motivation I can think of is that he's trying to actively damage the ecosystem he writes code in, but that would make no sense to me). My response will probably not please him. I think that we need to take away sharp knifes from people who have a tendency to cut themselves. I was always proud that in Scala you could do in a library where in other languages you had to change the compiler. Inevitably, some of this is cutting edge stuff. We have tried many times to clarify the boundaries, for instance when I defined the Scala levels http://www.scala-lang.org/node/8610 http://www.scala-lang.org/node/8610 But we can't prevent a developer who prides himself to "stroll right through level L3" to get hurt. So, I believe here is what we need do: Truly advanced, and dangerously powerful, features such as implicit conversions and higher-kinded types will in the future be enabled only under a special compiler flag. The flag will come with documentation that if you enable it, you take the responsibility. Even with the flag disabled, Scala will be a more powerful language than any of the alternatives I can think of. And enabling the flag to do advanced stuff is in any case much easier than hacking your own compiler. I would be interested to read your comments on this proposal.

jonshea 15y ago

If anyone is able, would you please explain to me these two questions from the quiz? I’m stumped. Why does `toSeq` compile, but not `toIndexedSeq`? Set(1,2,3).toIndexedSeq sortBy (-_) Set(1,2,3).toSeq sortBy (-_) Why does `h` compile, but `f` does not? def add(x: Int, y: Int) = x + y val f = add(1,_) val h = add(_,_)

extempore 15y ago
toIndexedSeq takes a type parameter (I'm not sure why - it appears unncessary) which means the result of toIndexedSeq is not known when it is attempting to infer the Ordering. toSeq doesn't take one, it's known to be Seq[Int]. The other looks like some quirk of partial application. It's unlikely there's any fundamental reason, only an implementation imperfection.
- jonshea 15y ago
  Good to know. Thanks Paul!

OlegYch 15y ago

Very good post. One point with which i cannot agree is "In fact, it is impossible to insert a new method that behaves like a normal collection method. " Please see http://ideone.com/ePUHG http://ideone.com/ePUHG An excerpt: import MyEnhancements._ println("qwe".quickSort) println(Array(2,0).quickSort) println(Seq(2,0).quickSort)

etorreborre 15y ago
Miles Sabin's solution is also worth a look: https://gist.github.com/f83892f65f63b14a1f75 https://gist.github.com/f83892f65f63b14a1f75 It uses dependent types which will be included by default in Scala 2.10. I don't consider this as "simple" but my point of view is that Computer Science is not "simple" :-). And having a language supporting that level of genericity is really helpful for type-safety and code reuse.
- milessabin 15y ago
  @OlegYch's solution also represents a good trade off of complexity vs. convenience: his solution is simpler but doesn't get along so nicely with type inference; mine gets on just fine with type inference, but is more complex and depends (no pun intended) on dependent method types. As ever YMMV.

pron 15y ago

I think that the source of the problem is that Scala is simply not an opinionated language. You can go functional, you can go OO; you can go immutable, you can go with locks etc. And because it tries to do everything it falls on its face, but worse than that - its users don't really know HOW you're supposed to use it. I always like to contrast Scala with Clojure and Erlang. Both are very opinionated: they have a philosophy of tackling problems, not just a set of tools, and that's why they are so elegant and beautiful (well, at least Clojure is. Erlang is showing its age at times). Even languages more similar to Scala like Groovy or Kotlin are more opinionated and more focused (and thus more elegant and simpler) than Scala. They are trying to be a more modern Java, a blue-collar OO language with some modern fun. But it seems like Scala is trying to push not only tools but concepts, only it hasn't decided which concepts are best so it's pushing all of them at the same time. The result is not only frustration but harmful education as to the best way go forward.

soc88 15y ago
Both "more opinionated" languages are considerably slower than Scala, so sometimes it is necessary to fall back to "ugly, imperative" code and those languages make sure you will hate that experience. The difference imho is that Scala doesn't punish you for trying to be fast where necessary. Apart from that I would really like to where Groovy or Kotlin are "more elegant or simpler". I would have probably looked into the specification but something like that doesn't even exist for Groovy. From my last journey into Groovy I learned that this language is substantially underdocumented and buggy as hell. I prefer not touching it anymore.
- pron 15y ago
  Actually falling back to ugly imperative and super-fast code in Clojure is extremely easy, and its integration with Java is a pleasure. But I don't want to pit a specific language against Scala. My point is that it isn't clear (to anyone possibly), what Scala IS, other than a kitchen-sink language for programming concepts. Some things in Scala are easy and some things are hard, but it's not always clear why. I'd expect the "good" things (whatever is considered good by the language designers, their expert opinion, that is) to be easy and the "bad" things hard. But in Scala it seems that whether something is easy or not depends on how well it fits with the language's algebraic model, and not how well it fits with recommended practice. Immutable is easy (that must be good, no?), but mutable is just as easy (so, wait, I'm supposed to... what exactly?); functional is easy, but so is imperative; implicits are easy but extension methods, as the blog post shows, can be really hard; type inference is easy except when it's not. See the problem?