4 ms·

Interesting analogy, incredible how Spark drives the Scala helicopter like a car; they seem to have nailed parallel parking ;-) Seriously though, I doubt Kotli

• mike_hearn 11y ago
  Do you actually know Kotlin? It has many features for DSL support, including operator overloading (though not the ability to invent random new operators fortunately), infix notation, a form of exhaustive pattern matching, extension functions, annotation processing, inlineable anonymous extension functions which sounds bizarre but actually is key to defining many DSLs in the language, and so on. It's a different approach to Scala but to say there's no way to do DSLs isn't right. As an example, check out funKtionale. It's a library that adds various functional features like function currying and partial application to Kotlin, entirely by using its DSL features.
  • pron 11y ago
    Or our Kotlin actor API: http://blog.paralleluniverse.co/2015/06/04/quasar-kotlin/ http://blog.paralleluniverse.co/2015/06/04/quasar-kotlin/
• pron 11y ago
  1. None of those features (except sealed ADTs, which really aren't necessary) is missing from Kotlin. The set of Scala features Spark is using is pretty much identical to Kotlin. 2. I don't see how IBM's investment has anything to do with Scala, though. Spark is a great product, and deserves an investment even if it were written in BASIC. 3. Even if Spark did rely on Scala features that aren't in Kotlin (it doesn't), Spark is a library/framework/middleware completely unlike enterprise projects. It is orders of magnitude smaller, for one. That is not to say it isn't awesome, but I wouldn't extrapolate from its choice of technologies (including the language) to the enterprise. 4. Clojure makes writing DSLs easier and with far less magic. Groovy DSLs are also as powerful as Scala's and easier to write. Kotlin DSLs are not as powerful, but are more than enough for Spark. 5. I've never, ever, said Scala is unusable. In fact, I've always said (and believed), it is a very powerful language. I have said that that kind of power is not what most companies want or need, and that that kind of unopinionated kitchen-sink, cutting-edge language is probably not suitable for long-time maintenance in large companies. I completely stand by this. My problem with Scala, again, is not that it's a bad helicopter, but that it insists that it's a good choice if what you want is a car (unless you actually try to drive it like a car -- as Spark does -- in which case you invite the ire of the entire community), and, IMO, it isn't. 6. It's OK to not like your language. It's OK to say it. It's OK to explain why. And it's OK for you to disagree. That doesn't make me biased, and certainly not some kind of bogeyman. It also doesn't justify ad hominem attacks, which -- unlike my criticism of Scala -- are completely unfounded.
  • virtualwhys 11y ago
    Good points, as always. > None of those features (except sealed ADTs, which really aren't necessary) is missing from Kotlin. The set of Scala features Spark is using is pretty much identical to Kotlin. I didn't mention implicit conversions, but that's one area where Scala has taken the high risk high reward path. If you look a bit further down on the linked Spark page you'll see the conversions defined [1]. I'd say Spark's style guide is more geared toward Scala consumers and not necessarily library authors (i.e. if you know what you're doing have at it; otherwise keep to a strict subset). Basically, in aggregate Scala's feature set is incredibly powerful; that power (despite the complexity, tooling challenges, and slow builds) is what draws dsl authors to Scala. When IBM throws down an absolute boatload of cash on a project written in Scala, by proxy Scala benefits. Right off the bat there are de facto 3,500 new Scala developers (since the initial IBM Spark team will of course be learning Scala). Furthermore, it's not like competitors aren't aware of this, Spark automically becomes more of a "big deal" as a result, which cannot help but be a boon for Scala. As for other JVM languages, agreed, Clojure has its own supply of awesome, though the lack of (built-in) static types are a no-op here. Groovy, meh, seems to be on a downtrend, I suspect non-immutable-by-default dynamic languages (except of course, javascript) will continue to fall out of favor. Now, Kotlin, it's a nice language. If they can deliver on near Java build times, might just be the Java killer that Scala will never be, or at least not in its current design (where, as you rightly say, there is often more than one way to do the same thing). Let's see how things play out over the next 3-5 years, JVM alternative language landscape is diverse and evolving quickly, good times now and great times ahead... [1] https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl/package.scala#L115 https://github.com/apache/spark/blob/master/sql/catalyst/src...
• pron 11y ago
  To add to my previous comment (now below), Scala DSLs are without rival only in the complexity of their composition. Clojure DSLs are based on macros. Groovy's are on builders and method-missing. Haskell's are on symbols, monad syntax (do) and lazy evaluation. Kotlin's are built with typesafe builders and inline functions. Scala's are built with symbols, right-associative operators, macros, implicits, monad syntax (for) and possibly even dynamics. Even this most powerful yet most dangerously-clever capability offered by some programming language is supported by like five different Scala features with ten different ways of achieving the same goal. It's absolutely perfect if you want to explore different ways of programming. It's not so great if you need to maintain your DSL for the next 10-15 years in a large organization. Besides, given that the JVM has two of most advanced meta-programming tools anywhere, Xtext[1] and JetBrains' own MPS[2], I wouldn't call any JVM language's embedded DSLs "without rival". All this is about to get even better with Truffle/Graal in Java 9. Saw an interesting talk about easily constructing very efficient languages with Truffle and object algebras at ECOOP[3]. So, especially on the JVM, going for an embedded DSL, let alone one built with very complicated techniques (rather than just Lisp macros) is probably not the best way to use DSLs. It has no advantages and lots of disadvantages. [1]: https://eclipse.org/Xtext/ https://eclipse.org/Xtext/ [2]: https://www.jetbrains.com/mps/ https://www.jetbrains.com/mps/ [3]: https://twitter.com/biboudis/status/618402989720662016 https://twitter.com/biboudis/status/618402989720662016