5 ms·

Scala's minor versions don't break compatibility. A year ago Scala 2.11 (major) was being released, which means you're talking about either 2.10.x or 2.11.x, ne

Scala's minor versions don't break compatibility. A year ago Scala 2.11 (major) was being released, which means you're talking about either 2.10.x or 2.11.x, neither of which broke binary compatibility between minor versions. And major versions usually have source level compatibility, our upgrade from 2.10 to 2.11 being pretty smooth. The fundamental difference is that Clojure gets distributed as source and not as compiled .class files. Being an interpreted language that gets compiled on the fly does have some advantages. But it also has drawbacks. Startup performance suffers, the Java interoperability story is worse, many tools such as Android's toolchain expect bytecode, etc ... The problem that Scala has (and also Clojure, as soon as you do AOT) is that Scala's idioms do not translate well into Java bytecode, as Java's bytecode is designed for, well, Java. Therefore even small and source-compatible changes, like adding another parameter with a default value to a method or like adding a method to a trait, can trigger binary incompatibilities. The plan for Scala is to embed in the .class, besides the bytecode, the abstract syntax tree built by the compiler and then the compiler can take a JAR and repurpose it for whatever platform you want. This is part of the TASTY and the Scala-meta projects. If you think about it it's not that far from what Clojure is doing, except that this is done in the context of a static language that doesn't rely on the presence of an interpreter at runtime. Of course, LISPs are pretty cool. And of course, you'll still need to recompile your projects, but at least then the dependencies won't have to be changed.

pron 11y ago
> and also Clojure, as soon as you do AOT Not so much. You can do separate compilation with Clojure to a far greater extent than Scala (perhaps even completely). The Java interfaces of Clojure functions haven't changed in incompatible ways in a very, very long time.
- mateuszf 11y ago
  That's just the power of good upfront design.
- bad_user 11y ago
  The Java interfaces that Clojure relies on don't matter that much. On the other hand what is the bytecode representation of a protocol or of a multi-method? Scala has an equivalent representation for everything it has. For example traits are just Java interfaces but with corresponding static methods. Another example is default parameters (a concept Java doesn't have) is done with method overloading. The Scala compiler has to infer Scala-specific stuff (like signatures using Scala features) from compiled bytecode. Functions that aren't methods (e.g. anonymous functions) get compiled either to static methods, or to classes that have to be instantiated (in case it's about closures closing over their context). Scala does not have the concept of static methods, but it has singleton objects, which can also implement interfaces. Of course, these get compiled to static methods, but there's also a singleton instance instantiated for those cases in which you want to use that singleton object as a real polymorphic instance. Etc, etc... So there's a protocol in place for how to encode this in the bytecode, there's a protocol for everything. And so even small changes in Scala's standard libraries can trigger big bytecode changes that end up being backwards incompatible. Clojure doesn't have to do this, because Clojure dependencies get distributed as source-code, as I've said.
  - pron 11y ago
    > Clojure doesn't have to do this, because Clojure dependencies get distributed as source-code Clojure doesn't have to do this because it was designed to allow separate compilation as much as possible[1], and because it hardly ever changes binary representation in backwards-incompatible ways. Protocols and multimethods are indeed handled at the call-site, but in such a way that a change to the protocol/multimethods don't require re-compilation of the callsite[2]. Similarly, Kotlin, a statically typed language distributed in binary, is also designed to allow separate compilation as much as possible[3]. If a feature would break that property (e.g. require re-compilation of a callsite when an implementation changes at the call target), that feature simply isn't added to the language. This is a design that admits that extra-linguistic features (like separate compilation) are as important as linguistic abstractions (sometimes more important). BTW, separate compilation isn't only a concern with Java class files. Object file linking also places limits on how languages can implement abstractions yet still support separate compilation. Some languages place less emphasis on this than others. [1]: In fact, as a Lisp, Clojure's unit of (separate) compilation isn't even the file but the top-level expression. No top-level expression should require re-compilation if anything else changes. [2]: The implementation of protocols: https://github.com/clojure/clojure/blob/master/src/clj/clojure/core_deftype.clj#L508 https://github.com/clojure/clojure/blob/master/src/clj/cloju... [3]: Even Java doesn't support 100% separate compilation. There are rare cases where changes to one class requires recompilation of another.
- sreque 11y ago
  This doesn't seem true. See http://comments.gmane.org/gmane.comp.java.clojure.user/85930 http://comments.gmane.org/gmane.comp.java.clojure.user/85930 "Binary compatibility across releases is not guaranteed, but it is something we strive to maintain if possible. "
  - pron 11y ago
    If you read what it says, they've changed an implementation of a function which may change ordering of unordered collections, and that may break some people's code if they relied on some specific ordering. They are not talking about breaking interfaces. In fact, they specifically say they are not aware of any binary incompatibilities. Their "strive to maintain" works out very well in practice, so far.
    - puredanger 11y ago
      I wrote that sentence and you should take it as written. In general, we strive to maintain binary compatibility (old AOT-compiled Clojure code should continue to run on newer version), we do not exhaustively test for or guarantee that. At times we have broken this in alphas etc and we take it as a high priority to fix such things.
puredanger 11y ago
Clojure itself actually is distributed as compiled .class files. We take some effort to ensure that there are not changes that break binary compatibility from AOT-compiled Clojure code (which is not uncommon) and break its calls into the Clojure compiler or runtime.

pron 11y ago

> and also Clojure, as soon as you do AOT Not so much. You can do separate compilation with Clojure to a far greater extent than Scala (perhaps even completely). The Java interfaces of Clojure functions haven't changed in incompatible ways in a very, very long time.

mateuszf 11y ago
That's just the power of good upfront design.
bad_user 11y ago
The Java interfaces that Clojure relies on don't matter that much. On the other hand what is the bytecode representation of a protocol or of a multi-method? Scala has an equivalent representation for everything it has. For example traits are just Java interfaces but with corresponding static methods. Another example is default parameters (a concept Java doesn't have) is done with method overloading. The Scala compiler has to infer Scala-specific stuff (like signatures using Scala features) from compiled bytecode. Functions that aren't methods (e.g. anonymous functions) get compiled either to static methods, or to classes that have to be instantiated (in case it's about closures closing over their context). Scala does not have the concept of static methods, but it has singleton objects, which can also implement interfaces. Of course, these get compiled to static methods, but there's also a singleton instance instantiated for those cases in which you want to use that singleton object as a real polymorphic instance. Etc, etc... So there's a protocol in place for how to encode this in the bytecode, there's a protocol for everything. And so even small changes in Scala's standard libraries can trigger big bytecode changes that end up being backwards incompatible. Clojure doesn't have to do this, because Clojure dependencies get distributed as source-code, as I've said.
- pron 11y ago
  > Clojure doesn't have to do this, because Clojure dependencies get distributed as source-code Clojure doesn't have to do this because it was designed to allow separate compilation as much as possible[1], and because it hardly ever changes binary representation in backwards-incompatible ways. Protocols and multimethods are indeed handled at the call-site, but in such a way that a change to the protocol/multimethods don't require re-compilation of the callsite[2]. Similarly, Kotlin, a statically typed language distributed in binary, is also designed to allow separate compilation as much as possible[3]. If a feature would break that property (e.g. require re-compilation of a callsite when an implementation changes at the call target), that feature simply isn't added to the language. This is a design that admits that extra-linguistic features (like separate compilation) are as important as linguistic abstractions (sometimes more important). BTW, separate compilation isn't only a concern with Java class files. Object file linking also places limits on how languages can implement abstractions yet still support separate compilation. Some languages place less emphasis on this than others. [1]: In fact, as a Lisp, Clojure's unit of (separate) compilation isn't even the file but the top-level expression. No top-level expression should require re-compilation if anything else changes. [2]: The implementation of protocols: https://github.com/clojure/clojure/blob/master/src/clj/clojure/core_deftype.clj#L508 https://github.com/clojure/clojure/blob/master/src/clj/cloju... [3]: Even Java doesn't support 100% separate compilation. There are rare cases where changes to one class requires recompilation of another.
sreque 11y ago
This doesn't seem true. See http://comments.gmane.org/gmane.comp.java.clojure.user/85930 http://comments.gmane.org/gmane.comp.java.clojure.user/85930 "Binary compatibility across releases is not guaranteed, but it is something we strive to maintain if possible. "
- pron 11y ago
  If you read what it says, they've changed an implementation of a function which may change ordering of unordered collections, and that may break some people's code if they relied on some specific ordering. They are not talking about breaking interfaces. In fact, they specifically say they are not aware of any binary incompatibilities. Their "strive to maintain" works out very well in practice, so far.
  - puredanger 11y ago
    I wrote that sentence and you should take it as written. In general, we strive to maintain binary compatibility (old AOT-compiled Clojure code should continue to run on newer version), we do not exhaustively test for or guarantee that. At times we have broken this in alphas etc and we take it as a high priority to fix such things.

puredanger 11y ago

Clojure itself actually is distributed as compiled .class files. We take some effort to ensure that there are not changes that break binary compatibility from AOT-compiled Clojure code (which is not uncommon) and break its calls into the Clojure compiler or runtime.