5 ms·

Code with many implicit parameters and higher-kinded types is hard to compile, as the compiler needs to solve essentially a Prolog-ish unification problem. Scal

by psuter 12y ago

Code with many implicit parameters and higher-kinded types is hard to compile, as the compiler needs to solve essentially a Prolog-ish unification problem. Scalaz is a popular library written largely in this style (e.g. [1]). (FWIW I've never seen 1 line/minute.) [1] https://github.com/scalaz/scalaz/blob/series/7.2.x/core/src/main/scala/scalaz/Foldable.scala https://github.com/scalaz/scalaz/blob/series/7.2.x/core/src/...

zak_mc_kracken 12y ago
OCaml and Haskell offer very similar functionalities and their compilation times are lightning fast. The main reason why Scala is slow is because the compiler is poorly written. Take a look at the articles from Paul Phillips, one of the Typesafe cofounders who left the company in frustration, he's probably the person who knows scalac the best and he says the code base is basically hopeless. Note that 1/3rd of Martin's presentation is about Dotty, and that's basically where he spends most of his time these days, he hardly contributes to Scala any more [1]. [1] https://twitter.com/odersky/status/574665768484339713 https://twitter.com/odersky/status/574665768484339713
- frowaway001 12y ago
  > OCaml and Haskell offer very similar functionalities and their compilation times are lightning fast. It took only one sentence to show that you never used OCaml or Haskell, so I'm not sure how serious the rest of your claims should be taken.
  - mercurial 12y ago
    OCaml compiles pretty fast, especially if you target bytecode. Haskell on the other hand... is still faster than Scala, but not "lightning fast".
    - frowaway001 12y ago
      > OCaml compiles pretty fast, especially if you target bytecode. Yes, but OCaml doesn't "offer very similar functionalities". Haskell is quite a bit slower if you consider that you often have to compile not only your own code, but also the sources of your dependencies in the beginning. Additionally, you have to fight with Cabal, which tends to be very unpleasant due to the lack of any reasonable support for exotic things like "versions". Then you might discover hacks like "sandboxing". Hours not spent coding.
- yummyfajitas 12y ago
  I can't speak about OCaml, but Haskell doesn't have subtypes. That's where a lot of the complexity of Scala comes from as well as being a source of annoying bugs: def f(x: Int) = if (x % 2 == 0) { x } else { None } (This compiles, but the type of `f` is not `f: Int => Option[Int]`.) [edit: the bug is not in the scala compiler, the bug is in my code - I almost certainly did not want `f` to have type `Int => Any`. My phrasing, now edited, was misleading. ]
  - kod 12y ago
    That is not a bug. That is subtype inference working correctly. If you (mistakenly) expected the return type of f to be Option[Int], say so, and you'll get a compile time error telling you about your mistake: scala> def f(x: Int): Option[Int] = if (x % 2 == 0) { x } else { None } <console>:7: error: type mismatch; found : Int required: Option[Int] def f(x: Int): Option[Int] = if (x % 2 == 0) { x } else { None }
  - azth 12y ago
    scala> def f(x: Int) = if (x % 2 == 0) { x } else { None } f: (x: Int)Any Which is expected. The only shared type between both branches of the `if` expression is `Any`.
  - zak_mc_kracken 12y ago
    This is correct given Scala's limitation: the only common supertype between Int and Option is Any. A language that supports union types (e.g. Ceylon) will type this expression as Int|Option[Int], which is as specific as you can get.
    - axiomabsolute 12y ago
      I believe Ceylon will actually further refine Int | Option[Int] to simply Option[Int]. Option[Int] is simply an alias to the type Int | Null, so Int | Option[Int] would become Int | Int | Null after replacement, which simplifies to Int | Null since Int | Int is equivalent to just Int.
modersky 12y ago
The one line per minute was observed when trying to compile a query over an HList backed record with more than 300 columns. The problem was a polynomial increase of the size of the implicit search tree. This is due to the way implicits are used in shapeless HList library. Slick has a different approach which is much faster, but slightly less general. The gist is that once you have [edit: recursive] typeclasses or implicits your search can become arbitrarily large and slow. There's nothing that can be done about it by the compiler except arbitrarily pruning the search tree. Or otherwise put, you should factor in implicit search complexity when designing your libraries.

by psuter 12y ago

zak_mc_kracken 12y ago

OCaml and Haskell offer very similar functionalities and their compilation times are lightning fast. The main reason why Scala is slow is because the compiler is poorly written. Take a look at the articles from Paul Phillips, one of the Typesafe cofounders who left the company in frustration, he's probably the person who knows scalac the best and he says the code base is basically hopeless. Note that 1/3rd of Martin's presentation is about Dotty, and that's basically where he spends most of his time these days, he hardly contributes to Scala any more [1]. [1] https://twitter.com/odersky/status/574665768484339713 https://twitter.com/odersky/status/574665768484339713

frowaway001 12y ago
> OCaml and Haskell offer very similar functionalities and their compilation times are lightning fast. It took only one sentence to show that you never used OCaml or Haskell, so I'm not sure how serious the rest of your claims should be taken.
- mercurial 12y ago
  OCaml compiles pretty fast, especially if you target bytecode. Haskell on the other hand... is still faster than Scala, but not "lightning fast".
  - frowaway001 12y ago
    > OCaml compiles pretty fast, especially if you target bytecode. Yes, but OCaml doesn't "offer very similar functionalities". Haskell is quite a bit slower if you consider that you often have to compile not only your own code, but also the sources of your dependencies in the beginning. Additionally, you have to fight with Cabal, which tends to be very unpleasant due to the lack of any reasonable support for exotic things like "versions". Then you might discover hacks like "sandboxing". Hours not spent coding.
yummyfajitas 12y ago
I can't speak about OCaml, but Haskell doesn't have subtypes. That's where a lot of the complexity of Scala comes from as well as being a source of annoying bugs: def f(x: Int) = if (x % 2 == 0) { x } else { None } (This compiles, but the type of `f` is not `f: Int => Option[Int]`.) [edit: the bug is not in the scala compiler, the bug is in my code - I almost certainly did not want `f` to have type `Int => Any`. My phrasing, now edited, was misleading. ]
- kod 12y ago
  That is not a bug. That is subtype inference working correctly. If you (mistakenly) expected the return type of f to be Option[Int], say so, and you'll get a compile time error telling you about your mistake: scala> def f(x: Int): Option[Int] = if (x % 2 == 0) { x } else { None } <console>:7: error: type mismatch; found : Int required: Option[Int] def f(x: Int): Option[Int] = if (x % 2 == 0) { x } else { None }
- azth 12y ago
  scala> def f(x: Int) = if (x % 2 == 0) { x } else { None } f: (x: Int)Any Which is expected. The only shared type between both branches of the `if` expression is `Any`.
- zak_mc_kracken 12y ago
  This is correct given Scala's limitation: the only common supertype between Int and Option is Any. A language that supports union types (e.g. Ceylon) will type this expression as Int|Option[Int], which is as specific as you can get.
  - axiomabsolute 12y ago
    I believe Ceylon will actually further refine Int | Option[Int] to simply Option[Int]. Option[Int] is simply an alias to the type Int | Null, so Int | Option[Int] would become Int | Int | Null after replacement, which simplifies to Int | Null since Int | Int is equivalent to just Int.

modersky 12y ago

The one line per minute was observed when trying to compile a query over an HList backed record with more than 300 columns. The problem was a polynomial increase of the size of the implicit search tree. This is due to the way implicits are used in shapeless HList library. Slick has a different approach which is much faster, but slightly less general. The gist is that once you have [edit: recursive] typeclasses or implicits your search can become arbitrarily large and slow. There's nothing that can be done about it by the compiler except arbitrarily pruning the search tree. Or otherwise put, you should factor in implicit search complexity when designing your libraries.