5 ms·

The one place where Scala really doesn't shine is ML. There's nothing even close to equivalent to PyTorch or Tensorflow. Actually, languages with rich type sys

The one place where Scala really doesn't shine is ML. There's nothing even close to equivalent to PyTorch or Tensorflow. Actually, languages with rich type systems in general seem to really struggle in this space.

lihaoyi 6y ago
I'd argue it's just "languages that are not Python struggle in this space". Python has an undoubtably unique ecosystem and gravity around ML, but that's more a unique advantage of Python than a weakness of every other language design
- MrPowers 6y ago
  Scala does have Spark's ML capabilities. Spark's ML capabilities might not be the best, but they can be run on huge datasets. I'm guessing Dask has better model support, but isn't as scalable as Spark ML.
currymj 6y ago
i don't know if it's the type system... Julia has a very rich type system. It's not exactly popular (yet?) but it's certainly very usable and good for numerical work and ML. but they started out with the goal of building a numerical language from day one. I can imagine if you have a completely general purpose language, and want to provide an library that's an interface to low-level numerical libraries, NumPy style, that this task might be more difficult to get right in a language with more complicated types. There seem to be a lot of half-assed unfinished numerical libraries in both Scala and Rust.
- dwohnitmok 6y ago
  Julia has dynamic types. I think the parent is referring to statically typed languages by "rich type systems" (because dynamic types can often have arbitrary richness since they're basically just another form of runtime errors/exceptions).
  - eigenspace 6y ago
    This depends a lot on what one means by richness I suppose. I'd say that Python, Javascript, mosts lisps that don't have CLOS, etc. are all dynamic languages with very non-rich type systems. Julia's type system (and to a lesser extent Common Lisp's) are very different. They can do a lot of extremely powerful things with their type systems, just not (usually) static type checking (though even that's changing in julia land with JET.jl: https://github.com/aviatesk/JET.jl https://github.com/aviatesk/JET.jl)
    - dwohnitmok 6y ago
      Yeah; I think the distinction with runtime types among dynamic languages is that you can always emulate any other system's dynamic type system with some amount of constant syntactic overhead in any other system. E.g. you could write a CLOS system or Julia's multiple dispatch in Javascript if you wanted. It would just be extremely ugly because you would have to wrap every function and every function call (so as to intercept the native function call and substitute your own). Nonetheless though that's "only" a constant amount of (painful) syntactic noise per call, which means that it is conceivable for a library to provide this functionality. And in fact occasionally libraries will require users to wrap all functions in order to give those functions certain "superpowers." It is of course nicer if it integrates very nicely with the language rather than being full of wrappers everywhere so there is still a lot of value in these runtime type systems being built from the ground-up with certain features in mind. But it's a very different world from static types, where usually if one static type system is missing features of another, no amount of emulation will get you there.
      - eigenspace 6y ago
        I'm not sure that's a particularly convincing argument. Bolting a type system on a dynamic language is incredibly hard to do if you're trying to make a type system that's worth using. It's a fundamental language design problem, and if the type system is doing anything worth while, it'd probably take around a decade to make one that is reasonably sound and robust. At that level of commitment, one can just turn around and look at your static language and say "Well, couldn't you just implement a new language with a new type system in your language?" (assuming your language is turing complete). This really isn't that different. Meanwhile, Julia's type system is actually a fundamental player in it's optimization schemes (the compiler essentially breaks up programs into statically typed chunks and optimizes those separately), so rather than inducing a constant overhead, the type system is responsible for some of the fastest code that's possible to write, including BLAS libraries that are competitive with OpenBLAS, BLIS, and MKL (https://github.com/JuliaLinearAlgebra/Octavian.jl/issues/24 https://github.com/JuliaLinearAlgebra/Octavian.jl/issues/24)
dwohnitmok 6y ago
> Actually, languages with rich type systems in general seem to really struggle in this space. I suspect there's two factors. First, I think the current crop of languages with "rich type systems" (I imagine you're thinking of Haskell, Scala, ML and the like?) basically all are in this weird uncanny valley where people get tempted into trying all sorts of cool type tricks to make everything line up, and the type systems are just not _quite_ expressive enough to make it all go well. Most of the time if you just use arrays/your language's usual collection type of numbers (or arrays of arrays... etc.) in those static languages you'll do great. To take it to next level and really model ML faithfully I think you probably will need some version of dependent types and there just aren't really any good implementations of that at the moment. Secondly, a lot of ML is powered by data scientists and statisticians to whom dynamic languages mesh probably better with their own educational background (when was the last time you saw types written out in a stats textbook?). It's really kind of jarring and annoying to have to deal with type systems when you've never done any professional programming before.
- bgorman 6y ago
  What I have seen in the wild is that in dynamic languages programs end up being "stringly" typed, where a mistyped string in the program results in an error, and programs needlessly rely on array positioning. ML languages that support variants eliminate these unnecessary errors are no cost to runtime performance. What problems have you encountered where a ML system couldn't represent things better than primitive type system? The two things that come to my mind are when the program has to accept arbitrary user input (strings), and writing an extremely expressive program i.e. a lisp interpreter with support for variable argument length functions. Even in these cases, I think the guarantees around the messy sections help with refactoring and my general sanity.
  - dwohnitmok 6y ago
    > What problems have you encountered where a ML system couldn't represent things better than primitive type system? I assume you mean a primitive type system to be a strict subset of an ML type system? So in that case the answer in theory is clearly negative. The problem is more that ML type systems give you more, and fool you into thinking that's enough to try to use the extra bits, when that's not quite enough. > ML languages that support variants eliminate these unnecessary errors are no cost to runtime performance. Closed variants are fantastic. I love them. I miss them every time I use a programming language without them. What I'm talking about is requirements like making sure that all your matrix dimensions line up (you mention variadic arguments, which is another good one if you're trying to follow a lot of mathematical notation). A lot of ML-inspired statically typed languages let you come really close to making all that work. But are just unergonomic enough to make things painful if you really try to statically verify those things.
MasterDDT 6y ago
Its because data scientists dont want to learn scala if they don't have to. As a scala developer, I see https://jupyter.org/ https://jupyter.org/ and https://almond.sh/ https://almond.sh/ and cant think of a good reason to use the first one. All the OP's comments would apply.
forgetfulness 6y ago
Maybe you can argue that it's been outclassed but Spark's ML toolbox is plenty serviceable for commercial applications and used all over the world. Now that all the APIs are centered around Dataframes, Scala's rich type system has been relegated to providing implicits for the SQL-like DSL, which is a shame, but it's still an eminently Scala-centric tool.

lihaoyi 6y ago

I'd argue it's just "languages that are not Python struggle in this space". Python has an undoubtably unique ecosystem and gravity around ML, but that's more a unique advantage of Python than a weakness of every other language design

MrPowers 6y ago
Scala does have Spark's ML capabilities. Spark's ML capabilities might not be the best, but they can be run on huge datasets. I'm guessing Dask has better model support, but isn't as scalable as Spark ML.

currymj 6y ago

i don't know if it's the type system... Julia has a very rich type system. It's not exactly popular (yet?) but it's certainly very usable and good for numerical work and ML. but they started out with the goal of building a numerical language from day one. I can imagine if you have a completely general purpose language, and want to provide an library that's an interface to low-level numerical libraries, NumPy style, that this task might be more difficult to get right in a language with more complicated types. There seem to be a lot of half-assed unfinished numerical libraries in both Scala and Rust.

dwohnitmok 6y ago
Julia has dynamic types. I think the parent is referring to statically typed languages by "rich type systems" (because dynamic types can often have arbitrary richness since they're basically just another form of runtime errors/exceptions).
- eigenspace 6y ago
  This depends a lot on what one means by richness I suppose. I'd say that Python, Javascript, mosts lisps that don't have CLOS, etc. are all dynamic languages with very non-rich type systems. Julia's type system (and to a lesser extent Common Lisp's) are very different. They can do a lot of extremely powerful things with their type systems, just not (usually) static type checking (though even that's changing in julia land with JET.jl: https://github.com/aviatesk/JET.jl https://github.com/aviatesk/JET.jl)
  - dwohnitmok 6y ago
    Yeah; I think the distinction with runtime types among dynamic languages is that you can always emulate any other system's dynamic type system with some amount of constant syntactic overhead in any other system. E.g. you could write a CLOS system or Julia's multiple dispatch in Javascript if you wanted. It would just be extremely ugly because you would have to wrap every function and every function call (so as to intercept the native function call and substitute your own). Nonetheless though that's "only" a constant amount of (painful) syntactic noise per call, which means that it is conceivable for a library to provide this functionality. And in fact occasionally libraries will require users to wrap all functions in order to give those functions certain "superpowers." It is of course nicer if it integrates very nicely with the language rather than being full of wrappers everywhere so there is still a lot of value in these runtime type systems being built from the ground-up with certain features in mind. But it's a very different world from static types, where usually if one static type system is missing features of another, no amount of emulation will get you there.
    - eigenspace 6y ago
      I'm not sure that's a particularly convincing argument. Bolting a type system on a dynamic language is incredibly hard to do if you're trying to make a type system that's worth using. It's a fundamental language design problem, and if the type system is doing anything worth while, it'd probably take around a decade to make one that is reasonably sound and robust. At that level of commitment, one can just turn around and look at your static language and say "Well, couldn't you just implement a new language with a new type system in your language?" (assuming your language is turing complete). This really isn't that different. Meanwhile, Julia's type system is actually a fundamental player in it's optimization schemes (the compiler essentially breaks up programs into statically typed chunks and optimizes those separately), so rather than inducing a constant overhead, the type system is responsible for some of the fastest code that's possible to write, including BLAS libraries that are competitive with OpenBLAS, BLIS, and MKL (https://github.com/JuliaLinearAlgebra/Octavian.jl/issues/24 https://github.com/JuliaLinearAlgebra/Octavian.jl/issues/24)

dwohnitmok 6y ago

> Actually, languages with rich type systems in general seem to really struggle in this space. I suspect there's two factors. First, I think the current crop of languages with "rich type systems" (I imagine you're thinking of Haskell, Scala, ML and the like?) basically all are in this weird uncanny valley where people get tempted into trying all sorts of cool type tricks to make everything line up, and the type systems are just not _quite_ expressive enough to make it all go well. Most of the time if you just use arrays/your language's usual collection type of numbers (or arrays of arrays... etc.) in those static languages you'll do great. To take it to next level and really model ML faithfully I think you probably will need some version of dependent types and there just aren't really any good implementations of that at the moment. Secondly, a lot of ML is powered by data scientists and statisticians to whom dynamic languages mesh probably better with their own educational background (when was the last time you saw types written out in a stats textbook?). It's really kind of jarring and annoying to have to deal with type systems when you've never done any professional programming before.

bgorman 6y ago
What I have seen in the wild is that in dynamic languages programs end up being "stringly" typed, where a mistyped string in the program results in an error, and programs needlessly rely on array positioning. ML languages that support variants eliminate these unnecessary errors are no cost to runtime performance. What problems have you encountered where a ML system couldn't represent things better than primitive type system? The two things that come to my mind are when the program has to accept arbitrary user input (strings), and writing an extremely expressive program i.e. a lisp interpreter with support for variable argument length functions. Even in these cases, I think the guarantees around the messy sections help with refactoring and my general sanity.
- dwohnitmok 6y ago
  > What problems have you encountered where a ML system couldn't represent things better than primitive type system? I assume you mean a primitive type system to be a strict subset of an ML type system? So in that case the answer in theory is clearly negative. The problem is more that ML type systems give you more, and fool you into thinking that's enough to try to use the extra bits, when that's not quite enough. > ML languages that support variants eliminate these unnecessary errors are no cost to runtime performance. Closed variants are fantastic. I love them. I miss them every time I use a programming language without them. What I'm talking about is requirements like making sure that all your matrix dimensions line up (you mention variadic arguments, which is another good one if you're trying to follow a lot of mathematical notation). A lot of ML-inspired statically typed languages let you come really close to making all that work. But are just unergonomic enough to make things painful if you really try to statically verify those things.

MasterDDT 6y ago

Its because data scientists dont want to learn scala if they don't have to. As a scala developer, I see https://jupyter.org/ https://jupyter.org/ and https://almond.sh/ https://almond.sh/ and cant think of a good reason to use the first one. All the OP's comments would apply.

forgetfulness 6y ago

Maybe you can argue that it's been outclassed but Spark's ML toolbox is plenty serviceable for commercial applications and used all over the world. Now that all the APIs are centered around Dataframes, Scala's rich type system has been relegated to providing implicits for the SQL-like DSL, which is a shame, but it's still an eminently Scala-centric tool.