3 ms·

That is true, and certainly relevant, but I believe it is also true that Scala does not meet the criteria for what that were trying to do, because it is not eas

That is true, and certainly relevant, but I believe it is also true that Scala does not meet the criteria for what that were trying to do, because it is not easy (perhaps not possible) to do the kind of static analysis necessary to achieve their goals. For a generic "what language with a good type system should we use with ML?" project, I would totally agree with you that Scala is an obvious choice. But this project was attempting a more specific thing, for which Scala is a less good fit.

johnnycerberus 6y ago
I really don't see how Scala is not fit for this. Swift for Tensorflow was mainly revolving around the Differentiable protocol. There is a paper published in 2019 from Purdue University where they show how AD can be combined with multi-stage programming in Scala. [1] DeepLearning.scala that does compile-time reverse AD. [2] Compute.scala for operator fusion at runtime (JITifying) [3] Here [4] is a Pre-SIP to add reifiable control flow to Scala compiler so AD and operator fusion could happen at compile-time. Moreover, Kotlin also has a library that does type-safe AD. [5] nVidia and Oracle want better interopabiltiy between CUDA and GraalVM through grCuda. [6] [1] Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator - https://arxiv.org/pdf/1803.10228.pdf https://arxiv.org/pdf/1803.10228.pdf [2] https://github.com/ThoughtWorksInc/DeepLearning.scala https://github.com/ThoughtWorksInc/DeepLearning.scala [3] https://github.com/ThoughtWorksInc/Compute.scala https://github.com/ThoughtWorksInc/Compute.scala [4] https://contributors.scala-lang.org/t/pre-sip-for-with-control-flow-an-alternative-to-scala-async-scala-continuations-and-scala-virtualized/4423 https://contributors.scala-lang.org/t/pre-sip-for-with-contr... [5] https://github.com/breandan/kotlingrad https://github.com/breandan/kotlingrad [6] https://developer.nvidia.com/blog/grcuda-a-polyglot-language-binding-for-cuda-in-graalvm/ https://developer.nvidia.com/blog/grcuda-a-polyglot-language...
- sanderjd 6y ago
  Neat! This may have not been well known when they kicked off the project and wrote their reasoning. Here is what they had to say about Scala at the time of the document linked up-thread[0]: "Java / C# / Scala (and other OOP languages with pervasive dynamic dispatch): These languages share most of the static analysis problems as Python: their primary abstraction features (classes and interfaces) are built on highly dynamic constructs, which means that static analysis of Tensor operations depends on "best effort" techniques like alias analysis and class hierarchy analysis. Further, because they are pervasively reference-based, it is difficult to reliably disambiguate pointer aliases." If they were wrong about that, or if the state of the art has progressed in the meantime, that's great! You may well be right that Scala would be a good / the best choice if they started the project today. [0]: https://github.com/tensorflow/swift/blob/main/docs/WhySwiftForTensorFlow.md#which-languages-fit-our-project-requirements https://github.com/tensorflow/swift/blob/main/docs/WhySwiftF...

johnnycerberus 6y ago

I really don't see how Scala is not fit for this. Swift for Tensorflow was mainly revolving around the Differentiable protocol. There is a paper published in 2019 from Purdue University where they show how AD can be combined with multi-stage programming in Scala. [1] DeepLearning.scala that does compile-time reverse AD. [2] Compute.scala for operator fusion at runtime (JITifying) [3] Here [4] is a Pre-SIP to add reifiable control flow to Scala compiler so AD and operator fusion could happen at compile-time. Moreover, Kotlin also has a library that does type-safe AD. [5] nVidia and Oracle want better interopabiltiy between CUDA and GraalVM through grCuda. [6] [1] Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator - https://arxiv.org/pdf/1803.10228.pdf https://arxiv.org/pdf/1803.10228.pdf [2] https://github.com/ThoughtWorksInc/DeepLearning.scala https://github.com/ThoughtWorksInc/DeepLearning.scala [3] https://github.com/ThoughtWorksInc/Compute.scala https://github.com/ThoughtWorksInc/Compute.scala [4] https://contributors.scala-lang.org/t/pre-sip-for-with-control-flow-an-alternative-to-scala-async-scala-continuations-and-scala-virtualized/4423 https://contributors.scala-lang.org/t/pre-sip-for-with-contr... [5] https://github.com/breandan/kotlingrad https://github.com/breandan/kotlingrad [6] https://developer.nvidia.com/blog/grcuda-a-polyglot-language-binding-for-cuda-in-graalvm/ https://developer.nvidia.com/blog/grcuda-a-polyglot-language...

sanderjd 6y ago
Neat! This may have not been well known when they kicked off the project and wrote their reasoning. Here is what they had to say about Scala at the time of the document linked up-thread[0]: "Java / C# / Scala (and other OOP languages with pervasive dynamic dispatch): These languages share most of the static analysis problems as Python: their primary abstraction features (classes and interfaces) are built on highly dynamic constructs, which means that static analysis of Tensor operations depends on "best effort" techniques like alias analysis and class hierarchy analysis. Further, because they are pervasively reference-based, it is difficult to reliably disambiguate pointer aliases." If they were wrong about that, or if the state of the art has progressed in the meantime, that's great! You may well be right that Scala would be a good / the best choice if they started the project today. [0]: https://github.com/tensorflow/swift/blob/main/docs/WhySwiftForTensorFlow.md#which-languages-fit-our-project-requirements https://github.com/tensorflow/swift/blob/main/docs/WhySwiftF...