3 ms·

I remember their terrible article trying to justify why Swift was the right choice out of a load of more appropriate languages. I'm still annoyed they didn't go

I remember their terrible article trying to justify why Swift was the right choice out of a load of more appropriate languages. I'm still annoyed they didn't go with Julia.

cglong 6y ago
Source? Curious to read that :D
- _Wintermute 6y ago
  Here you are: https://github.com/tensorflow/swift/blob/main/docs/WhySwiftForTensorFlow.md#which-languages-fit-our-project-requirements https://github.com/tensorflow/swift/blob/main/docs/WhySwiftF...
  - sanderjd 6y ago
    I always figured they would have chosen rust if it weren't for Lattner's sway (real or perceived) in the Swift compiler development community. One underestimated advantage of rust over swift in my view is that it has interested practitioners across a much more diverse range of problems spaces. I think it was always going to be hard to convince people to use a language for for ML when most of its practitioners are solely mobile developers. But people already used rust for lots of different kinds of data processing heavy use cases.
    - johnnycerberus 6y ago
      You can't be serious. Rust would just replace C++ in the current formula, you can't use Rust as if it was Python. ML people don't need to waste time proving memory properties at compile time, cmon! Indeed, there were alternatives. Two already come to my mind, one is Scala (which syntax-wise is strikingly similar to Python) and Julia. Both having features like type-safety, macros, JIT, FFIs, etc.
      - sanderjd 6y ago
        None of this is less true for swift. The whole idea is that there may be an advantage to better static analysis that overcomes the perceived downside of using a language with a strict compiler and lots of rules. That idea may or may not be worthwhile, but it is equally applicable to swift and rust. Julia I agree but Scala strikes me as a bit of a non sequitur. It is much more complex and foot-gunny (and thus difficult for scientists who only know python to get right) than either swift or rust, and much more difficult for static analyzers.
        pjmlp 6y ago
        Scala already has a foothold on the same research domain due to large data frameworks like Hadoop and Spark, with NVidia caring for polyglot support on CUDA, including JVM based languages.
        sanderjd 6y ago
        That is true, and certainly relevant, but I believe it is also true that Scala does not meet the criteria for what that were trying to do, because it is not easy (perhaps not possible) to do the kind of static analysis necessary to achieve their goals. For a generic "what language with a good type system should we use with ML?" project, I would totally agree with you that Scala is an obvious choice. But this project was attempting a more specific thing, for which Scala is a less good fit.
        johnnycerberus 6y ago
        I really don't see how Scala is not fit for this. Swift for Tensorflow was mainly revolving around the Differentiable protocol. There is a paper published in 2019 from Purdue University where they show how AD can be combined with multi-stage programming in Scala. [1] DeepLearning.scala that does compile-time reverse AD. [2] Compute.scala for operator fusion at runtime (JITifying) [3] Here [4] is a Pre-SIP to add reifiable control flow to Scala compiler so AD and operator fusion could happen at compile-time. Moreover, Kotlin also has a library that does type-safe AD. [5] nVidia and Oracle want better interopabiltiy between CUDA and GraalVM through grCuda. [6] [1] Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator - https://arxiv.org/pdf/1803.10228.pdf https://arxiv.org/pdf/1803.10228.pdf [2] https://github.com/ThoughtWorksInc/DeepLearning.scala https://github.com/ThoughtWorksInc/DeepLearning.scala [3] https://github.com/ThoughtWorksInc/Compute.scala https://github.com/ThoughtWorksInc/Compute.scala [4] https://contributors.scala-lang.org/t/pre-sip-for-with-control-flow-an-alternative-to-scala-async-scala-continuations-and-scala-virtualized/4423 https://contributors.scala-lang.org/t/pre-sip-for-with-contr... [5] https://github.com/breandan/kotlingrad https://github.com/breandan/kotlingrad [6] https://developer.nvidia.com/blog/grcuda-a-polyglot-language-binding-for-cuda-in-graalvm/ https://developer.nvidia.com/blog/grcuda-a-polyglot-language...

I remember their terrible article trying to justify why Swift was the right choice out of a load of more appropriate languages. I'm still annoyed they didn't go with Julia.

cglong 6y ago

Source? Curious to read that :D

_Wintermute 6y ago
Here you are: https://github.com/tensorflow/swift/blob/main/docs/WhySwiftForTensorFlow.md#which-languages-fit-our-project-requirements https://github.com/tensorflow/swift/blob/main/docs/WhySwiftF...
- sanderjd 6y ago
  I always figured they would have chosen rust if it weren't for Lattner's sway (real or perceived) in the Swift compiler development community. One underestimated advantage of rust over swift in my view is that it has interested practitioners across a much more diverse range of problems spaces. I think it was always going to be hard to convince people to use a language for for ML when most of its practitioners are solely mobile developers. But people already used rust for lots of different kinds of data processing heavy use cases.
  - johnnycerberus 6y ago
    You can't be serious. Rust would just replace C++ in the current formula, you can't use Rust as if it was Python. ML people don't need to waste time proving memory properties at compile time, cmon! Indeed, there were alternatives. Two already come to my mind, one is Scala (which syntax-wise is strikingly similar to Python) and Julia. Both having features like type-safety, macros, JIT, FFIs, etc.
    - sanderjd 6y ago
      None of this is less true for swift. The whole idea is that there may be an advantage to better static analysis that overcomes the perceived downside of using a language with a strict compiler and lots of rules. That idea may or may not be worthwhile, but it is equally applicable to swift and rust. Julia I agree but Scala strikes me as a bit of a non sequitur. It is much more complex and foot-gunny (and thus difficult for scientists who only know python to get right) than either swift or rust, and much more difficult for static analyzers.
      - pjmlp 6y ago
        Scala already has a foothold on the same research domain due to large data frameworks like Hadoop and Spark, with NVidia caring for polyglot support on CUDA, including JVM based languages.
        sanderjd 6y ago
        That is true, and certainly relevant, but I believe it is also true that Scala does not meet the criteria for what that were trying to do, because it is not easy (perhaps not possible) to do the kind of static analysis necessary to achieve their goals. For a generic "what language with a good type system should we use with ML?" project, I would totally agree with you that Scala is an obvious choice. But this project was attempting a more specific thing, for which Scala is a less good fit.
        johnnycerberus 6y ago
        I really don't see how Scala is not fit for this. Swift for Tensorflow was mainly revolving around the Differentiable protocol. There is a paper published in 2019 from Purdue University where they show how AD can be combined with multi-stage programming in Scala. [1] DeepLearning.scala that does compile-time reverse AD. [2] Compute.scala for operator fusion at runtime (JITifying) [3] Here [4] is a Pre-SIP to add reifiable control flow to Scala compiler so AD and operator fusion could happen at compile-time. Moreover, Kotlin also has a library that does type-safe AD. [5] nVidia and Oracle want better interopabiltiy between CUDA and GraalVM through grCuda. [6] [1] Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator - https://arxiv.org/pdf/1803.10228.pdf https://arxiv.org/pdf/1803.10228.pdf [2] https://github.com/ThoughtWorksInc/DeepLearning.scala https://github.com/ThoughtWorksInc/DeepLearning.scala [3] https://github.com/ThoughtWorksInc/Compute.scala https://github.com/ThoughtWorksInc/Compute.scala [4] https://contributors.scala-lang.org/t/pre-sip-for-with-control-flow-an-alternative-to-scala-async-scala-continuations-and-scala-virtualized/4423 https://contributors.scala-lang.org/t/pre-sip-for-with-contr... [5] https://github.com/breandan/kotlingrad https://github.com/breandan/kotlingrad [6] https://developer.nvidia.com/blog/grcuda-a-polyglot-language-binding-for-cuda-in-graalvm/ https://developer.nvidia.com/blog/grcuda-a-polyglot-language...