4 ms·

So much bashing on static typing on deep learning:) Does any one from Google can explain the benefit since you guys are working on swift in tensorflow https://

So much bashing on static typing on deep learning:) Does any one from Google can explain the benefit since you guys are working on swift in tensorflow https://medium.com/tensorflow/introducing-swift-for-tensorflow-b75722c58df0 https://medium.com/tensorflow/introducing-swift-for-tensorfl...

shoyer 8y ago
Static typing for catching errors is only a small part of the vision for Swift on TensorFlow. The real advantage of static typing is that it enables the compiler to reason to about your code, e.g., to automatically rewrite it for a hardware accelerator with guaranteed correct semantics: https://github.com/tensorflow/swift/blob/master/docs/DesignOverview.md https://github.com/tensorflow/swift/blob/master/docs/DesignO... This is obviously possible in Python as well (e.g., see Numba) but is clearly has additional challenges: https://github.com/tensorflow/swift/blob/master/docs/WhySwiftForTensorFlow.md https://github.com/tensorflow/swift/blob/master/docs/WhySwif... (I work at Google, but not on the TensorFlow team.)
- yunfeng_lin 8y ago
  Thanks! that's a very interesting idea! Definitely worth exploring. Not sure it is my false sense. It seems that many python deep learning people are so proud of their chose, it is difficult to convince them.
- KenoFischer 8y ago
  Static typing has very little to do with what the compiler can say about your code. You can have dynamic languages with very strong type systems and semantics as well as static languages with weak semantics. The only difference between static and dynamic languages is whether the compiler enforces completeness of the analysis or not.
  - yunfeng_lin 8y ago
    I am not sure I agree with you. You do need compile time type to generate efficient hardware accelerated code. Python has strong type, but that is only available at run time, which is not useful to generate code. But now python also have optional type. this might be utilized in generating more efficient code though
    - KenoFischer 8y ago
      Look at e.g. Julia for a dynamic system with a strong type system that allows the compiler to reason about the code without forcing completeness.
      - yunfeng_lin 8y ago
        Julia has optional typing which is static as well
        KenoFischer 8y ago
        As always in these sorts of discussions, it depends how you define your terminology. However, by most definitions of static typing, Julia's type system is not static. The julia type system is very much a property of the runtime language and behaves as such. It is quite strong, true, but still not statically enforced as you would expect from a static language. In particular, (to the extent that you can identify one), you never get any sort of compile-time type errors in julia.
- seanmcdirmid 8y ago
  Static typing enables some optimizations, but not as many as we’d hope given the inexpressiveness of most type systems. The real advantage of static typing is code completion, which allows us to forget the nuances of our library naming schemes. TypeScript is so awesome in this regard, being neither sound nor used for optimizations, but still being very useful.

shoyer 8y ago

Static typing for catching errors is only a small part of the vision for Swift on TensorFlow. The real advantage of static typing is that it enables the compiler to reason to about your code, e.g., to automatically rewrite it for a hardware accelerator with guaranteed correct semantics: https://github.com/tensorflow/swift/blob/master/docs/DesignOverview.md https://github.com/tensorflow/swift/blob/master/docs/DesignO... This is obviously possible in Python as well (e.g., see Numba) but is clearly has additional challenges: https://github.com/tensorflow/swift/blob/master/docs/WhySwiftForTensorFlow.md https://github.com/tensorflow/swift/blob/master/docs/WhySwif... (I work at Google, but not on the TensorFlow team.)

yunfeng_lin 8y ago
Thanks! that's a very interesting idea! Definitely worth exploring. Not sure it is my false sense. It seems that many python deep learning people are so proud of their chose, it is difficult to convince them.
KenoFischer 8y ago
Static typing has very little to do with what the compiler can say about your code. You can have dynamic languages with very strong type systems and semantics as well as static languages with weak semantics. The only difference between static and dynamic languages is whether the compiler enforces completeness of the analysis or not.
- yunfeng_lin 8y ago
  I am not sure I agree with you. You do need compile time type to generate efficient hardware accelerated code. Python has strong type, but that is only available at run time, which is not useful to generate code. But now python also have optional type. this might be utilized in generating more efficient code though
  - KenoFischer 8y ago
    Look at e.g. Julia for a dynamic system with a strong type system that allows the compiler to reason about the code without forcing completeness.
    - yunfeng_lin 8y ago
      Julia has optional typing which is static as well
      - KenoFischer 8y ago
        As always in these sorts of discussions, it depends how you define your terminology. However, by most definitions of static typing, Julia's type system is not static. The julia type system is very much a property of the runtime language and behaves as such. It is quite strong, true, but still not statically enforced as you would expect from a static language. In particular, (to the extent that you can identify one), you never get any sort of compile-time type errors in julia.
seanmcdirmid 8y ago
Static typing enables some optimizations, but not as many as we’d hope given the inexpressiveness of most type systems. The real advantage of static typing is code completion, which allows us to forget the nuances of our library naming schemes. TypeScript is so awesome in this regard, being neither sound nor used for optimizations, but still being very useful.