3 ms·

> Having a half way decent type system solves so many problems What problems does it solve for you? When building ML applications I mean. Because that's what S

> Having a half way decent type system solves so many problems What problems does it solve for you? When building ML applications I mean. Because that's what S4TF was supposed to be. ML researchers produce models that are used by ML applications for a particular purpose. The code to train the model _could_ be statically typed, sure, but I really don't see what the improvement would be. It would be more verbose, less readable, have significantly more noise. Just try reading some TensorFlow C++ code equivalents of their Python API. From the "WhySwiftForTensorFlow.md" document (https://github.com/tensorflow/swift/blob/main/docs/WhySwiftForTensorFlow.md#static-vs-dynamic-type-systems https://github.com/tensorflow/swift/blob/main/docs/WhySwiftF...): .. static types: * can catch bugs at compile time instead of runtime. People get very frustrated when a silly type error brings down a long training run hours into it. * directly improve tooling experience like code completion/intellisense, jump to definition, etc. * make it easy to know what operations are Tensor operations. > can catch bugs at compile time instead of runtime. OK, but what bugs? For most use cases of TensorFlow that I've seen, if there is a type error somewhere, you will catch it long before it hits production, and if you only catch it 100hr into training, your training code is either non-deterministic or there's some other issue that is not type related. Unless we're talking about dependent types a-la Idris, I don't see how Swift's type system can help catch these kinds of bugs. > directly improve tooling experience like code completion/intellisense Sure, but type-hints provide the same improved experience with language servers like PyLance [0]. Not a reason to jump to a new language. > make it easy to know what operations are Tensor operations. Easy. Anything that starts with "tf." is a tensor operation. Yes, I'm being a bit trite, but I was never convinced by the premise of S4TF, because I've never heard an ML researcher or engineer say "static typing will fix my problems". [0]: https://devblogs.microsoft.com/python/announcing-pylance-fast-feature-rich-language-support-for-python-in-visual-studio-code/ https://devblogs.microsoft.com/python/announcing-pylance-fas...

skohan 6y ago
My guess is that's because ML researchers typically have not spent much time with statically typed languages. I was also once convinced that static typing was not so valuable - when I was working a lot with js, python and ruby, but the more time I spend with static typing the more I like it. There is an "activation energy" to overcome with static typing: when you first start it feels like a tedious burden to have to add type annotations, when your python code ran just fine without them. But in my experience, with a little bit of practice the type system starts to feel completely effortless, and it saves you from all kinds of strange runtime issues you run into when nobody is checking your types. Also type systems encode a lot of intention into the code. If I look at a function signature in Swift, I know exactly what has to be passed in, and exactly what will come out. In python, I have to rely on documentation, or even maybe I have to dig into the source to see what this function expects. In my experience, the real difference is seen when you come back to a weakly-typed language after working with types for an extended period. Returning to Python after spending time with Rust or Swift feels like walking down the street with one of my senses missing. There are whole categories of dangers I now have to watch out for which the compiler would have caught for me automatically. It is not pleasant to go back.
- marcinzm 6y ago
  I've done a lot of statically typed code and have gone to conference where Haskell was on the keynote. I've done ML work in both dynamically and statically typed languages. For the ML work I do I don't honestly see the advantage. Most of the complexity is in the matrix and data operations which existing type systems don't check. And the type annotations and IDE support is good enough to catch most of the other boring errors. In my experience most deep learning ML models are also not particularly complicated code from a code perspective.
- swagonomixxx 6y ago
  I am totally with you on statically typed vs. dynamically typed languages _in general_. I'm not denying that there are major advantages to using one for most kinds of programs. But I just don't see a benefit for ML. You can easily find out if you like it or not, you can use the C++ API for TF/PyTorch and you'll have a statically typed program running your model for you. Sure, it's not Haskell, but C++ is statically typed, and although it's not the most popular or loved language on HN, you can use it to get stuff done.
  - pjmlp 6y ago
    And with C++20 relatively pleasing to use.

skohan 6y ago

My guess is that's because ML researchers typically have not spent much time with statically typed languages. I was also once convinced that static typing was not so valuable - when I was working a lot with js, python and ruby, but the more time I spend with static typing the more I like it. There is an "activation energy" to overcome with static typing: when you first start it feels like a tedious burden to have to add type annotations, when your python code ran just fine without them. But in my experience, with a little bit of practice the type system starts to feel completely effortless, and it saves you from all kinds of strange runtime issues you run into when nobody is checking your types. Also type systems encode a lot of intention into the code. If I look at a function signature in Swift, I know exactly what has to be passed in, and exactly what will come out. In python, I have to rely on documentation, or even maybe I have to dig into the source to see what this function expects. In my experience, the real difference is seen when you come back to a weakly-typed language after working with types for an extended period. Returning to Python after spending time with Rust or Swift feels like walking down the street with one of my senses missing. There are whole categories of dangers I now have to watch out for which the compiler would have caught for me automatically. It is not pleasant to go back.

marcinzm 6y ago
I've done a lot of statically typed code and have gone to conference where Haskell was on the keynote. I've done ML work in both dynamically and statically typed languages. For the ML work I do I don't honestly see the advantage. Most of the complexity is in the matrix and data operations which existing type systems don't check. And the type annotations and IDE support is good enough to catch most of the other boring errors. In my experience most deep learning ML models are also not particularly complicated code from a code perspective.
swagonomixxx 6y ago
I am totally with you on statically typed vs. dynamically typed languages _in general_. I'm not denying that there are major advantages to using one for most kinds of programs. But I just don't see a benefit for ML. You can easily find out if you like it or not, you can use the C++ API for TF/PyTorch and you'll have a statically typed program running your model for you. Sure, it's not Haskell, but C++ is statically typed, and although it's not the most popular or loved language on HN, you can use it to get stuff done.
- pjmlp 6y ago
  And with C++20 relatively pleasing to use.