43 ms·

There's a bit of general dev snobbery to assume that data scientists can't code, or don't want to learn to code, or are able to understand so many other things

by jstx1 3y ago

There's a bit of general dev snobbery to assume that data scientists can't code, or don't want to learn to code, or are able to understand so many other things but not a compiled language or a static type system.

chongli 3y ago
It's not about ability to code, willingness to learn, or ability to understand. It's about friction. When you're iterating quickly and you're thinking about complicated mathematics and ideas, a complicated type system just adds a ton of friction. This is especially annoying when you know what you're doing is correct but the type system is unsatisfied and demands proof.
- throwawaymaths 3y ago
  the place in DL/ML where typing could be the most useful is to keep you from trying to multiply differently sized tensors together. So the irony is that the one place where typing WOULD help DL/ML, is the place where rust is weak in its typing system (dependent types and const generics)
  - dgacmu 3y ago
    I keep checking the progress on dfdx for this reason. It does what I (and, I assume from context, you) want: Provides static checking of tensor shapes. Which is fantastic. Not quite as much inference as I'd like but I love getting compile-time errors that I forgot to transpose before a matmul. It depends on the generic_const_exprs feature which is still, to quote, "highly experimental": https://github.com/rust-lang/rust/issues/76560 https://github.com/rust-lang/rust/issues/76560 Definitely not for production use, but it gives a flavor for where things can head in the medium term, and it's .. it's nice. You could imagine future type support allowing even more inference for some intermediate shapes, of course, but even what it has now is really nice. Like this cute little convnet example: https://github.com/coreylowman/dfdx/blob/main/examples/nightly-conv-net.rs https://github.com/coreylowman/dfdx/blob/main/examples/night...

by jstx1 3y ago

chongli 3y ago

It's not about ability to code, willingness to learn, or ability to understand. It's about friction. When you're iterating quickly and you're thinking about complicated mathematics and ideas, a complicated type system just adds a ton of friction. This is especially annoying when you know what you're doing is correct but the type system is unsatisfied and demands proof.

throwawaymaths 3y ago
the place in DL/ML where typing could be the most useful is to keep you from trying to multiply differently sized tensors together. So the irony is that the one place where typing WOULD help DL/ML, is the place where rust is weak in its typing system (dependent types and const generics)
- dgacmu 3y ago
  I keep checking the progress on dfdx for this reason. It does what I (and, I assume from context, you) want: Provides static checking of tensor shapes. Which is fantastic. Not quite as much inference as I'd like but I love getting compile-time errors that I forgot to transpose before a matmul. It depends on the generic_const_exprs feature which is still, to quote, "highly experimental": https://github.com/rust-lang/rust/issues/76560 https://github.com/rust-lang/rust/issues/76560 Definitely not for production use, but it gives a flavor for where things can head in the medium term, and it's .. it's nice. You could imagine future type support allowing even more inference for some intermediate shapes, of course, but even what it has now is really nice. Like this cute little convnet example: https://github.com/coreylowman/dfdx/blob/main/examples/nightly-conv-net.rs https://github.com/coreylowman/dfdx/blob/main/examples/night...