3 ms·

Only thing "interesting" to me there would be the automatic differentiation bugs ...but is there any argument as to them being the fault of the language, instea

Only thing "interesting" to me there would be the automatic differentiation bugs ...but is there any argument as to them being the fault of the language, instead of just poor engineering from the library developers' part? I mean, one can't expect all algorithms to work correctly with all datatypes just because the compiler allows that code to run ...you write tests and guarantee numerical stability for a small subset of types you can actually do it for, and then it's the code's consumers' job to ensure it work with types it's not documented to work and such, no? ...Julia is quite a dynamic language, JITed or what not, its semantics are closer to Python and Lisp than to Rust or Haskell ...maybe don't expect guarantees that aren't there and just code more defensively when making libraries others depends on? Probably the Python + C(++) ecosystems works better bc their devs know they are working in loose, dynamic and weekly typed shoot-your-foot-off type languages and just take action and code defensively and test things properly, whereas Julia devs expect the language to give them guarantees that aren't there.

Q6T46nT668w6i3m 4y ago
I think the author addresses this. It’s a Catch-22. If you restrict use to a small subset of types you’re undermining one of Julia’s best features. As someone who has been writing a lot of numerical analysis code recently, I would absolutely love a type system that could describe and enforce numerical stability traits.
- catchclose8919 4y ago
  > a type system that could describe and enforce numerical stability traits Wow, that sounds cool! have your reasearched if anyone has done anything in this are? how would you even start to approach the problem? Do you think it has any change of being done without massive sacrifices to performance?
- one-more-minute 4y ago
  Right. It's important to remember that tools like JAX and PyTorch have total control over the numerical libraries they are differentiating, and have freedom to impose whatever semantics, rules and restrictions are convenient (immutability and referential transparency in JAX, for example). Seemingly small decisions in an existing language and library can have a big impact on the feasibility and practicality of AD.
  - dklend122 4y ago
    That's exactly where Dex might improve over Julia, with language level control over mutability and effect handlers and array access safety ... time will tell. So packages just use those features Maybe it will hit the right trade off, or maybe Julia will adopt similar language level tools, but adjusted for dynamic semantics. Is that even possible?
- jstrong 4y ago
  in rust code, I like using `debug_assert!` to represent numerical expectations/assumptions of the implementation. later if I have a problem, I can turn on debug assertions and I will get a bunch of additional checks. but I can also turn them off and not pay for them all the time.

Q6T46nT668w6i3m 4y ago

I think the author addresses this. It’s a Catch-22. If you restrict use to a small subset of types you’re undermining one of Julia’s best features. As someone who has been writing a lot of numerical analysis code recently, I would absolutely love a type system that could describe and enforce numerical stability traits.

catchclose8919 4y ago
> a type system that could describe and enforce numerical stability traits Wow, that sounds cool! have your reasearched if anyone has done anything in this are? how would you even start to approach the problem? Do you think it has any change of being done without massive sacrifices to performance?
one-more-minute 4y ago
Right. It's important to remember that tools like JAX and PyTorch have total control over the numerical libraries they are differentiating, and have freedom to impose whatever semantics, rules and restrictions are convenient (immutability and referential transparency in JAX, for example). Seemingly small decisions in an existing language and library can have a big impact on the feasibility and practicality of AD.
- dklend122 4y ago
  That's exactly where Dex might improve over Julia, with language level control over mutability and effect handlers and array access safety ... time will tell. So packages just use those features Maybe it will hit the right trade off, or maybe Julia will adopt similar language level tools, but adjusted for dynamic semantics. Is that even possible?
jstrong 4y ago
in rust code, I like using `debug_assert!` to represent numerical expectations/assumptions of the implementation. later if I have a problem, I can turn on debug assertions and I will get a bunch of additional checks. but I can also turn them off and not pay for them all the time.