43 ms·

Only if the library is not written in Julia, yes. The point here is that Julia is efficient enough that deep learning libraries can be written in pure Julia, no

Only if the library is not written in Julia, yes. The point here is that Julia is efficient enough that deep learning libraries can be written in pure Julia, not C/C++/Fortran.

nl 6y ago
And yet... no widely used DL library is in Julia, and they are all in Python. It's kind of a silly point to try to score: "it's possible to write an efficient deep learning library in Julia (although no one has done it yet), and yes, you can do the same in Numpy in Python, and XLA in Python will outpeform it, but someone else wrote some C/C++ there to make that possible!" You are much more likely to want to write CUDA kernels (in C!) than you are to write C framework code to interface with Python for machine learning. The person is looking to "get into ML". I've been working as a professional ML developer for 6 years, and I've never written any C or C++ for it.
- BadInformatics 6y ago
  > although no one has done it yet This may be generally true (though the benchmarks I've seen show Knet.jl and sometimes Flux.jl on par with TF/PyTorch with a single machine + single GPU), but there are definitely domains where it is categorically not. The most prominent one is neural *DEs, where the SciML [1] ecosystem has SOTA performance. You can really see Python/C++-based frameworks struggle here because they have slow "glue code" and don't (one could argue can't effectively) optimize for latency. That's not a problem for most CV models and transformers, but really stunts research into more dynamic approaches. [1] https://sciml.ai/ https://sciml.ai/
  - nl 6y ago
    This sounds more like it's a new field. I started looking for benchmarks (because it sounds like the kind of thing JAX would do well) and the very first link I clicked included: Wraps for common C/Fortran methods like Sundials and Hairer's radau which is exactly what was claimed wasn't needed.
    - amkkma 6y ago
      Those aren't needed, but they are options. "he templated implementation allows arbitrary array and number types to be compatible, giving compatibility with arbitrary precision floating point numbers, GPU-based computations, unit-checked arithmetic, and other features. " Those are exactly the things wrapping fortran and c won't give you
    - ChrisRackauckas 6y ago
      >I started looking for benchmarks (because it sounds like the kind of thing JAX would do well) Currently Jax has an in-progress stiff ODE solver that's about 200x slower than SciPy https://github.com/google/jax/issues/3686#issuecomment-657090547 https://github.com/google/jax/issues/3686#issuecomment-65709... and SciPy (with JIT) is about 50x-100x slower than the pure Julia methods https://benchmarks.sciml.ai/html/MultiLanguage/wrapper_packages.html https://benchmarks.sciml.ai/html/MultiLanguage/wrapper_packa... so Jax has more than a little bit of a way to go.

Only if the library is not written in Julia, yes. The point here is that Julia is efficient enough that deep learning libraries can be written in pure Julia, not C/C++/Fortran.

nl 6y ago

And yet... no widely used DL library is in Julia, and they are all in Python. It's kind of a silly point to try to score: "it's possible to write an efficient deep learning library in Julia (although no one has done it yet), and yes, you can do the same in Numpy in Python, and XLA in Python will outpeform it, but someone else wrote some C/C++ there to make that possible!" You are much more likely to want to write CUDA kernels (in C!) than you are to write C framework code to interface with Python for machine learning. The person is looking to "get into ML". I've been working as a professional ML developer for 6 years, and I've never written any C or C++ for it.

BadInformatics 6y ago
> although no one has done it yet This may be generally true (though the benchmarks I've seen show Knet.jl and sometimes Flux.jl on par with TF/PyTorch with a single machine + single GPU), but there are definitely domains where it is categorically not. The most prominent one is neural *DEs, where the SciML [1] ecosystem has SOTA performance. You can really see Python/C++-based frameworks struggle here because they have slow "glue code" and don't (one could argue can't effectively) optimize for latency. That's not a problem for most CV models and transformers, but really stunts research into more dynamic approaches. [1] https://sciml.ai/ https://sciml.ai/
- nl 6y ago
  This sounds more like it's a new field. I started looking for benchmarks (because it sounds like the kind of thing JAX would do well) and the very first link I clicked included: Wraps for common C/Fortran methods like Sundials and Hairer's radau which is exactly what was claimed wasn't needed.
  - amkkma 6y ago
    Those aren't needed, but they are options. "he templated implementation allows arbitrary array and number types to be compatible, giving compatibility with arbitrary precision floating point numbers, GPU-based computations, unit-checked arithmetic, and other features. " Those are exactly the things wrapping fortran and c won't give you
  - ChrisRackauckas 6y ago
    >I started looking for benchmarks (because it sounds like the kind of thing JAX would do well) Currently Jax has an in-progress stiff ODE solver that's about 200x slower than SciPy https://github.com/google/jax/issues/3686#issuecomment-657090547 https://github.com/google/jax/issues/3686#issuecomment-65709... and SciPy (with JIT) is about 50x-100x slower than the pure Julia methods https://benchmarks.sciml.ai/html/MultiLanguage/wrapper_packages.html https://benchmarks.sciml.ai/html/MultiLanguage/wrapper_packa... so Jax has more than a little bit of a way to go.