5 ms·

> With numpy, you might have to be writing Python FTFY cf Numba http://numba.pydata.org http://numba.pydata.org

krastanov 6y ago
Numba is great (as well as Jax, which is even better), but its limitations are drastic compared to Julia. So much of what is great about python (e.g., compossibility, introspection, simplicity) is lost with these libraries, while Julia still retains them.
- marmaduke 6y ago
  The main thing missing from Numba is user defined structures, the jit classes are a performance penalty usually. That said, in terms of composability you can jit over the closure to achieve a lot of what you might want, e.g. def make_loop(f): @jit def fn(x): for i in range(x.shape[0]): x[i] = f(x[i]) return fn for any jit'd function f will be just as fast as if you had inlined the body of f. For introspection and simplicity, I think, in high performance, you simply have to choose two of fast, simple and generic. Julia clearly chooses fast and generic.
  - krastanov 6y ago
    Could you elaborate on why you are saying julia is not as simple? Certainly, there are performance tips one needs to be aware of, but that is the case with python too. I have used python for relatively high-performance numerics for research work for the last decade, and now that I am exploring Julia, it certainly addresses all of the concerns I mentioned above with code which is at least as legible/simple and much more introspective. It has its warts, but those warts are on the roadmap and I have seen significant improvement between v1.3 and v1.6 (e.g. compilation latency and debuggability).
    - marmaduke 6y ago
      Compiler errors can be on par with heavily templates C++, with similarly difficult to read code bases. It’s of course easier to work with than C++ but not the panacea that seems to be implied in a lot of the comments. In terms of deployment, you essentially have to have Julia installed wherever you want to run, which frequently ok, and if not there’s always PackageCompiler except it’s not that easy to get a working shared lib. There are other things I find complex but probably because I’ve used Python too long so Julia is different etc. I think Julia is a great choice for HPC generally except for challenging SIMD codes where it’s hard to vectorize without an explicit ILP model.
- datanecdote 6y ago
  How does Jax lose composability or introspection?
  - krastanov 6y ago
    E.g. jax does not autodifferentiate anything that is not jax (scipy ode solvers, special functions, image processing libraries, special number types (mpmath), domain-specific libraries). Compare that to Zygote.jl
    - datanecdote 6y ago
      It is true that Jax cannot differentiate through C code. But it can differentiate through python code that was written to accept Numpy.
      - amkkma 6y ago
        Which is extremely limited compared to Zygote which can do custom types, dicts, custom arrays, complex type system and multiple dispatch uses etc
        datanecdote 6y ago
        Try reading the docs before making sweeping negative comments about what a piece of software can and cannot do. https://jax.readthedocs.io/en/latest/notebooks/autodiff_cookbook.html https://jax.readthedocs.io/en/latest/notebooks/autodiff_cook...
        krastanov 6y ago
        Are you talking about "Differentiating with respect to nested lists, tuples, and dicts" from that page? The comment to which you are responding covers quite a bit more. The jax documentation specifically says "standard Python containers". Zygote.jl and other less stable Julia auto-diff libraries go far beyond the built-ins and can work with structures defined by packages never designed to be used with automatic differentiation. Of course, there are limitations, but quite a bit less severe than the one in jax (and again, I am saying this while being a big jax fan).

Numba is great (as well as Jax, which is even better), but its limitations are drastic compared to Julia. So much of what is great about python (e.g., compossibility, introspection, simplicity) is lost with these libraries, while Julia still retains them.

marmaduke 6y ago
The main thing missing from Numba is user defined structures, the jit classes are a performance penalty usually. That said, in terms of composability you can jit over the closure to achieve a lot of what you might want, e.g. def make_loop(f): @jit def fn(x): for i in range(x.shape[0]): x[i] = f(x[i]) return fn for any jit'd function f will be just as fast as if you had inlined the body of f. For introspection and simplicity, I think, in high performance, you simply have to choose two of fast, simple and generic. Julia clearly chooses fast and generic.
- krastanov 6y ago
  Could you elaborate on why you are saying julia is not as simple? Certainly, there are performance tips one needs to be aware of, but that is the case with python too. I have used python for relatively high-performance numerics for research work for the last decade, and now that I am exploring Julia, it certainly addresses all of the concerns I mentioned above with code which is at least as legible/simple and much more introspective. It has its warts, but those warts are on the roadmap and I have seen significant improvement between v1.3 and v1.6 (e.g. compilation latency and debuggability).
  - marmaduke 6y ago
    Compiler errors can be on par with heavily templates C++, with similarly difficult to read code bases. It’s of course easier to work with than C++ but not the panacea that seems to be implied in a lot of the comments. In terms of deployment, you essentially have to have Julia installed wherever you want to run, which frequently ok, and if not there’s always PackageCompiler except it’s not that easy to get a working shared lib. There are other things I find complex but probably because I’ve used Python too long so Julia is different etc. I think Julia is a great choice for HPC generally except for challenging SIMD codes where it’s hard to vectorize without an explicit ILP model.
datanecdote 6y ago
How does Jax lose composability or introspection?
- krastanov 6y ago
  E.g. jax does not autodifferentiate anything that is not jax (scipy ode solvers, special functions, image processing libraries, special number types (mpmath), domain-specific libraries). Compare that to Zygote.jl
  - datanecdote 6y ago
    It is true that Jax cannot differentiate through C code. But it can differentiate through python code that was written to accept Numpy.
    - amkkma 6y ago
      Which is extremely limited compared to Zygote which can do custom types, dicts, custom arrays, complex type system and multiple dispatch uses etc
      - datanecdote 6y ago
        Try reading the docs before making sweeping negative comments about what a piece of software can and cannot do. https://jax.readthedocs.io/en/latest/notebooks/autodiff_cookbook.html https://jax.readthedocs.io/en/latest/notebooks/autodiff_cook...
        krastanov 6y ago
        Are you talking about "Differentiating with respect to nested lists, tuples, and dicts" from that page? The comment to which you are responding covers quite a bit more. The jax documentation specifically says "standard Python containers". Zygote.jl and other less stable Julia auto-diff libraries go far beyond the built-ins and can work with structures defined by packages never designed to be used with automatic differentiation. Of course, there are limitations, but quite a bit less severe than the one in jax (and again, I am saying this while being a big jax fan).