4 ms·

Julia is looking more and more like a viable alternative to Python's data science stack.

by atulvi 3y ago

isoprophlex 3y ago
I want this to become reality. Are there any IDEs for Julia that come close to the ergonomics and feature set of Pycharm?
- cbkeller 3y ago
  The most IDE-ish experience is probably currently the vscode plugin. I haven’t used pycharm specifically for comparison, but I suspect “probably not” as compared to a polished paid IDE in general. There’s progress being made though.
nextos 3y ago
There's still one major bottleneck. There's no TF / PyTorch / JAX replacement. For small things, interop with R is good, so one can defer things to R packages and get access to a great set of functionality.
- ssivark 3y ago
  > There's no TF / PyTorch / JAX replacement What do you mean by that? AFAICT for the most part Julia is already like the best that Jax could ever hope to be — since there is language level support for JIT compilation, gradients, etc — given how well libraries compose without having to pick different incompatible subsets of the language.
  - the_svd_doctor 3y ago
    I mean, yes but also no. JAX will JIT on accelerators OOTB and has outstanding (imho) support for multi node. Julia does not have that OOTB AFAIK. Now I agree that Julia, with its JIT, is well positioned to do all this. But I don’t think it does.
    - ssivark 3y ago
      Ah, I see. There’s XLA.jl for TPUs and a bunch of stuff from the JuliaGPU organization for GPUs — I wonder how they compare with TF/PyTorch/JAX. I’m curious to get perspectives on what remains to be done to solve the missing piece, and what are the biggest challenges to overcome (assuming one gets access to the appropriate low level APIs on top of the hardware — like OpenAI’s Triton).
      - patagurbon 3y ago
        XLA.jl is essentially abandoned, no funding IIRC. Coil.jl and maybe some future hypothetical work on MLIR.jl could bridge the gap pretty easily.
  - nextos 3y ago
    That's right, but at the same time Flux, the default choice is not mature at all compared to JAX. So that advantage is only on paper. I would really prefer to use Julia, I actually dislike Python, but it's unrealistic to implement large architectures in Flux. It is buggy, lots of gradient calculations are unreliable. See some discussion here for instance: https://kidger.site/thoughts/jax-vs-julia https://kidger.site/thoughts/jax-vs-julia
- enkursigilo 3y ago
  How about Flux.jl?

Julia is looking more and more like a viable alternative to Python's data science stack.

by atulvi 3y ago

Julia is looking more and more like a viable alternative to Python's data science stack.

I want this to become reality. Are there any IDEs for Julia that come close to the ergonomics and feature set of Pycharm?

cbkeller 3y ago
The most IDE-ish experience is probably currently the vscode plugin. I haven’t used pycharm specifically for comparison, but I suspect “probably not” as compared to a polished paid IDE in general. There’s progress being made though.

nextos 3y ago

There's still one major bottleneck. There's no TF / PyTorch / JAX replacement. For small things, interop with R is good, so one can defer things to R packages and get access to a great set of functionality.

ssivark 3y ago
> There's no TF / PyTorch / JAX replacement What do you mean by that? AFAICT for the most part Julia is already like the best that Jax could ever hope to be — since there is language level support for JIT compilation, gradients, etc — given how well libraries compose without having to pick different incompatible subsets of the language.
- the_svd_doctor 3y ago
  I mean, yes but also no. JAX will JIT on accelerators OOTB and has outstanding (imho) support for multi node. Julia does not have that OOTB AFAIK. Now I agree that Julia, with its JIT, is well positioned to do all this. But I don’t think it does.
  - ssivark 3y ago
    Ah, I see. There’s XLA.jl for TPUs and a bunch of stuff from the JuliaGPU organization for GPUs — I wonder how they compare with TF/PyTorch/JAX. I’m curious to get perspectives on what remains to be done to solve the missing piece, and what are the biggest challenges to overcome (assuming one gets access to the appropriate low level APIs on top of the hardware — like OpenAI’s Triton).
    - patagurbon 3y ago
      XLA.jl is essentially abandoned, no funding IIRC. Coil.jl and maybe some future hypothetical work on MLIR.jl could bridge the gap pretty easily.
- nextos 3y ago
  That's right, but at the same time Flux, the default choice is not mature at all compared to JAX. So that advantage is only on paper. I would really prefer to use Julia, I actually dislike Python, but it's unrealistic to implement large architectures in Flux. It is buggy, lots of gradient calculations are unreliable. See some discussion here for instance: https://kidger.site/thoughts/jax-vs-julia https://kidger.site/thoughts/jax-vs-julia
enkursigilo 3y ago
How about Flux.jl?