3 ms·

Out of curiosity, how's the state of DL for Julia. Can I use PyTorch or JAX comfortably in Julia?

teruakohatu 3y ago
Flux is quite a nice lower level library: https://github.com/FluxML/Flux.jl https://github.com/FluxML/Flux.jl On top of that there are many higher level libraries such as Transformers.jl https://github.com/chengchingwen/Transformers.jl https://github.com/chengchingwen/Transformers.jl
markkitti 3y ago
There is https://github.com/FluxML/Torch.jl https://github.com/FluxML/Torch.jl. There are also Julia native frameworks such as FluxML, https://fluxml.ai/ https://fluxml.ai/ .
ChrisRackauckas 3y ago
Lux.jl does a really good job at being clear with syntax and hackable. I couldn't recommend it more. https://lux.csail.mit.edu/ https://lux.csail.mit.edu/. Here's good materials to start with: https://lux.csail.mit.edu/dev/tutorials/beginner/1_Basics https://lux.csail.mit.edu/dev/tutorials/beginner/1_Basics
- Tarrosion 3y ago
  What's the user-facing difference between Lux and Flux?
  - darsnack 3y ago
    How you interact with parameters. Lux is similar to Flax (Jax) where the parameters are kept in a separate variable from the model definition, and they are passed in on the forward pass. Notably, this design choice allows Lux to accept parameters built with ComponentArrays.jl which can be especially helpful when working with libraries that expect flat vectors of parameters. Flux lies somewhere between Jax and PyTorch. Like PyTorch, the parameters are stored as part of the model. Unlike traditional PyTorch, Flux has “functional” conventions, e.g. `g = gradient(loss, model)` vs. `loss.backward()`. Similar to Flax, the model is a tree of parameters.
staunton 3y ago
> Can I use PyTorch or JAX comfortably in Julia? No. And it doesn't seem like that will become possible any time soon.
darsnack 3y ago
> Can I use PyTorch or JAX comfortably in Julia? There is https://github.com/rejuvyesh/PyCallChainRules.jl https://github.com/rejuvyesh/PyCallChainRules.jl which makes this possible. But using some of the native Julia ML libraries that others have mentioned is preferable.

Flux is quite a nice lower level library: https://github.com/FluxML/Flux.jl https://github.com/FluxML/Flux.jl On top of that there are many higher level libraries such as Transformers.jl https://github.com/chengchingwen/Transformers.jl https://github.com/chengchingwen/Transformers.jl

markkitti 3y ago

There is https://github.com/FluxML/Torch.jl https://github.com/FluxML/Torch.jl. There are also Julia native frameworks such as FluxML, https://fluxml.ai/ https://fluxml.ai/ .

ChrisRackauckas 3y ago

Lux.jl does a really good job at being clear with syntax and hackable. I couldn't recommend it more. https://lux.csail.mit.edu/ https://lux.csail.mit.edu/. Here's good materials to start with: https://lux.csail.mit.edu/dev/tutorials/beginner/1_Basics https://lux.csail.mit.edu/dev/tutorials/beginner/1_Basics

Tarrosion 3y ago
What's the user-facing difference between Lux and Flux?
- darsnack 3y ago
  How you interact with parameters. Lux is similar to Flax (Jax) where the parameters are kept in a separate variable from the model definition, and they are passed in on the forward pass. Notably, this design choice allows Lux to accept parameters built with ComponentArrays.jl which can be especially helpful when working with libraries that expect flat vectors of parameters. Flux lies somewhere between Jax and PyTorch. Like PyTorch, the parameters are stored as part of the model. Unlike traditional PyTorch, Flux has “functional” conventions, e.g. `g = gradient(loss, model)` vs. `loss.backward()`. Similar to Flax, the model is a tree of parameters.

staunton 3y ago

> Can I use PyTorch or JAX comfortably in Julia? No. And it doesn't seem like that will become possible any time soon.

darsnack 3y ago

> Can I use PyTorch or JAX comfortably in Julia? There is https://github.com/rejuvyesh/PyCallChainRules.jl https://github.com/rejuvyesh/PyCallChainRules.jl which makes this possible. But using some of the native Julia ML libraries that others have mentioned is preferable.