4 ms·

Using pytorch/keras/TF from Julia is possible, and even pleasant: https://twitter.com/aureliengeron/status/1277751121440698368 https://twitter.com/aureliengeron

by amkkma 6y ago

Using pytorch/keras/TF from Julia is possible, and even pleasant: https://twitter.com/aureliengeron/status/1277751121440698368 https://twitter.com/aureliengeron/status/1277751121440698368 Though I much prefer Flux.jl

UncleOxidant 6y ago
In my experience Flux.jl has a ways to go before it reaches Pytorch performance. Apparently this has something to do with some very efficient GPU kernels that Pytorch uses - I wonder if it's possible for Flux to "borrow" those GPU kernels? It's possible to write GPU kernels in Julia so hopefully this will help close the gap.
- amkkma 6y ago
  The short answer is yes, it's possible and easy. The longer answer is, that's a bit of an apples to oranges comparison. Pytorch is calling optimized c++ kernels, whereas Flux is completely generic and can do GPU and CPU codegen (including AMD). Flux can do things Pytorch just doesn't have CUDA code for, It's more generic and composable and it's getting there with a much smaller team/budget. That means it has more potential, with the drawback that it isn't quite as fast, for now, in some of the common use cases the pytorch has been hard coded/ hyperoptimized for. In other more complex cases beyond big fat matmuls and such, it's much faster, even comparing to torchscript (sciml, neural ODEs etc). Still, the codegen is getting there with the funded work on the GPU compiler and memory allocation and there eventually won't be any sort of tradeoff. In the meantime, Yes, there is a package at allows calling pytorch kernels without modifying your code. This is also only possible with such finesse due to Julia's power and composability. See here for the package: https://fluxml.ai/2020/06/29/acclerating-flux-torch.html https://fluxml.ai/2020/06/29/acclerating-flux-torch.html
- ViralBShah 6y ago
  Flux already has the ability to borrow Torch kernels. This was released fairly recently. https://github.com/FluxML/Torch.jl https://github.com/FluxML/Torch.jl https://fluxml.ai/2020/06/29/acclerating-flux-torch.html https://fluxml.ai/2020/06/29/acclerating-flux-torch.html Also the gap between native GPU codegen in Julia and the hand-optimized kernels in PyTorch is lesser than you'd imagine. In fact, see this talk at JuliaCon, which compares Julia on a 1000 GPUs against a CUDA/C implementation, and finds them to be extremely close. https://www.youtube.com/watch?v=vPsfZUqI4_0 https://www.youtube.com/watch?v=vPsfZUqI4_0 https://github.com/eth-cscs/ImplicitGlobalGrid.jl https://github.com/eth-cscs/ImplicitGlobalGrid.jl
  - UncleOxidant 6y ago
    Is this just wrapping the torch kernels or the entire torch API? I was under the impression it was the latter.
- eigenspace 6y ago
  I feel I should point out that if you need performance and you're mostly only interested in classic PyTorch or TensorFlow style machine learning (i.e. you don't need Flux.jl's characteristic flexibility), then you should really check out KNet.jl [1]. KNet is generally around the same flexibility and performance as something like PyTorch. It doesn't get nearly as much attention as Flux because it's less novel, but it really should be looked at more. [1] https://github.com/denizyuret/Knet.jl https://github.com/denizyuret/Knet.jl
  - ViralBShah 6y ago
    Knet.jl is pretty awesome. The main challenge has been compiling its cuda kernel library. Deniz has helped anyone who ran into issues. However, we are integrating a lot of the underlying infrastructure to the point that both Knet and Flux will only differ in API approach.
- ChrisRackauckas 6y ago
  It really depends on where you're looking. We tested neural ODE compatible solver benchmarks from 3 ODEs and about 1 million ODEs and saw an advantage for Julia of >30x across the board [1]. Then showed the spiral neural ODE from the original paper train 100x faster in Julia [2]. So it should at least be qualified as, PyTorch has some good kernels, and it has some things it doesn't optimize close to as much as Julia. [1] https://gist.github.com/ChrisRackauckas/cc6ac746e2dfd285c28e0584a2bfd320 https://gist.github.com/ChrisRackauckas/cc6ac746e2dfd285c28e... [2] https://gist.github.com/ChrisRackauckas/4a4d526c15cc4170ce37da837bfc32c4 https://gist.github.com/ChrisRackauckas/4a4d526c15cc4170ce37...

by amkkma 6y ago

UncleOxidant 6y ago

In my experience Flux.jl has a ways to go before it reaches Pytorch performance. Apparently this has something to do with some very efficient GPU kernels that Pytorch uses - I wonder if it's possible for Flux to "borrow" those GPU kernels? It's possible to write GPU kernels in Julia so hopefully this will help close the gap.

amkkma 6y ago
The short answer is yes, it's possible and easy. The longer answer is, that's a bit of an apples to oranges comparison. Pytorch is calling optimized c++ kernels, whereas Flux is completely generic and can do GPU and CPU codegen (including AMD). Flux can do things Pytorch just doesn't have CUDA code for, It's more generic and composable and it's getting there with a much smaller team/budget. That means it has more potential, with the drawback that it isn't quite as fast, for now, in some of the common use cases the pytorch has been hard coded/ hyperoptimized for. In other more complex cases beyond big fat matmuls and such, it's much faster, even comparing to torchscript (sciml, neural ODEs etc). Still, the codegen is getting there with the funded work on the GPU compiler and memory allocation and there eventually won't be any sort of tradeoff. In the meantime, Yes, there is a package at allows calling pytorch kernels without modifying your code. This is also only possible with such finesse due to Julia's power and composability. See here for the package: https://fluxml.ai/2020/06/29/acclerating-flux-torch.html https://fluxml.ai/2020/06/29/acclerating-flux-torch.html
ViralBShah 6y ago
Flux already has the ability to borrow Torch kernels. This was released fairly recently. https://github.com/FluxML/Torch.jl https://github.com/FluxML/Torch.jl https://fluxml.ai/2020/06/29/acclerating-flux-torch.html https://fluxml.ai/2020/06/29/acclerating-flux-torch.html Also the gap between native GPU codegen in Julia and the hand-optimized kernels in PyTorch is lesser than you'd imagine. In fact, see this talk at JuliaCon, which compares Julia on a 1000 GPUs against a CUDA/C implementation, and finds them to be extremely close. https://www.youtube.com/watch?v=vPsfZUqI4_0 https://www.youtube.com/watch?v=vPsfZUqI4_0 https://github.com/eth-cscs/ImplicitGlobalGrid.jl https://github.com/eth-cscs/ImplicitGlobalGrid.jl
- UncleOxidant 6y ago
  Is this just wrapping the torch kernels or the entire torch API? I was under the impression it was the latter.
eigenspace 6y ago
I feel I should point out that if you need performance and you're mostly only interested in classic PyTorch or TensorFlow style machine learning (i.e. you don't need Flux.jl's characteristic flexibility), then you should really check out KNet.jl [1]. KNet is generally around the same flexibility and performance as something like PyTorch. It doesn't get nearly as much attention as Flux because it's less novel, but it really should be looked at more. [1] https://github.com/denizyuret/Knet.jl https://github.com/denizyuret/Knet.jl
- ViralBShah 6y ago
  Knet.jl is pretty awesome. The main challenge has been compiling its cuda kernel library. Deniz has helped anyone who ran into issues. However, we are integrating a lot of the underlying infrastructure to the point that both Knet and Flux will only differ in API approach.
ChrisRackauckas 6y ago
It really depends on where you're looking. We tested neural ODE compatible solver benchmarks from 3 ODEs and about 1 million ODEs and saw an advantage for Julia of >30x across the board [1]. Then showed the spiral neural ODE from the original paper train 100x faster in Julia [2]. So it should at least be qualified as, PyTorch has some good kernels, and it has some things it doesn't optimize close to as much as Julia. [1] https://gist.github.com/ChrisRackauckas/cc6ac746e2dfd285c28e0584a2bfd320 https://gist.github.com/ChrisRackauckas/cc6ac746e2dfd285c28e... [2] https://gist.github.com/ChrisRackauckas/4a4d526c15cc4170ce37da837bfc32c4 https://gist.github.com/ChrisRackauckas/4a4d526c15cc4170ce37...