3 ms·

Whatever happened to Julia? Wasn't it supposed to incorporate all these incredible abstractions at the language level and run quickly on GPUs and everything in-

Whatever happened to Julia? Wasn't it supposed to incorporate all these incredible abstractions at the language level and run quickly on GPUs and everything in-between? Is it just lack of adoption or is has it something else?

fspeech 7y ago
Julia works with CUDA on Windows without the need for Visual Studio since Julia is LLVM based.
ddragon 7y ago
If you mean Zygote.jl, it's a very ambitious project (like Swift for Tensorflow which has been under development for even longer I believe) with not many people working on it compared to Tensorflow and pytorch. And Pytorch for example only supports the method it decides to overload, while Zygote aims to support everything in the language (including stuff that isn't as obvious like state, IO, control flow in general). And then you have optimizations over the computation graph, memory management on GPU and many corner cases I can't imagine. Though you can already use very clean Pytorch style libraries like Flux and Knet or the Tensorflow bindings to leverage the benefits of Julia for high performance numerical processing on the adjacent tasks such as data preprocessing.
- carapace 7y ago
  "A Differentiable Programming System to Bridge Machine Learning and Scientific Computing" https://arxiv.org/abs/1907.07587 https://arxiv.org/abs/1907.07587 https://news.ycombinator.com/item?id=20477873 https://news.ycombinator.com/item?id=20477873 From the abstract: > We describe Zygote, a Differentiable Programming system that is able to take gradients of general program structures. We implement this system in the Julia programming language. Our system supports almost all language constructs (control flow, recursion, mutation, etc.) and compiles high-performance code without requiring any user intervention or refactoring to stage computations. Just linking to this for those who haven't seen it.
chillee 7y ago
I wanted to write something about Julia/Swift, but I had a bit of a hard time writing a cohesive narrative about them and ran out of time. You can take a look at https://discourse.julialang.org/t/where-does-julia-provide-the-biggest-benefits-over-other-ml-frameworks-for-research/28481/34 https://discourse.julialang.org/t/where-does-julia-provide-t... for some of my questions. Essentially, the biggest advantage imo is that Julia offers a single cohesive language, where compilers can do anything at the language level. I don't think this will allow for a single killer application - almost anything Julia can do ca n be simulated by some combination of Python/C++. However, what might be true is that using a single language allows for much faster development and iteration than a combination of Python/C++. I think the way that'll manifest is in more and more high quality libraries coming out for Julia that are higher quality than the Python ones. Maybe wait 5 years, and we'll see what happens :)
samcodes 7y ago
Julia is a fantastic language. If I did original research I would use it, but I'm in industry and focus on productionalizing things.
drewm1980 7y ago
Coming from using Tensorflow in industry, I recently played with Flux at home. Language level support for AD should be a game changer, but it's a hard transition mentally. You have to understand one language deeply rather than two languages shallowly. I found myself bogged down solving lispy puzzles involving functions composing other functions. In tf (and most AD frameworks) you churn out some ugly procedural code in an ergonomic language that generates some ugly pure functional code in a more limited language (the computational graph). Different cognitive overheads. Julia hasn't been 1.0 for very long; it may still take off.
logicchains 7y ago
I've been using Julia's Flux, it's great for when you have some arbitrary model you want to run gradient descent on that isn't just a bunch of matrix ops, as the framework overhead is way less than TF or PyTorch due to Julia being 100x faster than pure Python.
xvilka 7y ago
For me, it would be the lack of GNN support in Flux[1]. I really root for Julia and Flux, and I hope it will take off soon. [1] https://github.com/FluxML/Flux.jl/issues/625 https://github.com/FluxML/Flux.jl/issues/625

fspeech 7y ago

Julia works with CUDA on Windows without the need for Visual Studio since Julia is LLVM based.

ddragon 7y ago

If you mean Zygote.jl, it's a very ambitious project (like Swift for Tensorflow which has been under development for even longer I believe) with not many people working on it compared to Tensorflow and pytorch. And Pytorch for example only supports the method it decides to overload, while Zygote aims to support everything in the language (including stuff that isn't as obvious like state, IO, control flow in general). And then you have optimizations over the computation graph, memory management on GPU and many corner cases I can't imagine. Though you can already use very clean Pytorch style libraries like Flux and Knet or the Tensorflow bindings to leverage the benefits of Julia for high performance numerical processing on the adjacent tasks such as data preprocessing.

carapace 7y ago
"A Differentiable Programming System to Bridge Machine Learning and Scientific Computing" https://arxiv.org/abs/1907.07587 https://arxiv.org/abs/1907.07587 https://news.ycombinator.com/item?id=20477873 https://news.ycombinator.com/item?id=20477873 From the abstract: > We describe Zygote, a Differentiable Programming system that is able to take gradients of general program structures. We implement this system in the Julia programming language. Our system supports almost all language constructs (control flow, recursion, mutation, etc.) and compiles high-performance code without requiring any user intervention or refactoring to stage computations. Just linking to this for those who haven't seen it.

chillee 7y ago

I wanted to write something about Julia/Swift, but I had a bit of a hard time writing a cohesive narrative about them and ran out of time. You can take a look at https://discourse.julialang.org/t/where-does-julia-provide-the-biggest-benefits-over-other-ml-frameworks-for-research/28481/34 https://discourse.julialang.org/t/where-does-julia-provide-t... for some of my questions. Essentially, the biggest advantage imo is that Julia offers a single cohesive language, where compilers can do anything at the language level. I don't think this will allow for a single killer application - almost anything Julia can do ca n be simulated by some combination of Python/C++. However, what might be true is that using a single language allows for much faster development and iteration than a combination of Python/C++. I think the way that'll manifest is in more and more high quality libraries coming out for Julia that are higher quality than the Python ones. Maybe wait 5 years, and we'll see what happens :)

samcodes 7y ago

Julia is a fantastic language. If I did original research I would use it, but I'm in industry and focus on productionalizing things.

drewm1980 7y ago

Coming from using Tensorflow in industry, I recently played with Flux at home. Language level support for AD should be a game changer, but it's a hard transition mentally. You have to understand one language deeply rather than two languages shallowly. I found myself bogged down solving lispy puzzles involving functions composing other functions. In tf (and most AD frameworks) you churn out some ugly procedural code in an ergonomic language that generates some ugly pure functional code in a more limited language (the computational graph). Different cognitive overheads. Julia hasn't been 1.0 for very long; it may still take off.

logicchains 7y ago

I've been using Julia's Flux, it's great for when you have some arbitrary model you want to run gradient descent on that isn't just a bunch of matrix ops, as the framework overhead is way less than TF or PyTorch due to Julia being 100x faster than pure Python.

xvilka 7y ago

For me, it would be the lack of GNN support in Flux[1]. I really root for Julia and Flux, and I hope it will take off soon. [1] https://github.com/FluxML/Flux.jl/issues/625 https://github.com/FluxML/Flux.jl/issues/625