4 ms·

I keep orbiting Julia, because of developments like this one. I am an old Lisper/Scheme guy. How does this compare to say R6RS-AD[1] or cl-autodiff[2] or Deriva

by eggy 7y ago

I keep orbiting Julia, because of developments like this one. I am an old Lisper/Scheme guy. How does this compare to say R6RS-AD[1] or cl-autodiff[2] or Deriva for Clojure/Java[3]? I realize Julia has femtolisp inside, yet another Lisp bait for me! [1] https://github.com/qobi/R6RS-AD https://github.com/qobi/R6RS-AD [2] https://github.com/masonium/cl-autodiff https://github.com/masonium/cl-autodiff [3] https://github.com/lambder/Deriva https://github.com/lambder/Deriva

ddragon 7y ago
Those are based on overloading functions to add the new behaviors (computing the forward diff or building the graph within a special variable for the reverse mode) which in Julia is implemented for example in [1], [2], [3] and [4], while the one in OP is based on source analysis and transformation. And the similarities with Lisp are more than just the parser being written in it, Julia's programming paradigm is based on the CLOS, everything is an expression, hygienic macros and reader macros and code is data. It mostly just lacks sexpr. [1] https://github.com/JuliaDiff/ForwardDiff.jl https://github.com/JuliaDiff/ForwardDiff.jl [2] https://github.com/dfdx/Yota.jl https://github.com/dfdx/Yota.jl [3] https://github.com/denizyuret/AutoGrad.jl https://github.com/denizyuret/AutoGrad.jl (from the Knet framework) [4] https://fluxml.ai/Flux.jl/stable/internals/tracker/ https://fluxml.ai/Flux.jl/stable/internals/tracker/ (from the Flux framework)
- eggy 7y ago
  Wow, thank you for such an informative reply! I am going to try this out. One of my pet peeves with Julia is that my main machine is a Windows 10 box, and things like cudanative.jl say they only install on Mac and Linux. I have an old 2013 Linux box (Lenovo T430u laptop). BTW, do you recommend Julia Pro install or vanilla Julia and build up for more general technical programming, not just ML and DL.
  - ddragon 7y ago
    I prefer vanilla Julia since it has all the packages at the latest released versions and it's the most used so it's easier to get help. CUDANative.jl documentation doesn't mention being mac/linux only (the only mention was in the 2 year old preview release), and if it is maybe you could try another library like ArrayFire.jl. And more general technical programming is Julia specialty (it was built for that as a high performance interactive language), the DL hype started after the language was first released.

by eggy 7y ago

ddragon 7y ago

Those are based on overloading functions to add the new behaviors (computing the forward diff or building the graph within a special variable for the reverse mode) which in Julia is implemented for example in [1], [2], [3] and [4], while the one in OP is based on source analysis and transformation. And the similarities with Lisp are more than just the parser being written in it, Julia's programming paradigm is based on the CLOS, everything is an expression, hygienic macros and reader macros and code is data. It mostly just lacks sexpr. [1] https://github.com/JuliaDiff/ForwardDiff.jl https://github.com/JuliaDiff/ForwardDiff.jl [2] https://github.com/dfdx/Yota.jl https://github.com/dfdx/Yota.jl [3] https://github.com/denizyuret/AutoGrad.jl https://github.com/denizyuret/AutoGrad.jl (from the Knet framework) [4] https://fluxml.ai/Flux.jl/stable/internals/tracker/ https://fluxml.ai/Flux.jl/stable/internals/tracker/ (from the Flux framework)

eggy 7y ago
Wow, thank you for such an informative reply! I am going to try this out. One of my pet peeves with Julia is that my main machine is a Windows 10 box, and things like cudanative.jl say they only install on Mac and Linux. I have an old 2013 Linux box (Lenovo T430u laptop). BTW, do you recommend Julia Pro install or vanilla Julia and build up for more general technical programming, not just ML and DL.
- ddragon 7y ago
  I prefer vanilla Julia since it has all the packages at the latest released versions and it's the most used so it's easier to get help. CUDANative.jl documentation doesn't mention being mac/linux only (the only mention was in the 2 year old preview release), and if it is maybe you could try another library like ArrayFire.jl. And more general technical programming is Julia specialty (it was built for that as a high performance interactive language), the DL hype started after the language was first released.