4 ms·
I've been excited by Turing, just to see a probabilistic programming framework like that in Julia. I think the expressiveness of Julia and it being native to th
by nerdponx 7y ago
I've been excited by Turing, just to see a probabilistic programming framework like that in Julia. I think the expressiveness of Julia and it being native to that framework will be helpful.
I hope so too. But hasn't Julia's TF/Torch equivalent, Flux, had performance problems? That was the rumor I heard anyway, I haven't had the chance to use it myself.
- dklend122 7y agoIt currently has problems with some classes of models on the GPU, but this is just due to memory management and is going to fixed soon. Julia is natively compiled, so doesn't use a tape, tracing, or for memory management simple ref counting (in this case not as good). This is slated to be fixed in the short term with an abstract tracing framework which eliminates memory allocations. Given Julia's type information and ability to manipulate IR from third party programs, this is more general and powerful than pytorch's tracing system. Works on a larger set of code (the whole language), doesn't require actually running the code (abstract tracing), and allows for other program transforms/ analysis like compilation to XLA, shape inference, compile time errors, source to source prob programming : https://github.com/MikeInnes/Poirot.jl https://github.com/MikeInnes/Poirot.jl, and other things. That's in the short term and should bring flux up to SOTA for speed (it already is on CPU). In the medium term, a general framework for optimizer passes will allow for more general compile time memory management.
- ChrisRackauckas 7y agoNote you can use this on Zygote to preallocate stuff: https://github.com/oxinabox/AutoPreallocation.jl https://github.com/oxinabox/AutoPreallocation.jl . It doesn't support GPUs yet, mainly because the dev on it needs a GPU CI setup, but it should mostly just work (issue https://github.com/oxinabox/AutoPreallocation.jl/issues/10 https://github.com/oxinabox/AutoPreallocation.jl/issues/10)
- krastanov 7y agoWow... This is mind blowing! I am surprised there has not been more fanfare about this package.
- darsnack 7y agoMost of that is not fundamental to Julia or Flux itself. It’s the difference between a monolithic package like TF and source-to-source AD in Julia. The former allows the designers to use their own data structures and external libraries to do optimizations. Source-to-source relies on the underlying IR used by Julia, making optimizations challenging without some compiler assistance. But all of that is in the pipeline with stopgap solutions on the way. As with most things in Julia, the code developers don’t just want to hack changes that work, but make changes that are flexible, extensible, and can solve many problems at once. So, Flux isn’t ready for prime time yet, but it is definitely worth keeping your eye on it.