It currently has problems with some classes of models on the GPU, but this is just due to memory management and is going to fixed soon. Julia is natively compil

It currently has problems with some classes of models on the GPU, but this is just due to memory management and is going to fixed soon. Julia is natively compiled, so doesn't use a tape, tracing, or for memory management simple ref counting (in this case not as good). This is slated to be fixed in the short term with an abstract tracing framework which eliminates memory allocations. Given Julia's type information and ability to manipulate IR from third party programs, this is more general and powerful than pytorch's tracing system. Works on a larger set of code (the whole language), doesn't require actually running the code (abstract tracing), and allows for other program transforms/ analysis like compilation to XLA, shape inference, compile time errors, source to source prob programming : https://github.com/MikeInnes/Poirot.jl https://github.com/MikeInnes/Poirot.jl, and other things. That's in the short term and should bring flux up to SOTA for speed (it already is on CPU). In the medium term, a general framework for optimizer passes will allow for more general compile time memory management.