4 ms·

On the other hand, there are a lot less Julia developers than Python developers so focusing on Zygote instead of splitting the effort over several AD packages m

On the other hand, there are a lot less Julia developers than Python developers so focusing on Zygote instead of splitting the effort over several AD packages makes some sense, doesn't it? There were already a few AD packages that were fairly mature: https://www.juliadiff.org/ https://www.juliadiff.org/ Also, I think performance is very important because Zygote is used in Flux (Julia's main deep learning framework at this point.) and if it's keeping Flux's performance from matching, say, PyTorch's then that's going to limit adoption of Flux and Julia. One of Julia's main claims to fame is performance so people coming to kick the tires are going to be disappointed if it's actually slower for this domain.

eigenspace 7y ago
I think my concern is that the main Zygote dev is not very responsive or communicative. That's fine, he doesn't owe the rest of the community anything, but it also seems irresponsible to put all our eggs in his basket (note I'm by no means knowledgeable about this and I'm open to the idea that it's a mis-characterization. This is merely my impression as a relative outsider looking in). Besides, any implementation of AD is going to have downsides. Having many different, hot swappable implementations is quite nice because you can tailor towards your needs.
oxinabox 7y ago
We have so many AD packages than that in julia. Julia suffers the lisp problem for AD. http://winestockwebdesign.com/Essays/Lisp_Curse.html http://winestockwebdesign.com/Essays/Lisp_Curse.html It is a real problem. One of my on going projects is ChainRules (http://www.juliadiff.org/ChainRulesCore.jl/dev/ http://www.juliadiff.org/ChainRulesCore.jl/dev/) which will unite them under one set of custom senstitivities and more generally make it easier to mix and match them I need to update the JuliaDiff website, I want to list all of them in a table with some some key points. Forward mode: - [ForwardDiff](https://github.com/JuliaDiff/ForwardDiff.jl https://github.com/JuliaDiff/ForwardDiff.jl) - [ForwardDiff2](https://github.com/YingboMa//ForwardDiff2.jl https://github.com/YingboMa//ForwardDiff2.jl) Reverse Mode: - [Nabla](https://github.com/invenia/Nabla.jl/ https://github.com/invenia/Nabla.jl/) - [Tracker](https://github.com/FluxML/Tracker.jl https://github.com/FluxML/Tracker.jl) - [Yota](https://github.com/dfdx/Yota.jl https://github.com/dfdx/Yota.jl) - [Zygote](https://github.com/FluxML/Zygote.jl https://github.com/FluxML/Zygote.jl) - [ReverseDiff](https://github.com/JuliaDiff/ReverseDiff.jl https://github.com/JuliaDiff/ReverseDiff.jl) - [AutoGrad.jl](https://github.com/denizyuret/AutoGrad.jl https://github.com/denizyuret/AutoGrad.jl) - [NiLang](https://github.com/GiggleLiu/NiLang.jl https://github.com/GiggleLiu/NiLang.jl) (arguably not reverse mode) Symbolic: - [ModelingToolKit](https://github.com/JuliaDiffEq/ModelingToolkit.jl https://github.com/JuliaDiffEq/ModelingToolkit.jl) - [XGrad.jl](https://github.com/dfdx/XGrad.jl https://github.com/dfdx/XGrad.jl) Finite Differencing: - [Calculus](https://github.com/JuliaMath/Calculus.jl https://github.com/JuliaMath/Calculus.jl) (please stop) - [FiniteDifferences](https://github.com/JuliaDiff/FiniteDifferences.jl https://github.com/JuliaDiff/FiniteDifferences.jl) - [FiniteDiff](https://github.com/JuliaDiff/FiniteDiff.jl https://github.com/JuliaDiff/FiniteDiff.jl)

eigenspace 7y ago

I think my concern is that the main Zygote dev is not very responsive or communicative. That's fine, he doesn't owe the rest of the community anything, but it also seems irresponsible to put all our eggs in his basket (note I'm by no means knowledgeable about this and I'm open to the idea that it's a mis-characterization. This is merely my impression as a relative outsider looking in). Besides, any implementation of AD is going to have downsides. Having many different, hot swappable implementations is quite nice because you can tailor towards your needs.

oxinabox 7y ago

We have so many AD packages than that in julia. Julia suffers the lisp problem for AD. http://winestockwebdesign.com/Essays/Lisp_Curse.html http://winestockwebdesign.com/Essays/Lisp_Curse.html It is a real problem. One of my on going projects is ChainRules (http://www.juliadiff.org/ChainRulesCore.jl/dev/ http://www.juliadiff.org/ChainRulesCore.jl/dev/) which will unite them under one set of custom senstitivities and more generally make it easier to mix and match them I need to update the JuliaDiff website, I want to list all of them in a table with some some key points. Forward mode: - [ForwardDiff](https://github.com/JuliaDiff/ForwardDiff.jl https://github.com/JuliaDiff/ForwardDiff.jl) - [ForwardDiff2](https://github.com/YingboMa//ForwardDiff2.jl https://github.com/YingboMa//ForwardDiff2.jl) Reverse Mode: - [Nabla](https://github.com/invenia/Nabla.jl/ https://github.com/invenia/Nabla.jl/) - [Tracker](https://github.com/FluxML/Tracker.jl https://github.com/FluxML/Tracker.jl) - [Yota](https://github.com/dfdx/Yota.jl https://github.com/dfdx/Yota.jl) - [Zygote](https://github.com/FluxML/Zygote.jl https://github.com/FluxML/Zygote.jl) - [ReverseDiff](https://github.com/JuliaDiff/ReverseDiff.jl https://github.com/JuliaDiff/ReverseDiff.jl) - [AutoGrad.jl](https://github.com/denizyuret/AutoGrad.jl https://github.com/denizyuret/AutoGrad.jl) - [NiLang](https://github.com/GiggleLiu/NiLang.jl https://github.com/GiggleLiu/NiLang.jl) (arguably not reverse mode) Symbolic: - [ModelingToolKit](https://github.com/JuliaDiffEq/ModelingToolkit.jl https://github.com/JuliaDiffEq/ModelingToolkit.jl) - [XGrad.jl](https://github.com/dfdx/XGrad.jl https://github.com/dfdx/XGrad.jl) Finite Differencing: - [Calculus](https://github.com/JuliaMath/Calculus.jl https://github.com/JuliaMath/Calculus.jl) (please stop) - [FiniteDifferences](https://github.com/JuliaDiff/FiniteDifferences.jl https://github.com/JuliaDiff/FiniteDifferences.jl) - [FiniteDiff](https://github.com/JuliaDiff/FiniteDiff.jl https://github.com/JuliaDiff/FiniteDiff.jl)