3 ms·
> and there are some great AD packages, but in no way is everything automatically differentiable (even with the nice packages), nor is that a design goal. I wo
by oxinabox 6y ago
> and there are some great AD packages, but in no way is everything automatically differentiable (even with the nice packages), nor is that a design goal.
I work on the AD infastructure for Julia.
That absolutely is a design goal.
Certainly we are not there yet; we still have a long way to go.
But that is where we want to go to.
With the cavet that thigns that are not mathematically defined to have derivatives (e.g. the derivative of `xs[i]` with respect to `i`) we won't differentiate those.
But for stuff like mutation (the big thing Zygote doesn't support (though some of our other ADs do)), we sure do want it to.
- JHonaker 6y agoSorry, I guess I was unclear. It's definitely a goal for the AD ecosystem, but, as far as I'm aware, AD is not a design goal of Julia itself. That was the point I was trying to make.
- eigenspace 6y agoPerhaps not originally, but now it is certainly a language design goal that the entire compilation pipeline can be parameterized and modified to allow for a huge variety of custom transforms. AD is a subset of the transformations they are endeavouring to support. E.g. see https://github.com/JuliaLang/julia/pull/33955 https://github.com/JuliaLang/julia/pull/33955 Generally speaking, if there is any sort of code transformation pass that's needed for AD but not supported, you can expect people to be working to support either that specific transformation, or a generalization of the transformation. This has been a theme in the language development for years now.
- JHonaker 6y agoThat’s really interesting. Thanks for sharing.