3 ms·
Check out ForwardDiff.jl [1], which is a really impressive implementation of this exact idea. The technique gets pretty magical results; you can apply it to any
by one-more-minute 10y ago
Check out ForwardDiff.jl [1], which is a really impressive implementation of this exact idea. The technique gets pretty magical results; you can apply it to any function, complete with loops, linalg etc, and it will compute a derivative with something like 10% overhead. The standard approach is finite differencing, which involves many-times overhead, isn't exact, and can easily blow up for common pathological cases like step functions.
Fede_V's points about the drawbacks of the technique are valid in C++, but Julia's duck-typing makes being generic the default (including in the standard library). ForwardDiff works out of the box, for free, in a huge number of cases.
[1] https://github.com/JuliaDiff/ForwardDiff.jl https://github.com/JuliaDiff/ForwardDiff.jl
- dnautics 10y agoI hesitate to call what julia does "duck typing" because types are usually checked at compile time, even though it looks like "run-time" because the system holds off on compilation until it's needed. If you write your functions correctly, (with type stability) there is no active type checking that's done after the first time you call the functions. There certainly is run-time checking in julia (true duck-typing) if you are not careful with how you write your functions, but I doubt that ForwardDiff, for example, uses it.
- one-more-minute 10y agoBy duck typing, I mean that you can write: f(x) = x^2 + 2 for example, and the function will happily run with any type that supports multiply and add. This is in contrast to languages where `x` has to explicitly inherit from a given supertype or implement an interface to be valid. It's true that Julia will specialise the function for ints or complex numbers or whatever, but that's an implementation detail which doesn't change the meaning of the program.