3 ms·

Clearly y'(0) = 0 here, since at x=0 the function is defined to be a constant. In calculus class this would be ill-defined / ambiguous, but the computer is sim

Clearly y'(0) = 0 here, since at x=0 the function is defined to be a constant. In calculus class this would be ill-defined / ambiguous, but the computer is simple-minded and picks one answer. This doesn't seem like a major flaw to me. What are you using this for which would care? Maybe you have something like a rectification which produces exactly x=0 much more often than 1 in 2^32... in which case a function like your y amounts to a special case for this. If you cared you could write a special case for the derivative too, here's one which will give y'(0)=42 but otherwise the automatic derivative: julia> y(x) = x>0 ? x : -2x julia> y(x::Dual) = x==0 ? dual(0, 42) : (x>0 ? x : -2x)

ChrisRackauckas 8y ago
Yeah, this is a good case to use multiple dispatch to flexibly tie into systems built on generic code. The problem with autodiff is it doesn't necessarily take all branches to know how to deal with these kinds of issues, but having the power to fix it yourself is crucial.
JadeNB 8y ago
> Clearly y'(0) = 0 here, since at x=0 the function is defined to be a constant. Every function is constant at every point. For example, I could define the identity function as `y ( x ) = if ( x > 0 ) { x } elsif ( x < 0 ) { x } else { 0 }`, but that doesn't mean that its derivative at the origin is `0`.
- improbable22 8y ago
  No, dataflow's function y is defined in three smooth pieces, one of which is a constant. That's what this differentiation is seeing. It is evaluating at x+ϵ, and at 0+ϵ takes the ==0 branch which returns always 0, which has gradient 0. Your function also has three pieces, and the only difference from `y ( x ) = x` is that you've chosen to specify a derivative at zero, i.e. y(x+ϵ) = if ( x==0 ) { 0+0ϵ } else { x+1ϵ }. Which you may have reason to do, say to tell your optimiser to stop pushing on the wall.

ChrisRackauckas 8y ago

Yeah, this is a good case to use multiple dispatch to flexibly tie into systems built on generic code. The problem with autodiff is it doesn't necessarily take all branches to know how to deal with these kinds of issues, but having the power to fix it yourself is crucial.

JadeNB 8y ago

> Clearly y'(0) = 0 here, since at x=0 the function is defined to be a constant. Every function is constant at every point. For example, I could define the identity function as `y ( x ) = if ( x > 0 ) { x } elsif ( x < 0 ) { x } else { 0 }`, but that doesn't mean that its derivative at the origin is `0`.

improbable22 8y ago
No, dataflow's function y is defined in three smooth pieces, one of which is a constant. That's what this differentiation is seeing. It is evaluating at x+ϵ, and at 0+ϵ takes the ==0 branch which returns always 0, which has gradient 0. Your function also has three pieces, and the only difference from `y ( x ) = x` is that you've chosen to specify a derivative at zero, i.e. y(x+ϵ) = if ( x==0 ) { 0+0ϵ } else { x+1ϵ }. Which you may have reason to do, say to tell your optimiser to stop pushing on the wall.