4 ms·

Not-o-matic Differentiation

50 points by jxub 8y ago

mlevental 8y ago
this might be the right place to ask: can you differentiate a recursively defined function?
- im3w1l 8y ago
  If I understand you correctly: yes. You differentiate both sides and then get an equation for the derivative.
  - deleted 8y ago
    [deleted]
- throwawayjava 8y ago
  Yes. You can also integrate a recursively defined function(s). E.g., recall your undergraduate differential equations course.
  - mlevental 8y ago
    not sure what this means?
    - EGreg 8y ago
      It means that technique is taught in the undergrad course.
      - mlevental 8y ago
        yes i'm asking which technique - the only thing i remember that vaguely resembles this is power series solutions
- danmg 8y ago
  yes. it generally won't have a closed form however.
  - deleted 8y ago
    [deleted]
- edflsafoiewq 8y ago
  Let's try pow x 0 = 1 pow x n = x * pow x (n-1) ergo pow' x 0 = 0 pow' x n = pow x (n-1) + x * pow' x (n-1) ergo pow' x 1 = pow x 0 + x * pow' x 0 = 1 + x * 0 = 1 pow' x 2 = pow x 1 + x * pow' x 1 = x + x = 2 * x pow' x 3 = pow x 2 + x * pow' x 2 = x * x + 2 * x * x = 3 * x * x
  - deleted 8y ago
    [deleted]
- perl4ever 8y ago
  Isn't differentiation itself a recursively defined function?
  - tome 8y ago
    Yes, but good luck differentiating differentiation!
joe_the_user 8y ago
I'm trying to figure why the person making the post failed. There are existing Haskell libraries for automatic differentiation [1]. As opposed to ordinary symbolic differentiation or numerical differentiation, automatic differentiation is a method that takes the code of a function and a value and returns the derivative of the function at run time. Of course, the author is working with a vector library and so his version will take N-dimension vectors/tensors and a functions of these and return N+something dimension vectors/tensors. That part is kind of head spinning and I assume that's where things fell apart. [1] http://hackage.haskell.org/package/ad http://hackage.haskell.org/package/ad
- eastWestMath 8y ago
  Well if you look at the categorical semantics of differentiation, it takes in a function A -> B and spits out AxA -> B. It’s a fairly well developed area of logic/type theory (even at the level of term rewriting systems), it seems the author of the blog post missed it. I probably would have emailed him some references if I’d seen the initial announcement...
- nmca 8y ago
  Because this was intended to support `accelerate` and uses the variable-mode ad approach presented in 'the simple essence of automatic differentiation' which afaik has not been implemented in conjunction with `accelerate` . (Your comment reads quite uncharitable/mean btw, hard to tell if it was intentional)
  - tree_of_item 8y ago
    > (Your comment reads quite uncharitable/mean btw, hard to tell if it was intentional) Really? I didn't get that at all. Was it edited?
eastWestMath 8y ago
There’s a fairly large body of work on the differential lambda calculus that you seem to have missed...
- heyjudy 8y ago
  My HP 48G(X) from the 1990's did automatic symbolic differentiation and integration. And, IIRC, there's probably a textbook in either Fortran or C for symbolic mathematical evaluation (partial evaluation, simplification and Monte Carlo fallback) and transformations. We had to deal a lot with this in the code used by a nuclear reactor simulator (in Fortran) that effectively did glorified finite element analysis with some actual direct calculations here and there.
  - eastWestMath 8y ago
    Huh?

50 points by jxub 8y ago

mlevental 8y ago

this might be the right place to ask: can you differentiate a recursively defined function?

im3w1l 8y ago
If I understand you correctly: yes. You differentiate both sides and then get an equation for the derivative.
- deleted 8y ago
  [deleted]
throwawayjava 8y ago
Yes. You can also integrate a recursively defined function(s). E.g., recall your undergraduate differential equations course.
- mlevental 8y ago
  not sure what this means?
  - EGreg 8y ago
    It means that technique is taught in the undergrad course.
    - mlevental 8y ago
      yes i'm asking which technique - the only thing i remember that vaguely resembles this is power series solutions
danmg 8y ago
yes. it generally won't have a closed form however.
- deleted 8y ago
  [deleted]
edflsafoiewq 8y ago
Let's try pow x 0 = 1 pow x n = x * pow x (n-1) ergo pow' x 0 = 0 pow' x n = pow x (n-1) + x * pow' x (n-1) ergo pow' x 1 = pow x 0 + x * pow' x 0 = 1 + x * 0 = 1 pow' x 2 = pow x 1 + x * pow' x 1 = x + x = 2 * x pow' x 3 = pow x 2 + x * pow' x 2 = x * x + 2 * x * x = 3 * x * x
- deleted 8y ago
  [deleted]
perl4ever 8y ago
Isn't differentiation itself a recursively defined function?
- tome 8y ago
  Yes, but good luck differentiating differentiation!

joe_the_user 8y ago

I'm trying to figure why the person making the post failed. There are existing Haskell libraries for automatic differentiation [1]. As opposed to ordinary symbolic differentiation or numerical differentiation, automatic differentiation is a method that takes the code of a function and a value and returns the derivative of the function at run time. Of course, the author is working with a vector library and so his version will take N-dimension vectors/tensors and a functions of these and return N+something dimension vectors/tensors. That part is kind of head spinning and I assume that's where things fell apart. [1] http://hackage.haskell.org/package/ad http://hackage.haskell.org/package/ad

eastWestMath 8y ago
Well if you look at the categorical semantics of differentiation, it takes in a function A -> B and spits out AxA -> B. It’s a fairly well developed area of logic/type theory (even at the level of term rewriting systems), it seems the author of the blog post missed it. I probably would have emailed him some references if I’d seen the initial announcement...
nmca 8y ago
Because this was intended to support `accelerate` and uses the variable-mode ad approach presented in 'the simple essence of automatic differentiation' which afaik has not been implemented in conjunction with `accelerate` . (Your comment reads quite uncharitable/mean btw, hard to tell if it was intentional)
- tree_of_item 8y ago
  > (Your comment reads quite uncharitable/mean btw, hard to tell if it was intentional) Really? I didn't get that at all. Was it edited?

eastWestMath 8y ago

There’s a fairly large body of work on the differential lambda calculus that you seem to have missed...

heyjudy 8y ago
My HP 48G(X) from the 1990's did automatic symbolic differentiation and integration. And, IIRC, there's probably a textbook in either Fortran or C for symbolic mathematical evaluation (partial evaluation, simplification and Monte Carlo fallback) and transformations. We had to deal a lot with this in the code used by a nuclear reactor simulator (in Fortran) that effectively did glorified finite element analysis with some actual direct calculations here and there.
- eastWestMath 8y ago
  Huh?