3 ms·

Something I don't understand about Automatic Differentiation is: Why not use a Computer Algebra System instead for generating derivatives of given functions?

by ktpsns 8y ago

salty_biscuits 8y ago
An analytic form of the derivative might not be able to be found by computer algebra or expressible in a compact form.
enriquto 8y ago
You can still do that, most of the times. But if your function has many conditionals the final expression may get cumbersome. Sometimes, the symbolic derivatives will be faster to compute, but most often they will be slower than using dual numbers for example. It is just a different tool to solve the problem. Most often, it is easier to use: you do not need a computer algebra system, you just change the type of your numbers and call it a day.
sampo 8y ago
> Why not use a Computer Algebra System instead for generating derivatives of given functions? You will generate expressions, and evaluating those expressions leads to the execution of exactly the same elementary operations as with forward mode AD. But with the overhead of needing the Computer Algebra System, and storing the expressions. (Ok, in some cases the CAS is maybe able to simplify the expressions a bit.)
one-more-minute 8y ago
That's essentially what a source-to-source AD does, just with support for the extra features that show up in programming languages. For example, handling variable bindings gets you the typical Wengert list, and handling function calls gets you the Pearlmutter and Siskind style backpropagator (I wrote a bit about the relationships at [0]). The short answer is that CAS systems work with a "programming language" that doesn't have these features and is therefore a bit too limited for the kinds of models we're interested in. [0] https://github.com/MikeInnes/diff-zoo https://github.com/MikeInnes/diff-zoo
deleted 8y ago
[deleted]
wenc 8y ago
CASes generate symbolic derivative expressions. This is typically a fairly expensive operation, and can be slow if the number of expressions are large or if the expressions are complicated. You have to pre-generate these expressions before you can do any work (paying an upfront cost). CAS'es also use a very general symbolic engine, which entails a performance penalty. If you're doing numerical work and don't need explicit derivative expressions, automatic differentiation offers a method that is both fast and convenient.

Something I don't understand about Automatic Differentiation is: Why not use a Computer Algebra System instead for generating derivatives of given functions?

by ktpsns 8y ago

Something I don't understand about Automatic Differentiation is: Why not use a Computer Algebra System instead for generating derivatives of given functions?

salty_biscuits 8y ago

An analytic form of the derivative might not be able to be found by computer algebra or expressible in a compact form.

enriquto 8y ago

You can still do that, most of the times. But if your function has many conditionals the final expression may get cumbersome. Sometimes, the symbolic derivatives will be faster to compute, but most often they will be slower than using dual numbers for example. It is just a different tool to solve the problem. Most often, it is easier to use: you do not need a computer algebra system, you just change the type of your numbers and call it a day.

sampo 8y ago

> Why not use a Computer Algebra System instead for generating derivatives of given functions? You will generate expressions, and evaluating those expressions leads to the execution of exactly the same elementary operations as with forward mode AD. But with the overhead of needing the Computer Algebra System, and storing the expressions. (Ok, in some cases the CAS is maybe able to simplify the expressions a bit.)

one-more-minute 8y ago

That's essentially what a source-to-source AD does, just with support for the extra features that show up in programming languages. For example, handling variable bindings gets you the typical Wengert list, and handling function calls gets you the Pearlmutter and Siskind style backpropagator (I wrote a bit about the relationships at [0]). The short answer is that CAS systems work with a "programming language" that doesn't have these features and is therefore a bit too limited for the kinds of models we're interested in. [0] https://github.com/MikeInnes/diff-zoo https://github.com/MikeInnes/diff-zoo

deleted 8y ago

[deleted]

wenc 8y ago

CASes generate symbolic derivative expressions. This is typically a fairly expensive operation, and can be slow if the number of expressions are large or if the expressions are complicated. You have to pre-generate these expressions before you can do any work (paying an upfront cost). CAS'es also use a very general symbolic engine, which entails a performance penalty. If you're doing numerical work and don't need explicit derivative expressions, automatic differentiation offers a method that is both fast and convenient.