4 ms·

Why are you calculating the Jacobian symbolically? AFAIK, for complex cases, the numerical Jacobian is often faster / more numerically stable.

funks_ 3y ago
Especially if you actually require vector-Jacobian or Jacobian-vector products instead of the full Jacobian.
eutectic 3y ago
I though finite difference differentiation was notoriously unstable.
- legobmw99 3y ago
  Perhaps the commenter means something like reverse mode automatic differentiation? Finite differences does indeed have stability issues, and even if you apply some tricks will only give you about half float precision
mitthrowaway2 3y ago
The poster is talking about using symbolic math to obtain a closed-form expression for the Jacobian that is then numerically evaluated. This will often be faster and more accurate. For example, if your function is sin(x), then the symbolic math tells you your derivative is cos(x), so you put that expression in your code and compile it. When calling this on the angle x = 0.123, your code then just evaluates numerically cosf(0.123), rather than (sinf(0.124) - sinf(0.122)).
toolslive 3y ago
There are plenty of situations where the scenario goes like this: you first do symbolical calculations. then fill in some of the free variables, your resulting expression will simplify a lot. Then you compile that resulting expression to native code and evaluate the simplified specialized code for a zillion parameter vectors. Anyway,if your day job needs something like this you're better off using a lisp than python.
- thebiss 3y ago
  For those of us that have never touched lisp - can you explain why?
  - toolslive 3y ago
    Lisp has quote, unquote, and friends allowing you to turn code into data, then manipulate it and then turn it back into code. These are all the tools you need to create on-the-fly compilers. Here's an article explaining the lost art: http://gms.freeshell.org/peval.html http://gms.freeshell.org/peval.html
abricq 3y ago
Well, the symbolic computation is only used to generate C code. The generated code then evaluates jacobians numerically. And we use several methods for numerical stability on top of that.

funks_ 3y ago

Especially if you actually require vector-Jacobian or Jacobian-vector products instead of the full Jacobian.

eutectic 3y ago

I though finite difference differentiation was notoriously unstable.

legobmw99 3y ago
Perhaps the commenter means something like reverse mode automatic differentiation? Finite differences does indeed have stability issues, and even if you apply some tricks will only give you about half float precision

mitthrowaway2 3y ago

The poster is talking about using symbolic math to obtain a closed-form expression for the Jacobian that is then numerically evaluated. This will often be faster and more accurate. For example, if your function is sin(x), then the symbolic math tells you your derivative is cos(x), so you put that expression in your code and compile it. When calling this on the angle x = 0.123, your code then just evaluates numerically cosf(0.123), rather than (sinf(0.124) - sinf(0.122)).

toolslive 3y ago

There are plenty of situations where the scenario goes like this: you first do symbolical calculations. then fill in some of the free variables, your resulting expression will simplify a lot. Then you compile that resulting expression to native code and evaluate the simplified specialized code for a zillion parameter vectors. Anyway,if your day job needs something like this you're better off using a lisp than python.

thebiss 3y ago
For those of us that have never touched lisp - can you explain why?
- toolslive 3y ago
  Lisp has quote, unquote, and friends allowing you to turn code into data, then manipulate it and then turn it back into code. These are all the tools you need to create on-the-fly compilers. Here's an article explaining the lost art: http://gms.freeshell.org/peval.html http://gms.freeshell.org/peval.html

abricq 3y ago

Well, the symbolic computation is only used to generate C code. The generated code then evaluates jacobians numerically. And we use several methods for numerical stability on top of that.