3 ms·
The optimal control framework Casadi has had the ability to compute sparse jacobians and hessians for a long time (maybe a decade?), which come up all the time
by patrick451 1y ago
The optimal control framework Casadi has had the ability to compute sparse jacobians and hessians for a long time (maybe a decade?), which come up all the time in trajectory optimization.This not only provides massive speed ups in both the differentiation and linear solver time, but also greatly reduces the memory requirements. If this catches on in machine learning, it will be interesting to see if we can finally move past first order optimization methods.
- gdalle 1y agoIndeed, CasADi is among the precursors in this area! The key difference with our approach is their use of a domain-specific language, with distinct mathematical functions and array types. This has lots of benefits, but it expects users to rewrite their existing code in the CasADi formalism. What we seek to achieve in Julia is compatibility with native code, without a DSL-imposed refactor. We share this ambition with the broader Julia autodiff ecosystem, which is focused on differentiating the language as a whole. Of course it doesn't always work, but in many cases, it enables a plug-and-play approach to (sparse) autodiff which makes really cool applications possible.