4 ms·
Part of the challenge is that most formulations of (reverse mode )auto diff wind up requiring extra runtime data structures for the backwards computation step.
by carterschonwald 5y ago
Part of the challenge is that most formulations of (reverse mode )auto diff wind up requiring extra runtime data structures for the backwards computation step.
There’s been some great work in this space in the past 5 years.
I’ve got some stuff I worked out this fall I’m overdue to write up and share some prototypes : there is a way to do reverse mode auto diff isolated to just being an invisible compiler pass! Without any of the extra complexity in what are otherwise equivalent formulations
- The_rationalist 5y agohttps://github.com/breandan/kotlingrad#coroutines https://github.com/breandan/kotlingrad#coroutines
- carterschonwald 5y agoThe coroutines link doesn’t work. And that aside it has a different strategy than the one I’ve been playing with ;)
- bmc7505 5y agoHey! Thanks for the link. We decided to postpone our coroutine-based implementation until native support for multi-shot delimited continuations becomes more stable. The linked section may be found here [1], for posterity. [1]: https://github.com/breandan/kotlingrad/tree/c02ac55325c05a2ecee6ec26b24bbd61c138e7e2#coroutines https://github.com/breandan/kotlingrad/tree/c02ac55325c05a2e...