3 ms·
Unfortunately, this isn't super easy since the time complexity heavily depends on what optimizations the compiler will apply and how the higher order AD is exac
by KenoFischer 7y ago
Unfortunately, this isn't super easy since the time complexity heavily depends on what optimizations the compiler will apply and how the higher order AD is exactly implemented (there's many ways to do so that all give you the same answer and you probably want the system to pick the best for you). What we could do fairly easily however is to have a compiler introspection tool that runs through the computation, but doesn't do any of the work, just record how much work it would have done. That would easily give you and idea and let you plot a parameter sweep over any dimension you care about with actual results corresponding to the capabilities of the system.
- jknz 7y agoThanks for the reply. Such introspection tool would be useful indeed--it seems already possible to do some inspection by hand with https://fluxml.ai/Zygote.jl/latest/adjoints/#Gradient-Reflection-1 https://fluxml.ai/Zygote.jl/latest/adjoints/#Gradient-Reflec... Anyway, great work and I look forward to what happens next!