4 ms·

How does this compare to zygote ?

• cbkeller 6y ago
  As far as I understand it (which is not very far): Zygote calculates derivatives using source-to-source automatic differentiation. This calculates function inverses (so to stretch the analogy a bit, it's kinda like "source-to-source automatic inversion")
  • ChrisRackauckas 6y ago
    Not necessarily: you can use it to build fast inverse of scalar functions. Here's an example on a vjp of an ODE f: https://gist.github.com/GiggleLiu/0c4608f70bda050f59992f5fc095e2cb https://gist.github.com/GiggleLiu/0c4608f70bda050f59992f5fc0...
    • cbkeller 6y ago
      I'm probably just missing something, but did you mean to say "fast derivatives of scalar functions" by any chance? I.e., that you can use the free inverses to get at the derivatives in some way?
      • ChrisRackauckas 6y ago
        oh yes, I meant derivatives.
• GiggleLiu 6y ago
  1. Scalar level Various benchmarks (including those in the paper) show NiLang is much better than Zygote to differentiate scalar functions. And Zygote is much faster than TF and PyTorch. 2. Tensor level Zygote, TF and PyTorch are much better than NiLang, because NiLang's matrix multiplication is not fully optimized, it is much slower than BLAS. (One can wrap BLAS into NiLang, but that does not measure NiLang's programming language level AD performance anymore)