5 ms·

The most next gen autodiff library probably is https://github.com/breandan/kotlingrad https://github.com/breandan/kotlingrad because of its features, ergonomy a

The most next gen autodiff library probably is https://github.com/breandan/kotlingrad https://github.com/breandan/kotlingrad because of its features, ergonomy and type safety

martinsmit 4y ago
Idk, Enzyme is pretty next gen, all the way down to LLVM code. https://github.com/EnzymeAD/Enzyme https://github.com/EnzymeAD/Enzyme
- mhh__ 4y ago
  So enzyme is pretty cool but IMO the documentation is terrible and the API is absolutely insane.
  - wmoses 4y ago
    Enzyme dev here. The documentation is not just bad, it is in fact actively missing for several things (like forward mode AD, split mode, recent vector). Having recently finished up a number of big technical challenges (forward, vector, split, BLAS), we're hoping to have a big documentation-athon at the end of summer (all are welcome)!
    - SemanticStrengh 4y ago
      On HN I am often seeing libraries/blog authors react very quickly to references of their projects. I'm curious, how do you achieve this, is it a coincidence and where you reading HN comments, or do you have a crwaler/API than send you a notification when your project github link is found in a new HN comment ?
      - wmoses 4y ago
        I was actually socializing with folks at the Cambridge Area Julia meetup where I gave a talk on Enzyme’s Julia bindings no less when someone shared this. So in this case pure coincidence.
        SemanticStrengh 4y ago
        Thanks for the data point :)
        adgjlsfhk1 4y ago
        Did you catch your plane?
        wmoses 4y ago
        I did!
    - mhh__ 4y ago
      Interesting. I'm mulling over integrating Enzyme into the LLVM D compiler so I may/will join in with that.
      - wmoses 4y ago
        Do it! Would be happy to help. Also we have weekly development meetings (time about to change for summer), which you’re welcome to join. Also feel free to email me (wmoses@mit.edu), would be happy to chat and or help bootstrap.
bmc7505 4y ago
Author here. You are very kind, but Kotlin∇ is definitely a work-in-progress and its claimed contributions are more modest in nature. It is true that Kotlin∇ was one of the first AD frameworks to introduce shape-safe tensor arithmetic, however we were not the only ones to realize the importance of this concept, which has since been adopted by more powerful type systems (e.g., Dex and Hasktorch). Right now, our efforts have mostly migrated from gradient-based machine learning in continuous spaces to generalized differentiation over finite fields (details forthcoming in our ARRAY'22 presentation), which allows us to take derivatives in discrete spaces such as bounded-width regular and context free languages. Our findings indicate that generalized AD has practical applications to robust-parsing, error-correcting codes, type inference and program synthesis.

The most next gen autodiff library probably is https://github.com/breandan/kotlingrad https://github.com/breandan/kotlingrad because of its features, ergonomy and type safety

martinsmit 4y ago

Idk, Enzyme is pretty next gen, all the way down to LLVM code. https://github.com/EnzymeAD/Enzyme https://github.com/EnzymeAD/Enzyme

mhh__ 4y ago
So enzyme is pretty cool but IMO the documentation is terrible and the API is absolutely insane.
- wmoses 4y ago
  Enzyme dev here. The documentation is not just bad, it is in fact actively missing for several things (like forward mode AD, split mode, recent vector). Having recently finished up a number of big technical challenges (forward, vector, split, BLAS), we're hoping to have a big documentation-athon at the end of summer (all are welcome)!
  - SemanticStrengh 4y ago
    On HN I am often seeing libraries/blog authors react very quickly to references of their projects. I'm curious, how do you achieve this, is it a coincidence and where you reading HN comments, or do you have a crwaler/API than send you a notification when your project github link is found in a new HN comment ?
    - wmoses 4y ago
      I was actually socializing with folks at the Cambridge Area Julia meetup where I gave a talk on Enzyme’s Julia bindings no less when someone shared this. So in this case pure coincidence.
      - SemanticStrengh 4y ago
        Thanks for the data point :)
      - adgjlsfhk1 4y ago
        Did you catch your plane?
        wmoses 4y ago
        I did!
  - mhh__ 4y ago
    Interesting. I'm mulling over integrating Enzyme into the LLVM D compiler so I may/will join in with that.
    - wmoses 4y ago
      Do it! Would be happy to help. Also we have weekly development meetings (time about to change for summer), which you’re welcome to join. Also feel free to email me (wmoses@mit.edu), would be happy to chat and or help bootstrap.

bmc7505 4y ago

Author here. You are very kind, but Kotlin∇ is definitely a work-in-progress and its claimed contributions are more modest in nature. It is true that Kotlin∇ was one of the first AD frameworks to introduce shape-safe tensor arithmetic, however we were not the only ones to realize the importance of this concept, which has since been adopted by more powerful type systems (e.g., Dex and Hasktorch). Right now, our efforts have mostly migrated from gradient-based machine learning in continuous spaces to generalized differentiation over finite fields (details forthcoming in our ARRAY'22 presentation), which allows us to take derivatives in discrete spaces such as bounded-width regular and context free languages. Our findings indicate that generalized AD has practical applications to robust-parsing, error-correcting codes, type inference and program synthesis.