3 ms·
You seem somewhat obsessed with the idea that reverse-mode autodiff is not the same technique as forward-mode autodiff. It makes you,,, angry? Seems like such a
by samatman 3y ago
You seem somewhat obsessed with the idea that reverse-mode autodiff is not the same technique as forward-mode autodiff. It makes you,,, angry? Seems like such a trivial thing to act a complete fool over.
What's up with that?
Anyway, here's a forward differentiation package with a file that might interest you
https://github.com/JuliaDiff/ForwardDiff.jl/blob/master/src/dual.jl https://github.com/JuliaDiff/ForwardDiff.jl/blob/master/src/...
- fpgamlirfanboy 3y agoit's amazing to me that pointing out a straight up mathematically factual inaccuracy is considered "angry" and "acting a fool".
- samatman 3y ago> there's a reason no one uses dual numbers (non-standard analysis) for anything (neither autodiff nor calculus itself) wrong > there are no [dual] numbers anywhere in [...] any other serious autodiff implementation wrong > please show me the source line number did > i'm wrong and this other guy is right correct. > this is why i hate this kind of "TIL, gee whiz" math tidbits acting a fool. angrily so.
- fpgamlirfanboy 3y agois this like some kind of version of truman show? the full sentence is >please show me the source line number in pytorch or tensorflow that defines this number why? because the original comment makes a claim about pytorch. it's all right there in black and white.
- samatman 3y agoWhich claim, this one? > Most autodiff packages (such as Pytorch) use something not much more advanced than this, although there are optimizations to speed it up (e.g. reverse mode diff). We get it: you're a mechanic who thinks that makes you an automotive engineer.
- fpgamlirfanboy 3y agoi haven't a clue what that means but when julia's autodiff is the only one that is implemented using dual numbers i think the claim is pretty obviously false and seemingly everyone has an issue admitting that. without a doubt you'll just say something else sarcastic rather than admit it.
- patrick451 3y agoI can't reply to the guy saying julia is the only one. But there are others. Ceres uses dual numbers https://github.com/ceres-solver/ceres-solver/blob/master/include/ceres/jet.h#L31 https://github.com/ceres-solver/ceres-solver/blob/master/inc... This library from google is used everywhere in robotics, so it's hardly some backwater little side project. So does c++ autodiff https://github.com/autodiff/autodiff/blob/main/autodiff/forward/dual/dual.hpp https://github.com/autodiff/autodiff/blob/main/autodiff/forw... So does Eigen: https://eigen.tuxfamily.org/dox/unsupported/AutoDiffScalar_8h_source.html https://eigen.tuxfamily.org/dox/unsupported/AutoDiffScalar_8...
- MikeBattaglia 3y agoSo does Pytorch: https://pytorch.org/tutorials/intermediate/forward_ad_usage.html https://pytorch.org/tutorials/intermediate/forward_ad_usage....