3 ms·
Not sure what this means - pytorch's autodiff is reverse mode accumulation. You can read pazschke's paper on it or just go look at the implementation.
by tubby12345 5y ago
Not sure what this means - pytorch's autodiff is reverse mode accumulation. You can read pazschke's paper on it or just go look at the implementation.
- fault1 5y agoI'm talking about forward - not reverse mode accumulation, since forward is VERY important for hpc. you ideally want 'forward over reverse' and other types of hybrid ad as well. it does look like a lot of progress has been made lately, but it isn't in a release yet: https://github.com/pytorch/pytorch/issues/10223 https://github.com/pytorch/pytorch/issues/10223