4 ms·
Yes, definitely, there are even battle-tested implementations for Fortran available. Though I have never seen AD frameworks used in the production contexts for
by IngoBlechschmid 8y ago
Yes, definitely, there are even battle-tested implementations for Fortran available.
Though I have never seen AD frameworks used in the production contexts for neural networks/backpropagation. As you say, the code for this seems to be mostly handrolled. Please take this negative statement with a grain of salt, I don't actually work in machine learning.
- kxyvr 8y agoBackpropagation is literally the reverse mode AD applied to neural nets. That said, I wish had a paper that I could point to that shows explicitly that. From what I can tell, the AD community has known this basically since the start, but for whatever reason backpropogation is taught rather than the more general technique.