4 ms·
do you know how modern ml works? do you know that every functional unit in a net needs to be differentiable and so needs to carry around either dual numbers (fo
by mlevental 8y ago
do you know how modern ml works? do you know that every functional unit in a net needs to be differentiable and so needs to carry around either dual numbers (forward mode) or adjoints (reverse mode)? it's not as simple as just writing a math library.
- ViralBShah 8y agoThere's an extensive discussion of AD in the slides (the topic of this HN post), and how it is done in Julia. Precisely because it is not as simple as writing a math library is why you need language and compiler support.
- mlevental 8y agoyes that's what i was pointing out the poster i responded to
- sitkack 8y agoPlease give the parent a benefit of the doubt. Do you even comments are alienating. It could have been written as Every functional unit in a net needs to be differentiable and so needs to carry around either dual numbers (forward mode) or adjoints (reverse mode). These requirements necessitate new languages and compilers that current imperative representations don’t currently support.