4 ms·
How ... whatever was the reason for thinking that you needed a ML-specific compiler? Do people also think you need a timecard-tracking-specific compiler?
by SilasX 8y ago
How ... whatever was the reason for thinking that you needed a ML-specific compiler?
Do people also think you need a timecard-tracking-specific compiler?
- KenoFischer 8y agoWell, I do think there are some very real shortcomings that current general purpose compilers have when applied to machine learning (aggrevated by the fact that lots of machine learning code is written in languages with poor compiler support), so lots people looked at that, wrote small optimizers and got good performance. But then it turned out that researchers wanted to do more and more things in their ML models and these small optimizers are turning into full-blown compilers, with not a lot of thought about whether that is truly the correct thing to do.
- mlevental 8y agodo you know how modern ml works? do you know that every functional unit in a net needs to be differentiable and so needs to carry around either dual numbers (forward mode) or adjoints (reverse mode)? it's not as simple as just writing a math library.
- ViralBShah 8y agoThere's an extensive discussion of AD in the slides (the topic of this HN post), and how it is done in Julia. Precisely because it is not as simple as writing a math library is why you need language and compiler support.
- mlevental 8y agoyes that's what i was pointing out the poster i responded to
- sitkack 8y agoPlease give the parent a benefit of the doubt. Do you even comments are alienating. It could have been written as Every functional unit in a net needs to be differentiable and so needs to carry around either dual numbers (forward mode) or adjoints (reverse mode). These requirements necessitate new languages and compilers that current imperative representations don’t currently support.
- chrisseaton 8y ago> How ... whatever was the reason for thinking that you needed a ML-specific compiler? As the slides say, you need the program to be differentiable. Do general purpose compilers make it easy to automatically differentiate a program? No, not until very recent work. So people thought a better idea until we figured out how to do that was ML-specific compilers and frameworks. > Do people also think you need a timecard-tracking-specific compiler? No, nobody thinks this, because timecard-tracking does not need any special properties such as being differentiable. The answer isn't as crazy as your extremely snarky question makes it out to be, is it?
- samcodes 8y agoGood point, timecard-tracking software and general purpose differentiable computing languages are roughly the same level of complexity.