3 ms·
The idea of composing complicated differentiable programs (that have control structures, recursion, etc.) from simpler differentiable programs by means of an ab
by cs702 10y ago
The idea of composing complicated differentiable programs (that have control structures, recursion, etc.) from simpler differentiable programs by means of an abstract algebra of differentiable programs is truly mind-bending.
It upends the conventional concept I have of "programming" for day-to-day purposes. It also upends the conventional concept I have of "designing" (i.e., handcrafting) machine learning models, like deep neural nets, for different problems.
Like you, I don't yet grok all the nitty-gritty details of the paper (it will take a me a long time to grok them), but it's clear to me that if the paper survives peer review (which I would expect, given that the authors have published working code), you're right: its formalization has a lot of potential for the fields of machine learning, AI, and computer science.
Holy cow.
EDIT: Removed the "could revolutionize" bit, which was over-enthusiastic.
- m_hack 10y agoAgreed. The paper seems to be about a lot more than AD, as AD isn't really the main subject. This approach to programming really does strike me as new and important. I too havent yet grasped all the details, having spent a few hours on it today. I commented a short synopsis bellow. I will try to get a better grasp of it tomorrow.