4 ms·

Deep Learning Algorithms: The Complete Guide

107 points by sergioskar 7y ago

Hydraulix989 7y ago
No deep reinforcement learning?
rckoepke 7y ago
Mainly I take issue with the title "...: The Complete Guide". More accurately, its a very abbreviated overview. The table of contents has a fairly reasonable list of topics, but for a complete guide I'd want each section there to be at least an entire chapter of material. Instead they are 4-6 sentences per section. Personally, I think that any comprehensive guide/textbook for machine learning, especially deep learning, should contain a chapter on the mathematics of control systems theory. The feedback/back-propagation is very similar to what EE's and ChemE's do with PID loops or Op-Amp tuning, and I feel that so much is lost academically when that topic is glossed over.
- SubiculumCode 7y ago
  I liked the birds eye view of this article, but that title
- jjoonathan 7y ago
  Yeah, pretty much everyone can benefit from a high-level grasp of phase margin and gain margin. Even in contexts where exact calculations are impossible, it's important to remember that cranking the gain too high turns your amplifier into an oscillator and that lengthening the feedback cycle reduces the gain ceiling above which this happens. Typically, I find that people have an intuitive understanding of the possibility of overshoot but not of oscillation and almost never of the connection between feedback lag and overshoot/oscillation.
- joe_the_user 7y ago
  I'm not even sure Deep Learning is at the level where a title like "The Complete Guide" could make sense. Deep Learning involves harnessing advanced, highly complex and rather ad-hoc algorithm to engage in systematic-but-heuristic prediction. A complete guide would include all natural prerequisites, the common approaches, the best practices and the areas of application. But all of these are in flux as the field races ahead. Moreover, the field still needs "artists", practitioners who can figure out the black-art of training networks. So whatever it's virtues, the field today seems incomplete to me.
- AgentMatt 7y ago
  Do you have a recommendation for a book explaining the relevant control systems concepts?
  - deleted 7y ago
    [deleted]
- throwqwerty 7y ago
  >The feedback/back-propagation is very similar i really wish people wouldn't say these things just to sound smart. it confuses people that don't know better. feedback loops have nothing to do with backprop, which is just a way to factorize the jacobian into matrix vector products instead of matrix matrix products. do you have some proof (and it would require a proof) that PID controllers actually compute the gradient of a function with respect to n-inputs? because that's what backprop does.
  - rckoepke 7y ago
    Thank you, non-sarcastically. Would edit if I could.
throwqwerty 7y ago
i've said it before and i'll say it again: this is content marketing (for some kind of bs courses or something). why are you people upvoting this

107 points by sergioskar 7y ago

No deep reinforcement learning?

Mainly I take issue with the title "...: The Complete Guide". More accurately, its a very abbreviated overview. The table of contents has a fairly reasonable list of topics, but for a complete guide I'd want each section there to be at least an entire chapter of material. Instead they are 4-6 sentences per section. Personally, I think that any comprehensive guide/textbook for machine learning, especially deep learning, should contain a chapter on the mathematics of control systems theory. The feedback/back-propagation is very similar to what EE's and ChemE's do with PID loops or Op-Amp tuning, and I feel that so much is lost academically when that topic is glossed over.

SubiculumCode 7y ago
I liked the birds eye view of this article, but that title
jjoonathan 7y ago
Yeah, pretty much everyone can benefit from a high-level grasp of phase margin and gain margin. Even in contexts where exact calculations are impossible, it's important to remember that cranking the gain too high turns your amplifier into an oscillator and that lengthening the feedback cycle reduces the gain ceiling above which this happens. Typically, I find that people have an intuitive understanding of the possibility of overshoot but not of oscillation and almost never of the connection between feedback lag and overshoot/oscillation.
joe_the_user 7y ago
I'm not even sure Deep Learning is at the level where a title like "The Complete Guide" could make sense. Deep Learning involves harnessing advanced, highly complex and rather ad-hoc algorithm to engage in systematic-but-heuristic prediction. A complete guide would include all natural prerequisites, the common approaches, the best practices and the areas of application. But all of these are in flux as the field races ahead. Moreover, the field still needs "artists", practitioners who can figure out the black-art of training networks. So whatever it's virtues, the field today seems incomplete to me.
AgentMatt 7y ago
Do you have a recommendation for a book explaining the relevant control systems concepts?
- deleted 7y ago
  [deleted]
throwqwerty 7y ago
>The feedback/back-propagation is very similar i really wish people wouldn't say these things just to sound smart. it confuses people that don't know better. feedback loops have nothing to do with backprop, which is just a way to factorize the jacobian into matrix vector products instead of matrix matrix products. do you have some proof (and it would require a proof) that PID controllers actually compute the gradient of a function with respect to n-inputs? because that's what backprop does.
- rckoepke 7y ago
  Thank you, non-sarcastically. Would edit if I could.

throwqwerty 7y ago

i've said it before and i'll say it again: this is content marketing (for some kind of bs courses or something). why are you people upvoting this