4 ms·

In the fast.ai course, there are two or three amazing videos where Jeremy Howard explains the basic concepts of deep learning (convolutions in this case) using

In the fast.ai course, there are two or three amazing videos where Jeremy Howard explains the basic concepts of deep learning (convolutions in this case) using Excel spreadsheets, highly recommended. I agree with mkagenius, I've played around a lot with deep learning and like physics it suffers a lot from unnecessary jargon that hides relatively simple concepts, for example this random quote: >ReLU stands for rectified linear unit, and is a type of activation function. Mathematically, it is defined as y = max(0, x). Now everybody could just use plain English - if you have a negative number, set it to zero, in all other cases, keep the number - I don't know why someone has to use 'rectified linear unit' to describe this simple operation.

probably_wrong 8y ago
> I don't know why someone has to use 'rectified linear unit' to describe this simple operation. To quote some snarky comedian whose name escapes me, because words have meanings. The way I understand it, "rectified" here has an analogous in physics. Beyond saving characters, it also tells that the purpose of this function is similar to the one people use when dealing with current. If I were either a physicist or someone used to deal with wave functions, the title alone would tell me everything I need to know. And I bet the type of people who care about activation functions are the type of people who knows what rectifiers are. I would argue that learning NNs includes learning the jargon needed to speak about them, the same way learning the meaning of "pointer" is required for programming. On the other hand, maybe I'm just rationalizing the time I spent learning jargon.
icelancer 8y ago
> Now everybody could just use plain English - if you have a negative number, set it to zero, in all other cases, keep the number - I don't know why someone has to use 'rectified linear unit' to describe this simple operation. I agree this is often a problem, but "rectified linear unit" isn't a deep learning term that was made up, as you obviously know from your physics background - its origins are in AC/DC power conversion. Anyone who knows basic signal processing knows what a rectifier is, and something that employs that function is a unit of it. There's plenty of nonsensical complexity in machine learning, but rectifiers and sigmoids and other 10-cent words describing maths aren't overly complicated, in my opinion. Lastly, saying "if you have a negative number, set it to zero, in all other cases, keep the number" is a lot more complicated and tedious than just saying "apply a rectifier." Would "ramp function" be acceptably plain?
anon946 8y ago
Intuitive descriptions using words have a place, and I do wish there were more of that. However, if you favor words over math notation, at some point you still have to call it something. What would you call it?
- deleted 8y ago
  [deleted]

probably_wrong 8y ago

> I don't know why someone has to use 'rectified linear unit' to describe this simple operation. To quote some snarky comedian whose name escapes me, because words have meanings. The way I understand it, "rectified" here has an analogous in physics. Beyond saving characters, it also tells that the purpose of this function is similar to the one people use when dealing with current. If I were either a physicist or someone used to deal with wave functions, the title alone would tell me everything I need to know. And I bet the type of people who care about activation functions are the type of people who knows what rectifiers are. I would argue that learning NNs includes learning the jargon needed to speak about them, the same way learning the meaning of "pointer" is required for programming. On the other hand, maybe I'm just rationalizing the time I spent learning jargon.

icelancer 8y ago

> Now everybody could just use plain English - if you have a negative number, set it to zero, in all other cases, keep the number - I don't know why someone has to use 'rectified linear unit' to describe this simple operation. I agree this is often a problem, but "rectified linear unit" isn't a deep learning term that was made up, as you obviously know from your physics background - its origins are in AC/DC power conversion. Anyone who knows basic signal processing knows what a rectifier is, and something that employs that function is a unit of it. There's plenty of nonsensical complexity in machine learning, but rectifiers and sigmoids and other 10-cent words describing maths aren't overly complicated, in my opinion. Lastly, saying "if you have a negative number, set it to zero, in all other cases, keep the number" is a lot more complicated and tedious than just saying "apply a rectifier." Would "ramp function" be acceptably plain?

anon946 8y ago

Intuitive descriptions using words have a place, and I do wish there were more of that. However, if you favor words over math notation, at some point you still have to call it something. What would you call it?

deleted 8y ago
[deleted]