4 ms·

Learn Neural Networks with Go, Not Math [video]

40 points by ngaut 7y ago

codesushi42 7y ago
You have no hope of understanding neural networks without learning math. Luckily the math needed is not that hard. Linear algebra and gradient based optimization are not out of reach for most people. If you already understand regression and maximum likelihood estimation, then you're almost there already.
notus 7y ago
This title doesn't even make any sense. Golang is not going to magically shield you from the math and machine learning teaching methods that do attempt to shield you from the math recognize that it is just a stepping stone to understanding that eventually results in learning the math. That being said the math isn't really that difficult. It sounds complex but I was able to develop a working knowledge without having had taken a calculus or linear algebra course. Also what does Go magically do that any of the other abstractions over machine learning algorithms don't do? The answer is nothing. If anything you're just making it more difficult for beginners since the machine learning in golang ecosystem isn't fleshed out nor does it have the documentation trail that python does.
- codesushi42 7y ago
  Maybe they meant "without math notation". Though a more fitting title would have been "learn neural networks with this one weird trick" given the lack of substance here.
  - tptacek 7y ago
    It's pretty clear from the first few minutes of the talk that math notation is in fact what they're talking about, not math in general. I'm not saying it's a great talk, but this particular critique of it isn't persuasive.
    - codesushi42 7y ago
      You are right, a more accurate critique would simply have been "click bait".
- ralph1268 7y ago
  It is such a strange inclination to want to hide the math behind programming. I find this just makes things even more obtuse. Every time I have to read code that implements a numerical algorithm, I have no idea what is going on until I reverse engineer (so to speak) the code back into standard mathematical notation.
  - newen 7y ago
    True. I think it's a symptom of people who had no math education taking up programming and wanting to learn ML etc.
omarhaneef 7y ago
The temptation to learn neural nets "with" a programming language as opposed to math, is probably because programming is more fun. And the reason it is more fun is because you are getting instant feedback, you fix the error and get a dopamine kick when it passed that error and so forth. All the programmers here know exactly what I mean. The math, meanwhile, is harder because you might read description, stare at an equation. Hope that you "get" it. Then read the next statement, realize you might have misunderstood the previous one and so on. You don't get the same sense of progress, and one bit of confusion can throw you off track for a long time. One contribution would be teach the math in an environment that is like programming. I am not sure what this would look like, but you would learn how to back propagate the derivative through the network, minimizing error at each step and forward propagate the signal. It can't just be a visualization because the student won't engage with it in the same way as a system with negative and positive feedback.
- codesushi42 7y ago
  There are already countless resources that do this effectively. Overall I am not an Andrew Ng fan, but his lecture on neural network basics from deeplearning.ai gets the job done. I'm sure there are other resources that are even better. The video linked here is a pointless exercise. The speaker exclaims that learning formulas and notations isn't needed, but then she dives into an even more confusing code exercise that doesn't explain anything about what gradients are and why we use them. A total waste of time. Just explain the math.
defertoreptar 7y ago
I've learned that when I actually implement a process (and not just call from a library), that's when the idea really "clicks." Usually it's much simpler than I realized. I think it has to do with how, in programming, everything can be decomposed to concrete terms. If something seems vague, then it's possible to follow the execution of the code to find out what is actually happening. Written formulas can be very ambiguous. I get the feeling that authors prefer to leave out as much info as they can get away with just for the sake of having the formula look "elegant." Many times, it's assumed you know enough to fill in the gaps, and it can be frustrating when you don't know where to even find that info in the first place.

40 points by ngaut 7y ago

You have no hope of understanding neural networks without learning math. Luckily the math needed is not that hard. Linear algebra and gradient based optimization are not out of reach for most people. If you already understand regression and maximum likelihood estimation, then you're almost there already.

notus 7y ago

This title doesn't even make any sense. Golang is not going to magically shield you from the math and machine learning teaching methods that do attempt to shield you from the math recognize that it is just a stepping stone to understanding that eventually results in learning the math. That being said the math isn't really that difficult. It sounds complex but I was able to develop a working knowledge without having had taken a calculus or linear algebra course. Also what does Go magically do that any of the other abstractions over machine learning algorithms don't do? The answer is nothing. If anything you're just making it more difficult for beginners since the machine learning in golang ecosystem isn't fleshed out nor does it have the documentation trail that python does.

codesushi42 7y ago
Maybe they meant "without math notation". Though a more fitting title would have been "learn neural networks with this one weird trick" given the lack of substance here.
- tptacek 7y ago
  It's pretty clear from the first few minutes of the talk that math notation is in fact what they're talking about, not math in general. I'm not saying it's a great talk, but this particular critique of it isn't persuasive.
  - codesushi42 7y ago
    You are right, a more accurate critique would simply have been "click bait".
ralph1268 7y ago
It is such a strange inclination to want to hide the math behind programming. I find this just makes things even more obtuse. Every time I have to read code that implements a numerical algorithm, I have no idea what is going on until I reverse engineer (so to speak) the code back into standard mathematical notation.
- newen 7y ago
  True. I think it's a symptom of people who had no math education taking up programming and wanting to learn ML etc.

omarhaneef 7y ago

The temptation to learn neural nets "with" a programming language as opposed to math, is probably because programming is more fun. And the reason it is more fun is because you are getting instant feedback, you fix the error and get a dopamine kick when it passed that error and so forth. All the programmers here know exactly what I mean. The math, meanwhile, is harder because you might read description, stare at an equation. Hope that you "get" it. Then read the next statement, realize you might have misunderstood the previous one and so on. You don't get the same sense of progress, and one bit of confusion can throw you off track for a long time. One contribution would be teach the math in an environment that is like programming. I am not sure what this would look like, but you would learn how to back propagate the derivative through the network, minimizing error at each step and forward propagate the signal. It can't just be a visualization because the student won't engage with it in the same way as a system with negative and positive feedback.

codesushi42 7y ago
There are already countless resources that do this effectively. Overall I am not an Andrew Ng fan, but his lecture on neural network basics from deeplearning.ai gets the job done. I'm sure there are other resources that are even better. The video linked here is a pointless exercise. The speaker exclaims that learning formulas and notations isn't needed, but then she dives into an even more confusing code exercise that doesn't explain anything about what gradients are and why we use them. A total waste of time. Just explain the math.

defertoreptar 7y ago

I've learned that when I actually implement a process (and not just call from a library), that's when the idea really "clicks." Usually it's much simpler than I realized. I think it has to do with how, in programming, everything can be decomposed to concrete terms. If something seems vague, then it's possible to follow the execution of the code to find out what is actually happening. Written formulas can be very ambiguous. I get the feeling that authors prefer to leave out as much info as they can get away with just for the sake of having the formula look "elegant." Many times, it's assumed you know enough to fill in the gaps, and it can be frustrating when you don't know where to even find that info in the first place.