4 ms·
Interesting read, but there's a few things I haven't understood. In the training [function](https://colab.research.google.com/drive/1YRp9k_ORH4wZMqXLNkc3Ir5w4B5
by gbersac 5y ago
Interesting read, but there's a few things I haven't understood. In the training [function](https://colab.research.google.com/drive/1YRp9k_ORH4wZMqXLNkc3Ir5w4B5f-8Pa?usp=sharing https://colab.research.google.com/drive/1YRp9k_ORH4wZMqXLNkc...):
1- In the instruction `hidden_layer.data[index] -= learning_rate * hidden_layer.grad.data[index]`where was the `hidden_layer.grad` value updated?
2- from what I've understood, we'll update the hidden_layer according to the inclination of the error function (because we want to minimize it). But where are `error.backward()` and `hidden_layer.grad` interconnected?
- Sirupsen 5y agoGreat questions, I struggled with this part the most when I was learning it. `.grad` is set by `autograd` when calling `backward()` Probably the easiest way to understand this is to play a bit with `.grad` and `backward()` on their own, with the first code sample in the `autograd` section [1]. [1]: https://sirupsen.com/napkin/neural-net#automagically-computing-the-slope-of-a-function-with-autograd https://sirupsen.com/napkin/neural-net#automagically-computi...
- gbersac 5y agoThank you!
- cercatrova 5y agoFYI Markdown type links don't work on HN, that's why you see people type footnotes [0] [0] like this