3 ms·
> I recently read in "The Little Learner" book that finding the right parameters is learning. That's the point. Everything we do in deep learning is focused on
by giovannibonetti 3y ago
> I recently read in "The Little Learner" book that finding the right parameters is learning. That's the point. Everything we do in deep learning is focused on choosing the right sequence of numbers and we call those numbers parameters. Every parameter has a specific role in our model. Parameters are our choice, those are the nobs that we (as a personified machine learning algorithm) get to adjust.
Be careful not to mistake parameters for hyperparameters.
- Parameters are the result of the training phase, as you mentioned. They start with random values and are discovered by the training algorithm;
- Hyperparameters, on the other hand, are the knobs you tweak to make the training process arrive at the "right" parameters. You can think of them as meta-parameters;
Also, it is important to think on the ML architecture - transformers, neural networks, random forests and so on - as the parameters change completely depending on which one you're using.
- Buttons840 3y agoYes, hyper-parameters are parameters about the parameters. Parameters we get to choose which control the parameters that the learning algorithm chooses. The other set of data the book called "arguments", which is the term they use to describe the data you are training on. That seems like an unnecessarily confusing term, and I haven't heard it anywhere else. I didn't learn anything truly new in all this, but it helped me sort my own thoughts to realize there is data, parameters, and hyperparameters. Data comes from the world and we cannot change it. Parameters are chosen by us indirectly through the learning algorithm, they are the most important outcome of successful learning. Hyperparameters are chosen by us directly and control the model and learning algorithm, and the resulting parameters.