4 ms·
Most things can't be learned via pure theory or pure practice. Almost nothing related to work in the modern day can. In ML not everything can be derived from t
by danielmarkbruce 2y ago
Most things can't be learned via pure theory or pure practice. Almost nothing related to work in the modern day can.
In ML not everything can be derived from theory. If it could, we'd not have been so surprised by the performance of really really large language models. At the same time, if you can't reason about the math involved, you are going to have a difficult time figuring why something isn't working or what options you have - could be around architecture or loss functions or choice of activation function or optimizer or hyperparameters or training time/resources or a dozen other things.
- cubefox 2y ago> In ML not everything can be derived from theory. And not every theory in ML has a lot of applications to practice. For example, statistical learning theory has only limited relevance in practice, and algorithmic learning theory has basically none at all. There are a lot of mathematical theories that are relatively old (often much older than the deep learning boom and definitely older than transformers) and that are more interesting from a conceptual perspective rather than from the point of practical applications.