15 ms·

Is anyone using any of this math? My guess is no. At best it provides "moral support" for deep learning researchers who want to feel reassured that what they ar

Is anyone using any of this math? My guess is no. At best it provides "moral support" for deep learning researchers who want to feel reassured that what they are attempting to do is not impossible. Glad to be proven wrong, though.

nerdponx 3y ago
Describing it as "moral support" really sells it short. Imagine computer science without sorting algorithms, search algorithms, etc that have been proven correct and have known proven properties. This math serves the same purpose as CS theory. So yes, if you're just fitting a model from a library like Keras, you're not really "using" the math. If you're working with data sets below a certain size, problems below a certain level of complexity, and models that have been deployed for many years and have well studied properties, you can do a lot with only a cursory understanding of the math, much like you can write perfectly functional web apps in Python or Java without really understanding how the language runtime works at a deep level. But if you don't actually know how it works, you're going to get stuck pretty badly if you encounter a situation that isn't already baked into a library. If you want to see what happens when you don't know the underlying math, look at the current generation of "data science" graduates, who don't know their math or statistics fundamentals. There are plenty of issues on the hiring side of course, but ultimately the reason those kids aren't getting jobs is that they don't actually know what they're doing, because they were never forced to learn this stuff.
nephanth 3y ago
According to the abstract it covers different ANN architectures, optimization algorithms, probably backpropagation.. so um yes? That is stuff anyoke in machine learning uses everyday?
danielmarkbruce 3y ago
Some people like to think and communicate in dense math notation. So, yes.
godelski 3y ago
There's something I tell my students. You don't need math to make good models, but you do need to know math to know why your models are wrong. So yes, math is needed. If you don't have math you're going to hoodwink yourself into thinking you can get to AGI by scale alone. You'll just use transformers everywhere because that's what everyone else does and you'll get confused between activation functions. You'll make models and models that work, but there's a big difference in working models and knowing where to expect your models to fail and understanding their limitations. I feel a lot of people just look at test set results and expect that to mean that the model isn't overfitting. (not to mention tuning hps based on test set results)
- famouswaffles 3y ago
  >If you don't have math you're going to hoodwink yourself into thinking you can get to AGI by scale alone. There are many researchers who "have math" and still believe this. Appeal to Authority is a fallacy at the best of times but it's usually a convincing one. Not so much when the authority hasn't formed consensus on the appeal.
- light_hue_1 3y ago
  Oh sure. I say the same to my students. But the particular spin on this book makes it look to non-experts that this is the math you need to do something useful with deep learning. And that's just not true. Certainly you need to understand what you're optimizing, how your optimizer works, what your objective function is doing, etc. But the vast majority of people don't need to know about theoretical approximation results for problems that they will never actually encounter in real life, etc. For example, I have never used used anything like "6.1.3 Lyapunov-type stability for GD optimization" in a decade of ML research. I'm sure people do! But not on the kinds of problems I work on. Just look at the comments here. People are complaining about the lack of context, but this is fine for the audience the book is aimed at. It's just the average HN reader. I think it would be better if the authors chose a different title. As it stands, non-experts will be attracted and then be put off, and experts will think the book is likely to be too generic.
  - 3y ago
fastneutron 3y ago
In the latter part of the book that covers PINNs and other PDE methods, it helps to frame these using the same kind of functional analysis that is used to develop more traditional numerical methods. In this case, it provides a way for practitioners to verify the physical consistency between the various methods.

nerdponx 3y ago

Describing it as "moral support" really sells it short. Imagine computer science without sorting algorithms, search algorithms, etc that have been proven correct and have known proven properties. This math serves the same purpose as CS theory. So yes, if you're just fitting a model from a library like Keras, you're not really "using" the math. If you're working with data sets below a certain size, problems below a certain level of complexity, and models that have been deployed for many years and have well studied properties, you can do a lot with only a cursory understanding of the math, much like you can write perfectly functional web apps in Python or Java without really understanding how the language runtime works at a deep level. But if you don't actually know how it works, you're going to get stuck pretty badly if you encounter a situation that isn't already baked into a library. If you want to see what happens when you don't know the underlying math, look at the current generation of "data science" graduates, who don't know their math or statistics fundamentals. There are plenty of issues on the hiring side of course, but ultimately the reason those kids aren't getting jobs is that they don't actually know what they're doing, because they were never forced to learn this stuff.

nephanth 3y ago

According to the abstract it covers different ANN architectures, optimization algorithms, probably backpropagation.. so um yes? That is stuff anyoke in machine learning uses everyday?

danielmarkbruce 3y ago

Some people like to think and communicate in dense math notation. So, yes.

godelski 3y ago

There's something I tell my students. You don't need math to make good models, but you do need to know math to know why your models are wrong. So yes, math is needed. If you don't have math you're going to hoodwink yourself into thinking you can get to AGI by scale alone. You'll just use transformers everywhere because that's what everyone else does and you'll get confused between activation functions. You'll make models and models that work, but there's a big difference in working models and knowing where to expect your models to fail and understanding their limitations. I feel a lot of people just look at test set results and expect that to mean that the model isn't overfitting. (not to mention tuning hps based on test set results)

famouswaffles 3y ago
>If you don't have math you're going to hoodwink yourself into thinking you can get to AGI by scale alone. There are many researchers who "have math" and still believe this. Appeal to Authority is a fallacy at the best of times but it's usually a convincing one. Not so much when the authority hasn't formed consensus on the appeal.
light_hue_1 3y ago
Oh sure. I say the same to my students. But the particular spin on this book makes it look to non-experts that this is the math you need to do something useful with deep learning. And that's just not true. Certainly you need to understand what you're optimizing, how your optimizer works, what your objective function is doing, etc. But the vast majority of people don't need to know about theoretical approximation results for problems that they will never actually encounter in real life, etc. For example, I have never used used anything like "6.1.3 Lyapunov-type stability for GD optimization" in a decade of ML research. I'm sure people do! But not on the kinds of problems I work on. Just look at the comments here. People are complaining about the lack of context, but this is fine for the audience the book is aimed at. It's just the average HN reader. I think it would be better if the authors chose a different title. As it stands, non-experts will be attracted and then be put off, and experts will think the book is likely to be too generic.
- 3y ago

fastneutron 3y ago

In the latter part of the book that covers PINNs and other PDE methods, it helps to frame these using the same kind of functional analysis that is used to develop more traditional numerical methods. In this case, it provides a way for practitioners to verify the physical consistency between the various methods.