4 ms·

As a mathematician, people routinely tell me that mathematics isn't applicable in a particular situation. Upon deeper investigation, I usually find that they si

As a mathematician, people routinely tell me that mathematics isn't applicable in a particular situation. Upon deeper investigation, I usually find that they simply have a very limited understanding of what math is. Most people associate it with numbers and equations -- but those are just a few tools that are taught early on. Actual math is about analysis and some of these tools MAY be used when preforming it. Even some of the deeper topics, such as calculus and algebra are simply tools in a greater framework of understanding. Even if you're using very little of what you think of as "math" to interact with your machine learning framework, to dismiss math as not being necessary is ignoring all of the underlying mathematics that went into creating it. You may not need calculus, or the like, but the math is there -- even if you don't recognize it as such.

TimPC 9y ago
The frameworks can abstract away much of the math. You won't have to actually take the derivatives yourself, but understanding how they work actually matters to network design. It's much harder to get meaningful insights out of papers for designing better networks if you can't read the math in them. And Neural Networks is an area where you will read research papers as part of your job in order to do it well, even in industry. When is RELU better than sigmoid? When should you use a bottleneck layer? When do you want fewer larger layers or more smaller layers? When is it appropriate to add some manual features in your problem? Or even when you get an error message back from your framework when is the error in your network code an when is it in your data design? It's very hard to develop good answers to the questions that come up if you don't have the math to read papers, don't have the math to double check expressions in the error code, or have problems going back and forth between expressions with Matrix and vectors and expressions in summation notation. There are a lot of people who can build a neural network when the architecture is done for them and nothing errors (using no math or stats), but training someone to make decisions on the architecture and handle the errors IS the job.
- throwawayjava 9y ago
  Yeah, this is how statistics is taught in a lot of psych departments. A bunch of tests and rules of thumb about when to use which equations, together with some training on a piece of software. IMO the "pragmatic rules-of-thumb" approach to teaching/understanding statistics is probably the genesis point of the reproducability crisis. This mindset toward stats has probably done more damage to Psychology than anything else in the history of the field. So, proceed with caution.
  - eli_gottlieb 9y ago
    >IMO the "pragmatic rules-of-thumb" approach to teaching/understanding statistics is probably the genesis point of the reproducability crisis. This mindset toward stats has probably done more damage to Psychology than anything else in the history of the field. Sure, but then we have another problem: if you need to know measure theory to run an experiment, nobody will run experiments. And lots of mathematical statistics just goes ahead and uses the measure theory.
    - throwawayjava 9y ago
      There's a middle-ground, though.
tprice7 9y ago
"Even if you're using very little of what you think of as "math" to interact with your machine learning framework, to dismiss math as not being necessary is ignoring all of the underlying mathematics that went into creating it." This seems like a straw man to me. The article never claimed that math is not necessary for the development of machine learning frameworks. In fact, it explicitly states the opposite: "First of all, math is particularly important if you’re doing machine learning research in an academic setting." "In particular, there are people at companies like Google and Facebook who are pushing the boundaries of machine learning – people working on bleeding edge tools. These people almost certainly employ calculus, linear algebra, and more advanced math routinely in their work."
- johnhenry 9y ago
  Perhaps adding that last part without acknowledging these claims from the article wasn't fair to the author. I meant it more of a rebuttal to the subject line. I probably should have ended with something along the lines of "data analysis is a different type of math" rather than trying to promote the idea that "there's still math in machine learning".
dsacco 9y ago
Agreed. Most people don't see how mathematics is applicable because they think of it as a framework for calculation, when mathematics is really much more about proofs. In addition, most people never learn more than calculus (if that), so they have no concept of what real world problems can be elegantly reduced to mathematical restatements. It's like the difference between calculus and analysis.
aswanson 9y ago
The longer I'm around, the more I'm convinced of this line of thought and the fundamental power, beauty and truth of math. We need more early educators of math with your mindset.
rossdavidh 9y ago
The basic point of the article is: - if you don't know calculus, differential equations, etc. you can still do good work in this field - if you do know those types of math, but you don't have experience with data analysis, you CAN'T do good work in this field (until you learn these skills). Having done both machine learning and data analysis more generally, these points are correct. The article is explicitly aimed at the question of what is needed to do an entry level job in this kind of setting.
auvrw 9y ago
first: i like math. i think the tone of the article is good, however, because it's definitely possible to start playing with pylearn, make some SVMs, and have a lot of fun without worrying too much about L^p spaces are in the first place. later on, it could be possible to interpolate more mathematical ideas for a smoother understanding of how to measure how much preprocessing is flattening out their data sets. (in other words, sssh this is another opportunity to trick people into doing math without immediately recognizing it as such! :-)

TimPC 9y ago

The frameworks can abstract away much of the math. You won't have to actually take the derivatives yourself, but understanding how they work actually matters to network design. It's much harder to get meaningful insights out of papers for designing better networks if you can't read the math in them. And Neural Networks is an area where you will read research papers as part of your job in order to do it well, even in industry. When is RELU better than sigmoid? When should you use a bottleneck layer? When do you want fewer larger layers or more smaller layers? When is it appropriate to add some manual features in your problem? Or even when you get an error message back from your framework when is the error in your network code an when is it in your data design? It's very hard to develop good answers to the questions that come up if you don't have the math to read papers, don't have the math to double check expressions in the error code, or have problems going back and forth between expressions with Matrix and vectors and expressions in summation notation. There are a lot of people who can build a neural network when the architecture is done for them and nothing errors (using no math or stats), but training someone to make decisions on the architecture and handle the errors IS the job.

throwawayjava 9y ago
Yeah, this is how statistics is taught in a lot of psych departments. A bunch of tests and rules of thumb about when to use which equations, together with some training on a piece of software. IMO the "pragmatic rules-of-thumb" approach to teaching/understanding statistics is probably the genesis point of the reproducability crisis. This mindset toward stats has probably done more damage to Psychology than anything else in the history of the field. So, proceed with caution.
- eli_gottlieb 9y ago
  >IMO the "pragmatic rules-of-thumb" approach to teaching/understanding statistics is probably the genesis point of the reproducability crisis. This mindset toward stats has probably done more damage to Psychology than anything else in the history of the field. Sure, but then we have another problem: if you need to know measure theory to run an experiment, nobody will run experiments. And lots of mathematical statistics just goes ahead and uses the measure theory.
  - throwawayjava 9y ago
    There's a middle-ground, though.

tprice7 9y ago

"Even if you're using very little of what you think of as "math" to interact with your machine learning framework, to dismiss math as not being necessary is ignoring all of the underlying mathematics that went into creating it." This seems like a straw man to me. The article never claimed that math is not necessary for the development of machine learning frameworks. In fact, it explicitly states the opposite: "First of all, math is particularly important if you’re doing machine learning research in an academic setting." "In particular, there are people at companies like Google and Facebook who are pushing the boundaries of machine learning – people working on bleeding edge tools. These people almost certainly employ calculus, linear algebra, and more advanced math routinely in their work."

johnhenry 9y ago
Perhaps adding that last part without acknowledging these claims from the article wasn't fair to the author. I meant it more of a rebuttal to the subject line. I probably should have ended with something along the lines of "data analysis is a different type of math" rather than trying to promote the idea that "there's still math in machine learning".

dsacco 9y ago

Agreed. Most people don't see how mathematics is applicable because they think of it as a framework for calculation, when mathematics is really much more about proofs. In addition, most people never learn more than calculus (if that), so they have no concept of what real world problems can be elegantly reduced to mathematical restatements. It's like the difference between calculus and analysis.

aswanson 9y ago

The longer I'm around, the more I'm convinced of this line of thought and the fundamental power, beauty and truth of math. We need more early educators of math with your mindset.

rossdavidh 9y ago

The basic point of the article is: - if you don't know calculus, differential equations, etc. you can still do good work in this field - if you do know those types of math, but you don't have experience with data analysis, you CAN'T do good work in this field (until you learn these skills). Having done both machine learning and data analysis more generally, these points are correct. The article is explicitly aimed at the question of what is needed to do an entry level job in this kind of setting.

auvrw 9y ago

first: i like math. i think the tone of the article is good, however, because it's definitely possible to start playing with pylearn, make some SVMs, and have a lot of fun without worrying too much about L^p spaces are in the first place. later on, it could be possible to interpolate more mathematical ideas for a smoother understanding of how to measure how much preprocessing is flattening out their data sets. (in other words, sssh this is another opportunity to trick people into doing math without immediately recognizing it as such! :-)