4 ms·

How does one expect to do ‘AI research’ without much of a background in math? Machine learning is pretty much all math. Researchers are generally expected to b

by afpx 9y ago

How does one expect to do ‘AI research’ without much of a background in math? Machine learning is pretty much all math. Researchers are generally expected to be experts in their field. The people writing the papers on arxiv likely spent most of their lives learning about machine learning and mathematics. Unfortunately, there’s not an easy path to become an expert. One just has to dig in and learn from the ground up. Edit: The good news is that it’s never been as easy to learn math than it is now. When I was an undergrad in math, there were almost no resources available to learn the intuitions behind the math. One just has to keep doing proofs and exercises over and hope that it would ‘click’ at some point. But, sometimes that wouldn’t happen until many years later. Nowadays, one can watch YouTube videos where experts describe the intuition behind the math. It’s awesome.

joshvm 9y ago
Absolutely you can do 'AI Research' without a degree in maths. Sure you need a grounding in linear algebra, stats, probability and calculus, but not much more than a CS or physics degree will teach you. That stuff is indeed learned easily, and it's not what the Reddit user is worried about. That also ignores applications of machine learning, which is also a massive (and lucrative) field. But because it's a trendy field, I think there is an obsession with people needing to understand everything theoretical that comes out for fear of missing the boat. Some of the really interesting papers that have come out over the last few years - for example artistic style transfer and Faster R-CNN - have hardly any maths in. You can count the equations on one hand in both those papers. No doubt the authors know their stuff, but how readable are those papers compared to e.g. a 100-page proof? Which did I learn more from? They're a combination of two things: intuitive network architecture and a clever loss function. The first thing is a combination of intuition and programming, the second involves a little maths, but nothing outrageous. [1] https://arxiv.org/abs/1508.06576 https://arxiv.org/abs/1508.06576 [2] https://arxiv.org/abs/1506.01497 https://arxiv.org/abs/1506.01497
- afpx 9y ago
  You're right - I see now when he says 'AI Research' he really means doing applied ML. And, that can certainly be done without knowing everything. If the goal is to make a lot of money doing applied ML, then become a consultant and aim to know 10% more than the customers. If the goal is to create models that are relatively effective, then read tutorials, play with data, experiment, and iterate. But, if the goal is to create very effective models and be able to actually explain why they work (which I think is what many companies want), then one has to understand the math. That is, showing some interesting relationships, trends, predictions, or inferences on a data analysis portal or consumer web site is one thing. But, using ML to dispense medication, regulate a medical device, drive a power plant, or identify criminal suspects - those may require different skills. (BTW I don't mean to disparage the middle group, as that's largely what I do. But, luckily I have people in the latter group who can validate what I'm doing.)
- Blazespinnaker 9y ago
  Not a degree, no, but you should have depth in the math techniques you are using to solve your problem. You should also develop a high level abstracted understanding of math to have the intuition that your techniques are likely the right ones. This approach will help make incremental improvements. You might even get lucky and hit in something that cites really well.
tomrod 9y ago
> Machine learning is pretty much all math. Eh, I would say math is the language we use to communicate to computers presently, but the underlying _concepts_ don't require "more maths" and can often be grasped through an intuitive approach. For example, almost everyone who uses photoshop understands the underlying concept of a convolution (e.g. Gaussian blur), even if they don't know the mathematics that can be used to describe the operation. Yes, there are difficult notions that formalization or generalization assist with--perhaps it is better to see math as augmenting the initial intuition, rather than driving the intuition?
- heavenlyblue 9y ago
  It's not always helpful even when someone gives you a correct direction but none of his abstracted vision of how to apply that direction. For example: convolution is not _just_ Gaussian blur; it allows someone to find an object on the scene (or a shape of something in time dimension). How is that related to Gaussian blur and why are they the same? It takes time to understand the full domain of the concept.
  - tomrod 9y ago
    To understand the underlying principles, but not to use it. In this case, it's a point and click -- if they like the effect, keep it. Literally no maths are required for understanding, yet the outcome is achieved.
evanb 9y ago
> Unfortunately, there’s not an easy path to become an expert. One just has to dig in and learn from the ground up. Indeed, there's no royal road to geometry!
- johnsonjo 9y ago
  I have no idea why you were downvoted this was the exact quote that came to my mind when I read the text you quoted as it sums up the principle perfectly that knowledge of math just doesn't come passively. Some context from wikipedia about the quote, 'Euclid is said to have replied to King Ptolemy's request for an easier way of learning mathematics that "there is no Royal Road to geometry," according to Proclus' [1]. [1]: https://en.wikipedia.org/wiki/Royal_Road#A_metaphorical_.E2.80.9CRoyal_Road.E2.80.9D_in_famous_quotations https://en.wikipedia.org/wiki/Royal_Road#A_metaphorical_.E2....
  - evanb 9y ago
    I guess I cannot rely on a widespread knowledge of the history of mathematics :( Still, I thought this line was famous "enough".
syllogism 9y ago
Actually I think it's sometimes harmful to take the maths too seriously. There are three parts to the ideal paper: 1. Describe a new technique; 2. Show that it works; 3. Explain why it works. Understanding why things work is easily the hardest thing. This is where the most maths gets deployed...But often people are reaching for the fancier maths when they can't find a simpler intuition behind the idea. You can also use fancier analysis to substitute for less impressive empirical results. These explanations might convince reviewers, but that doesn't make them any more likely to be correct. I find it effective to take a very "computer's eye view" of things. Instead of thinking primarily about the formalisation, I mostly think about what's being computed. What sort of information is flowing around, during both the prediction and the updates? What dynamics emerge?

by afpx 9y ago

joshvm 9y ago

Absolutely you can do 'AI Research' without a degree in maths. Sure you need a grounding in linear algebra, stats, probability and calculus, but not much more than a CS or physics degree will teach you. That stuff is indeed learned easily, and it's not what the Reddit user is worried about. That also ignores applications of machine learning, which is also a massive (and lucrative) field. But because it's a trendy field, I think there is an obsession with people needing to understand everything theoretical that comes out for fear of missing the boat. Some of the really interesting papers that have come out over the last few years - for example artistic style transfer and Faster R-CNN - have hardly any maths in. You can count the equations on one hand in both those papers. No doubt the authors know their stuff, but how readable are those papers compared to e.g. a 100-page proof? Which did I learn more from? They're a combination of two things: intuitive network architecture and a clever loss function. The first thing is a combination of intuition and programming, the second involves a little maths, but nothing outrageous. [1] https://arxiv.org/abs/1508.06576 https://arxiv.org/abs/1508.06576 [2] https://arxiv.org/abs/1506.01497 https://arxiv.org/abs/1506.01497

afpx 9y ago
You're right - I see now when he says 'AI Research' he really means doing applied ML. And, that can certainly be done without knowing everything. If the goal is to make a lot of money doing applied ML, then become a consultant and aim to know 10% more than the customers. If the goal is to create models that are relatively effective, then read tutorials, play with data, experiment, and iterate. But, if the goal is to create very effective models and be able to actually explain why they work (which I think is what many companies want), then one has to understand the math. That is, showing some interesting relationships, trends, predictions, or inferences on a data analysis portal or consumer web site is one thing. But, using ML to dispense medication, regulate a medical device, drive a power plant, or identify criminal suspects - those may require different skills. (BTW I don't mean to disparage the middle group, as that's largely what I do. But, luckily I have people in the latter group who can validate what I'm doing.)
Blazespinnaker 9y ago
Not a degree, no, but you should have depth in the math techniques you are using to solve your problem. You should also develop a high level abstracted understanding of math to have the intuition that your techniques are likely the right ones. This approach will help make incremental improvements. You might even get lucky and hit in something that cites really well.

tomrod 9y ago

> Machine learning is pretty much all math. Eh, I would say math is the language we use to communicate to computers presently, but the underlying _concepts_ don't require "more maths" and can often be grasped through an intuitive approach. For example, almost everyone who uses photoshop understands the underlying concept of a convolution (e.g. Gaussian blur), even if they don't know the mathematics that can be used to describe the operation. Yes, there are difficult notions that formalization or generalization assist with--perhaps it is better to see math as augmenting the initial intuition, rather than driving the intuition?

heavenlyblue 9y ago
It's not always helpful even when someone gives you a correct direction but none of his abstracted vision of how to apply that direction. For example: convolution is not _just_ Gaussian blur; it allows someone to find an object on the scene (or a shape of something in time dimension). How is that related to Gaussian blur and why are they the same? It takes time to understand the full domain of the concept.
- tomrod 9y ago
  To understand the underlying principles, but not to use it. In this case, it's a point and click -- if they like the effect, keep it. Literally no maths are required for understanding, yet the outcome is achieved.

evanb 9y ago

> Unfortunately, there’s not an easy path to become an expert. One just has to dig in and learn from the ground up. Indeed, there's no royal road to geometry!

johnsonjo 9y ago
I have no idea why you were downvoted this was the exact quote that came to my mind when I read the text you quoted as it sums up the principle perfectly that knowledge of math just doesn't come passively. Some context from wikipedia about the quote, 'Euclid is said to have replied to King Ptolemy's request for an easier way of learning mathematics that "there is no Royal Road to geometry," according to Proclus' [1]. [1]: https://en.wikipedia.org/wiki/Royal_Road#A_metaphorical_.E2.80.9CRoyal_Road.E2.80.9D_in_famous_quotations https://en.wikipedia.org/wiki/Royal_Road#A_metaphorical_.E2....
- evanb 9y ago
  I guess I cannot rely on a widespread knowledge of the history of mathematics :( Still, I thought this line was famous "enough".

syllogism 9y ago

Actually I think it's sometimes harmful to take the maths too seriously. There are three parts to the ideal paper: 1. Describe a new technique; 2. Show that it works; 3. Explain why it works. Understanding why things work is easily the hardest thing. This is where the most maths gets deployed...But often people are reaching for the fancier maths when they can't find a simpler intuition behind the idea. You can also use fancier analysis to substitute for less impressive empirical results. These explanations might convince reviewers, but that doesn't make them any more likely to be correct. I find it effective to take a very "computer's eye view" of things. Instead of thinking primarily about the formalisation, I mostly think about what's being computed. What sort of information is flowing around, during both the prediction and the updates? What dynamics emerge?