3 ms·

This is not my field, but a serious question--I once read that part of the motivation of pharmaceutical companies in hiring researchers was not so that they wou

by ylem 10y ago

This is not my field, but a serious question--I once read that part of the motivation of pharmaceutical companies in hiring researchers was not so that they would all produce ground breaking independent research, but rather because they would be capable of reading the literature (again, not my field). Is that true at all for machine learning? Would companies hire people who would be up to date with the literature so that they could implement algorithms that others have developed in an academic context and put them into production?

feral 10y ago
Yes, I think that's true. Basically, the interfaces of the models/tools/abstractions people will use will be 'leaky'. For example, you can take a machine learning method from scikit learn, which works really well on the scikit learn example, and apply it to your problem. Any developer can do this pretty quickly. If it works, and gets good accuracy out-of-the-box, then great. But if it doesn't work, what do you do? How do you know where to look, what could trip it up? Are your features OK, or is it a problem with your model? Or maybe you framed the problem wrong? When you get into this sort of area, that's when you need an ML expert, who knows whats actually going on under the hood - or can at least learn the particularly model quickly - and can make progress faster. A more general developer will slow down drastically once they start reading the documentation for how the model/system actually works. And hopefully the expert will frame the project better from the very start, solving issues before they even arise, because they know the kind of issue that can occur, or suggest easier paths to solutions. So, agree with your point there. But how many such experts do you need? In my experience, only a small number on a bigger team. ML folk are highly leveraged, but need a lot of support to get their product into production - to manage the data (if its worth applying ML to, there's probably a lot of data, so a big data engineering task, maybe connected to a live system), to think about the UX etc. This will all evolve on several levels of course: - The tooling will get better; model/data deployment/management will get easier. But also and non-experts will be able to get more done as ML becomes more robust out-of-the-box. - We'll get better at building ML products (e.g. team structure, data infrastructure, UX (designers learning how these things work), company org (e.g. a lot of friction between Agile and ML)) - Businesses will want to do bigger things But if I had to guess, we'll be picking up low-hanging fruit for a while, and most of the work, for most companies, will be in the support infrastructure and application, with just a minority of specialist ML roles.
ska 10y ago
Absolutely they will, and should. But they'll need relatively few people on the team(s) who can do this.

by ylem 10y ago

feral 10y ago

Yes, I think that's true. Basically, the interfaces of the models/tools/abstractions people will use will be 'leaky'. For example, you can take a machine learning method from scikit learn, which works really well on the scikit learn example, and apply it to your problem. Any developer can do this pretty quickly. If it works, and gets good accuracy out-of-the-box, then great. But if it doesn't work, what do you do? How do you know where to look, what could trip it up? Are your features OK, or is it a problem with your model? Or maybe you framed the problem wrong? When you get into this sort of area, that's when you need an ML expert, who knows whats actually going on under the hood - or can at least learn the particularly model quickly - and can make progress faster. A more general developer will slow down drastically once they start reading the documentation for how the model/system actually works. And hopefully the expert will frame the project better from the very start, solving issues before they even arise, because they know the kind of issue that can occur, or suggest easier paths to solutions. So, agree with your point there. But how many such experts do you need? In my experience, only a small number on a bigger team. ML folk are highly leveraged, but need a lot of support to get their product into production - to manage the data (if its worth applying ML to, there's probably a lot of data, so a big data engineering task, maybe connected to a live system), to think about the UX etc. This will all evolve on several levels of course: - The tooling will get better; model/data deployment/management will get easier. But also and non-experts will be able to get more done as ML becomes more robust out-of-the-box. - We'll get better at building ML products (e.g. team structure, data infrastructure, UX (designers learning how these things work), company org (e.g. a lot of friction between Agile and ML)) - Businesses will want to do bigger things But if I had to guess, we'll be picking up low-hanging fruit for a while, and most of the work, for most companies, will be in the support infrastructure and application, with just a minority of specialist ML roles.

ska 10y ago

Absolutely they will, and should. But they'll need relatively few people on the team(s) who can do this.