6 ms·

No, it's not. Im an AI researcher at a big tech company. It would have been far wiser financially to start as a software engineer after my Bachelor's. Also, ha

No, it's not. Im an AI researcher at a big tech company. It would have been far wiser financially to start as a software engineer after my Bachelor's. Also, having a few NEURIPS or ICML papers is not a hiring guarantee any more. It's decidedly not 2015-17 any longer. In particular, I feel bad for the people who started their PhDs with $$$ in their eyes around that time. Don't get me wrong, they will all be employable, but the field moves so fast, I have my doubts there will be cushy FAANG jobs for everyone capable of playing around with network architectures in a few years. It's a terrible idea to start one based on the hype few years ago.

alphagrep12345 7y ago
I have a couple of questions - - Would you say that you'd have got an AI researcher position without a Ph.D.? - Also, why is NEURIPS or ICML papers is not a hiring guarantee? I thought they're highly sought after.
- joshvm 7y ago
  > Would you say that you'd have got an AI researcher position without a Ph.D.? It's difficult to get any true researcher position without a PhD. It doesn't mean that PhD has to be in AI. Research involves a lot of reading and writing papers, which a PhD is supposedly training you how to do. That said most places will say "equivalent practical experience" and it's entirely possible to be competent in AI/ML without a PhD. I did a PhD in space science, I now do machine learning in ecology and spent the summer working on machine learning for disaster management. The interesting jobs (to me) are where domains cross, and it's also (hint hint) much easier to get a job doing AI for X than it is doing "fundamental AI". In any case, you're often doing stuff that nobody has done before anyway, but you don't need to spend your life hunting for the new ResNet. > Also, why is NEURIPS or ICML papers is not a hiring guarantee? What the OP probably might be implying is that everyone has a publication in NeurIPS nowadays. I think it goes deeper than that though, publishing in machine learning is broken. Having 10k people at one conference is not an efficient way to distribute research. You have to submit a full paper in November for a conference next Summer - pretty much only computer science does this madness. What's interesting is how unique this attitude is. In astronomy, for example, conferences are a fun place to catch up with folks in your niche. There might be a few hundred people and probably it'll be single-track. We publish whatever journal is the most relevant and they're generally all considered equivalent. Nobody cares if you publish in ApJ vs A&A vs MNRAS, if your research is good. There are also concerns that the quality of these venues is decreasing because the pressure to publish in them is so high. See https://arxiv.org/abs/1807.03341 https://arxiv.org/abs/1807.03341
  - neuralsp 7y ago
    >I did a PhD in space science, I now do machine learning in ecology and spent the summer working on machine learning for disaster management. Do you think it is possible to that without any background in anything? I mean could someone apply black box frameworks without understanding them. How would they be caught?
    - joshvm 7y ago
      > Do you think it is possible to that without any background in anything? To do machine learning research? Or work in some random domain? > I mean could someone apply black box frameworks without understanding them. How would they be caught? Machine learning is rapidly becoming commoditised, but lots of people still don't understand just how much effort it is to get a good dataset and to prep Domain experts scoff at machine learning people who are trying to solve Big Problems using unrepresentative toy datasets, but also tend to have much higher expectations of what ML can do. Machine learning people scoff at domain experts for using outdated techniques and bad data science, but then propose ridiculous solutions that would never work in the real world (e.g. use our model, it takes a week on 8xV100s to train and you can only run it on a computer the size of a bus). There are also a lot of people (and companies) touting machine learning as a solution to problems that don't exist. Overfitting models is probably the most rampant crime that researchers commit.
      - neuralsp 7y ago
        My question is whether someone could fake it and not be caught/fired. (So yes, I meant: "Or work in some random domain?") From the second half of your comment it seems that the answer is yes? Maybe a comparison would help: someone pretending to be an experienced iOS/Android developer without any qualifications or ability would quickly be caught. Since they couldn't produce any working app or use a compiler, and anyone can judge an app for themselves. You can't really just make it up out of whole cloth, people judge the results. You would have to start actually doing that, and if you couldn't or didn't want to, then unless you outsourced your own job or something the jig would be up pretty much instantly. (Unless you caught up.) So, how about machine learning? Do you think a fraud could land and keep such a job, without any knowledge, qualifications, ability, or even interest in getting up to speed? Just, a pure, simple fraud. What's your guess?
- mjn 7y ago
  > Also, why is NEURIPS or ICML papers is not a hiring guarantee? I thought they're highly sought after. They're sought after, but the conferences have also grown huge. NeurIPS 2018 accepted around 1,000 papers! Based on a query of the DBLP [1] dataset, there were 4,409 distinct authors who had a paper at either NeurIPS 2018 or ICML 2018 (or both). If you add in a few of the other big AI and ML conferences (AAAI, IJCAI, ICLR), the number grows to 10,995 distinct authors, again solely for the year 2018. The field is hot, but is it hot enough for ten thousand people to be automatically hired because of one paper? There's also decreasing confidence in the big conferences' review processes I think. NeurIPS 2014 actually did a study to estimate how random acceptance was by assigning some papers to two different sets of reviewers and checking how similar the decisions were [2], and found there was a much higher degree of luck in acceptance/rejection decisions than they had expected. I personally have more confidence in the review processes of smaller and more focused conferences (and journals!), though they don't have the same level of name recognition. [1] https://dblp.uni-trier.de/ https://dblp.uni-trier.de/ [2] http://blog.mrtz.org/2014/12/15/the-nips-experiment.html http://blog.mrtz.org/2014/12/15/the-nips-experiment.html
  - throwawayjava 7y ago
    A Ph.D.'s worth of first authored NeurIPS/ICML papers will get you a very good job pretty easily still. But AI slices papers very thin and author lists are inflated relative to other subfields of CS. A single paper in one of the major conferences is a pretty marginal contribution, especially if you're in the middle of a long author list. Also, NeurIPS reviewing has gone to absolute hell. I mean, peer review everywhere has problems. But I've never seen something quite this bad. At this point I think it's safe to say that most reviewers wouldn't even make it to an on-site interview for a faculty position at a research university. That's definitely nowhere near normal. You can't really blame anyone, I guess; the community is growing way too quickly for any real quality control. Frankly, I think those conferences have outlived their usefulness as anything except marquee marketing events. I'm now mostly attending smaller and more specialized conferences.
    - mjn 7y ago
      I've gone in a similar direction. Only at smaller conferences can you have any kind of confidence that your reviewers are people with actual expertise in the field. That's pretty useful, not only because it makes it less likely you'll get reviews that are very annoying, but also because a review by a knowledgeable person can be genuinely valuable. The big conferences are full of reviews written by 2nd-year grad students, because with this many submissions, any warm body with anything approaching credentials is needed. Besides just "quality" in the general sense, one thing this has really hurt, I think, is any sense of history or continuity. There are a ton of reviewers who have basically no familiarity with the pre-2010 ML literature, and it kind of shows in both the reviews and the papers that get published. I mean I get that deep learning beats a lot of older methods on major benchmarks, but it's still not the case that literally every problem, controversy, and technique was first studied post-2010.
rantwasp 7y ago
Isn't this always the case though? A PhD is a long term investment and the field that you go into is probably not the one that will be hot when you finish your PhD. So driving your decisions on $$$ seems foolish at best.
- mellosouls 7y ago
  Yep. If you're empty enough to follow a PhD solely for the money, there are far safer financial bets. Go into medicine or something. Not that I'd want to experience that bedside manner...
  - nefitty 7y ago
    It is not fair to call someone "empty" because their motivation stems from something you don't share. It does not make their work and contribution any less difficult or necessary.
    - xpe 7y ago
      Though harsh and (of course) generalized, I think the word "empty" is a useful description here. Think of these meanings of empty: - "containing nothing; not filled or occupied" - "lacking meaning or sincerity" - "having no value or purpose" In the context of academia, there is an expectation of a certain fundamental motivation; namely, to pursue knowledge in a particular field and share it back via writing and/or teaching. If one lacks that primary motivation, they certainly "lack" it and are "not filled" with the academic ethos. So "empty" is a fair term. Now, I understand the realities as well -- seeking publications, recognition, tenure, and funding are also political activities. But this does not contradict the underlying community norm that I mention above. In fact, it supports it -- it explains why so many people endure a tough, grueling, political process despite it not being their wheelhouse.
      - rat9988 7y ago
        >If one lacks that primary motivation, they certainly "lack" it and are "not filled" with the academic ethos. So "empty" is a fair term. No idea why you think this. A phd has a contrat with a salary, starting and end dates.
hdrujvw-4579 7y ago
>I have my doubts there will be cushy FAANG jobs for everyone capable of playing around with network architectures in a few years. Presumably because technical needs of FAANG might be moving in other directions. Could someone comment why this might be the case, and what other fields might look relevant
- sgt101 7y ago
  I think that the other force is that the skills that are being created by ML Ph.D programs get commodified. I don't think this will happen, because typing "import tensorflow.keras.*" isn't the skill that an ML Ph.D develops, and it is the part of the skill set that is (and will be) commodity along with the automl stuff. Constructing a problem, handling the data and running a proper process is harder, and it's the value that will put processes that use ML at risk, and deliver differentiating value for the ones where it works.
jcims 7y ago
What about domain-specific applications of the technology? I've been in the information security industry for ~25 years and while 'AI' has been increasing in use but is still woefully short of its potential in a few different areas. It seems that someone with deep domain expertise that also has PhD level chops in ML would be a profit machine.

alphagrep12345 7y ago

I have a couple of questions - - Would you say that you'd have got an AI researcher position without a Ph.D.? - Also, why is NEURIPS or ICML papers is not a hiring guarantee? I thought they're highly sought after.

joshvm 7y ago
> Would you say that you'd have got an AI researcher position without a Ph.D.? It's difficult to get any true researcher position without a PhD. It doesn't mean that PhD has to be in AI. Research involves a lot of reading and writing papers, which a PhD is supposedly training you how to do. That said most places will say "equivalent practical experience" and it's entirely possible to be competent in AI/ML without a PhD. I did a PhD in space science, I now do machine learning in ecology and spent the summer working on machine learning for disaster management. The interesting jobs (to me) are where domains cross, and it's also (hint hint) much easier to get a job doing AI for X than it is doing "fundamental AI". In any case, you're often doing stuff that nobody has done before anyway, but you don't need to spend your life hunting for the new ResNet. > Also, why is NEURIPS or ICML papers is not a hiring guarantee? What the OP probably might be implying is that everyone has a publication in NeurIPS nowadays. I think it goes deeper than that though, publishing in machine learning is broken. Having 10k people at one conference is not an efficient way to distribute research. You have to submit a full paper in November for a conference next Summer - pretty much only computer science does this madness. What's interesting is how unique this attitude is. In astronomy, for example, conferences are a fun place to catch up with folks in your niche. There might be a few hundred people and probably it'll be single-track. We publish whatever journal is the most relevant and they're generally all considered equivalent. Nobody cares if you publish in ApJ vs A&A vs MNRAS, if your research is good. There are also concerns that the quality of these venues is decreasing because the pressure to publish in them is so high. See https://arxiv.org/abs/1807.03341 https://arxiv.org/abs/1807.03341
- neuralsp 7y ago
  >I did a PhD in space science, I now do machine learning in ecology and spent the summer working on machine learning for disaster management. Do you think it is possible to that without any background in anything? I mean could someone apply black box frameworks without understanding them. How would they be caught?
  - joshvm 7y ago
    > Do you think it is possible to that without any background in anything? To do machine learning research? Or work in some random domain? > I mean could someone apply black box frameworks without understanding them. How would they be caught? Machine learning is rapidly becoming commoditised, but lots of people still don't understand just how much effort it is to get a good dataset and to prep Domain experts scoff at machine learning people who are trying to solve Big Problems using unrepresentative toy datasets, but also tend to have much higher expectations of what ML can do. Machine learning people scoff at domain experts for using outdated techniques and bad data science, but then propose ridiculous solutions that would never work in the real world (e.g. use our model, it takes a week on 8xV100s to train and you can only run it on a computer the size of a bus). There are also a lot of people (and companies) touting machine learning as a solution to problems that don't exist. Overfitting models is probably the most rampant crime that researchers commit.
    - neuralsp 7y ago
      My question is whether someone could fake it and not be caught/fired. (So yes, I meant: "Or work in some random domain?") From the second half of your comment it seems that the answer is yes? Maybe a comparison would help: someone pretending to be an experienced iOS/Android developer without any qualifications or ability would quickly be caught. Since they couldn't produce any working app or use a compiler, and anyone can judge an app for themselves. You can't really just make it up out of whole cloth, people judge the results. You would have to start actually doing that, and if you couldn't or didn't want to, then unless you outsourced your own job or something the jig would be up pretty much instantly. (Unless you caught up.) So, how about machine learning? Do you think a fraud could land and keep such a job, without any knowledge, qualifications, ability, or even interest in getting up to speed? Just, a pure, simple fraud. What's your guess?
mjn 7y ago
> Also, why is NEURIPS or ICML papers is not a hiring guarantee? I thought they're highly sought after. They're sought after, but the conferences have also grown huge. NeurIPS 2018 accepted around 1,000 papers! Based on a query of the DBLP [1] dataset, there were 4,409 distinct authors who had a paper at either NeurIPS 2018 or ICML 2018 (or both). If you add in a few of the other big AI and ML conferences (AAAI, IJCAI, ICLR), the number grows to 10,995 distinct authors, again solely for the year 2018. The field is hot, but is it hot enough for ten thousand people to be automatically hired because of one paper? There's also decreasing confidence in the big conferences' review processes I think. NeurIPS 2014 actually did a study to estimate how random acceptance was by assigning some papers to two different sets of reviewers and checking how similar the decisions were [2], and found there was a much higher degree of luck in acceptance/rejection decisions than they had expected. I personally have more confidence in the review processes of smaller and more focused conferences (and journals!), though they don't have the same level of name recognition. [1] https://dblp.uni-trier.de/ https://dblp.uni-trier.de/ [2] http://blog.mrtz.org/2014/12/15/the-nips-experiment.html http://blog.mrtz.org/2014/12/15/the-nips-experiment.html
- throwawayjava 7y ago
  A Ph.D.'s worth of first authored NeurIPS/ICML papers will get you a very good job pretty easily still. But AI slices papers very thin and author lists are inflated relative to other subfields of CS. A single paper in one of the major conferences is a pretty marginal contribution, especially if you're in the middle of a long author list. Also, NeurIPS reviewing has gone to absolute hell. I mean, peer review everywhere has problems. But I've never seen something quite this bad. At this point I think it's safe to say that most reviewers wouldn't even make it to an on-site interview for a faculty position at a research university. That's definitely nowhere near normal. You can't really blame anyone, I guess; the community is growing way too quickly for any real quality control. Frankly, I think those conferences have outlived their usefulness as anything except marquee marketing events. I'm now mostly attending smaller and more specialized conferences.
  - mjn 7y ago
    I've gone in a similar direction. Only at smaller conferences can you have any kind of confidence that your reviewers are people with actual expertise in the field. That's pretty useful, not only because it makes it less likely you'll get reviews that are very annoying, but also because a review by a knowledgeable person can be genuinely valuable. The big conferences are full of reviews written by 2nd-year grad students, because with this many submissions, any warm body with anything approaching credentials is needed. Besides just "quality" in the general sense, one thing this has really hurt, I think, is any sense of history or continuity. There are a ton of reviewers who have basically no familiarity with the pre-2010 ML literature, and it kind of shows in both the reviews and the papers that get published. I mean I get that deep learning beats a lot of older methods on major benchmarks, but it's still not the case that literally every problem, controversy, and technique was first studied post-2010.

rantwasp 7y ago

Isn't this always the case though? A PhD is a long term investment and the field that you go into is probably not the one that will be hot when you finish your PhD. So driving your decisions on $$$ seems foolish at best.

mellosouls 7y ago
Yep. If you're empty enough to follow a PhD solely for the money, there are far safer financial bets. Go into medicine or something. Not that I'd want to experience that bedside manner...
- nefitty 7y ago
  It is not fair to call someone "empty" because their motivation stems from something you don't share. It does not make their work and contribution any less difficult or necessary.
  - xpe 7y ago
    Though harsh and (of course) generalized, I think the word "empty" is a useful description here. Think of these meanings of empty: - "containing nothing; not filled or occupied" - "lacking meaning or sincerity" - "having no value or purpose" In the context of academia, there is an expectation of a certain fundamental motivation; namely, to pursue knowledge in a particular field and share it back via writing and/or teaching. If one lacks that primary motivation, they certainly "lack" it and are "not filled" with the academic ethos. So "empty" is a fair term. Now, I understand the realities as well -- seeking publications, recognition, tenure, and funding are also political activities. But this does not contradict the underlying community norm that I mention above. In fact, it supports it -- it explains why so many people endure a tough, grueling, political process despite it not being their wheelhouse.
    - rat9988 7y ago
      >If one lacks that primary motivation, they certainly "lack" it and are "not filled" with the academic ethos. So "empty" is a fair term. No idea why you think this. A phd has a contrat with a salary, starting and end dates.

hdrujvw-4579 7y ago

>I have my doubts there will be cushy FAANG jobs for everyone capable of playing around with network architectures in a few years. Presumably because technical needs of FAANG might be moving in other directions. Could someone comment why this might be the case, and what other fields might look relevant

sgt101 7y ago
I think that the other force is that the skills that are being created by ML Ph.D programs get commodified. I don't think this will happen, because typing "import tensorflow.keras.*" isn't the skill that an ML Ph.D develops, and it is the part of the skill set that is (and will be) commodity along with the automl stuff. Constructing a problem, handling the data and running a proper process is harder, and it's the value that will put processes that use ML at risk, and deliver differentiating value for the ones where it works.

jcims 7y ago

What about domain-specific applications of the technology? I've been in the information security industry for ~25 years and while 'AI' has been increasing in use but is still woefully short of its potential in a few different areas. It seems that someone with deep domain expertise that also has PhD level chops in ML would be a profit machine.