4 ms·

As someone near graduation with a PhD in mathematics with research focusing on machine learning / image processing and beginning the job search, this didn't lea

As someone near graduation with a PhD in mathematics with research focusing on machine learning / image processing and beginning the job search, this didn't leave me feeling very optimistic. Is this a problem of not setting proper expectations for a role on the company's part or not fully understanding the expected duties of the position before hired on the data scientist's part?

ska 9y ago
It's a bit of both. You are unlikely to ever achieve a position in industry that has the same freedom of investigation and time that you experienced as a graduate student, let alone the holy grail of a self-funded post-doc or the like. On the other hand, you won't achieve that as faculty either. In industry you're going to end up spending a lot of time on activities and meetings that don't relate to what you now consider "what you do" (e.g. ETL, domain interview, co-ordination etc.). Your success and happiness in this will depend a lot on whether you consider those things a waste of time taking you away from what you "really do", or an important part of how you do such activities in larger groups. On the other hand, data science has a similar problem as software programming; failing to know how to find or produce good managers for it, companies tend to promote from the ranks of their best individual contributors. This can work well with enough support and mentorship (assuming the candidate wants to do it) but it can be a disaster without a plan and that support.
- chestervonwinch 9y ago
  > You are unlikely to ever achieve a position in industry that has the same freedom of investigation and time that you experienced as a graduate student, let alone the holy grail of a self-funded post-doc or the like. On the other hand, you won't achieve that as faculty either. which explains my advisor's "enjoy it while you still can!" attitude whenever we talk of such things :)
  - ska 9y ago
    They speak truth :)
soVeryTired 9y ago
A major point that's underappreciated in academia is quality of execution. The more complicated a model is, the less well you can execute it. Academics are rewarded for building complicated things, but it often takes graduate PhDs entering industry a long time to learn about quality of execution. I think that's the fundamental tension between business and academia.
- ska 9y ago
  I wanted to say that I disagreed with you, but then I realized I don't know what you mean by "execution" here, though. This statement "The more complicated a model is, the less well you can execute it" is fishy though. The typical problem in practice with complicated models is the data quality and quantity to support it, not the implementation or interpretation. I suppose here we also need to define "complexity" - I am using this to mean roughly number of parameters.
  - jackcosgrove 9y ago
    I took "execution" here to mean "scaling up". In a trivial case, a model may fail to scale up if it's computationally too slow or too sensitive to noisy data. Sometimes these shortcomings cannot be tested in a laboratory environment with limited data sets.
RHSman2 9y ago
Get in with a small, clever company. Do, fail and repeat but better. Just like data science. Large companies. Meh.

ska 9y ago

It's a bit of both. You are unlikely to ever achieve a position in industry that has the same freedom of investigation and time that you experienced as a graduate student, let alone the holy grail of a self-funded post-doc or the like. On the other hand, you won't achieve that as faculty either. In industry you're going to end up spending a lot of time on activities and meetings that don't relate to what you now consider "what you do" (e.g. ETL, domain interview, co-ordination etc.). Your success and happiness in this will depend a lot on whether you consider those things a waste of time taking you away from what you "really do", or an important part of how you do such activities in larger groups. On the other hand, data science has a similar problem as software programming; failing to know how to find or produce good managers for it, companies tend to promote from the ranks of their best individual contributors. This can work well with enough support and mentorship (assuming the candidate wants to do it) but it can be a disaster without a plan and that support.

chestervonwinch 9y ago
> You are unlikely to ever achieve a position in industry that has the same freedom of investigation and time that you experienced as a graduate student, let alone the holy grail of a self-funded post-doc or the like. On the other hand, you won't achieve that as faculty either. which explains my advisor's "enjoy it while you still can!" attitude whenever we talk of such things :)
- ska 9y ago
  They speak truth :)

soVeryTired 9y ago

A major point that's underappreciated in academia is quality of execution. The more complicated a model is, the less well you can execute it. Academics are rewarded for building complicated things, but it often takes graduate PhDs entering industry a long time to learn about quality of execution. I think that's the fundamental tension between business and academia.

ska 9y ago
I wanted to say that I disagreed with you, but then I realized I don't know what you mean by "execution" here, though. This statement "The more complicated a model is, the less well you can execute it" is fishy though. The typical problem in practice with complicated models is the data quality and quantity to support it, not the implementation or interpretation. I suppose here we also need to define "complexity" - I am using this to mean roughly number of parameters.
- jackcosgrove 9y ago
  I took "execution" here to mean "scaling up". In a trivial case, a model may fail to scale up if it's computationally too slow or too sensitive to noisy data. Sometimes these shortcomings cannot be tested in a laboratory environment with limited data sets.

RHSman2 9y ago

Get in with a small, clever company. Do, fail and repeat but better. Just like data science. Large companies. Meh.