3 ms·

I have never understood the what a good ML engineer couldn't do and a Data scientist could in _majority_ situations. When you need a decision to be made based o

I have never understood the what a good ML engineer couldn't do and a Data scientist could in _majority_ situations. When you need a decision to be made based on data its just common sense risk analysis added together with basic statistics. I feel some good field training in statistics(Look up Andrew Gelman) a couple of good courses on Linear, Bayesian Regression is all you need, rest is just engineering skill. The dichotomy between ML Engg and Datascience is as stupid as was between Systems Engg and Application Engg before Devops came along.

TrackerFF 4y ago
IMO a data scientist should also be a domain expert, in the same way analysts are. But of course, too many view DS as some abstract skill where domain knowledge is not needed, and where the methodology will solve all problems / provide insight.
- _Wintermute 4y ago
  I agree completely, but if a data scientist should be a domain expert, surely we should just focus more on programming and quantitative skills in these fields?
adamsmith143 4y ago
Can you make pretty graphs in Matplotlib and sound smart in front of Executives?
time_to_smile 4y ago
I think the qualifying term here is "good". I've worked with a surprising number of MLEs that don't really understand gradient descent or how most models really work under the hood. They certainly couldn't implement most things from scratch if they needed to (neither could most data scientists). I used to think an MLE was a solid engineer who also had a strong quantitative and numerical computing background. The kind of engineer that always has a copy of Numerical Recipes handy, and if needed, could reimplement core components of statsmodels and sklearn in javascript. I think after this current contraction in tech is over we'll see that most of the remaining "data scientists/MLEs" will be the type of engineer I imagine an MLE to be.
- itissid 4y ago
  > I used to think an MLE was a solid engineer who also had a strong quantitative and numerical computing background. Application of Computational Stats/ML Models are not all that hard to aquire but essential. I think we need a fundamental rethink of how applied stats/ML is taught to engineers to make them effective. Here are a few things I can think of: 1. Getting a solid understanding of actually coming up with a simple enough model to do the job 2. Do Power Analysis to figure out how many samples we need. Creating datasets with Hard Negatives and overcoming sampling bias. 3. Using things like Multiple Regression to do EDA. i.e. using models as a tool vs the end goal to understand a problem space.
- xxpor 4y ago
  >They certainly couldn't implement most things from scratch if they needed to (neither could most data scientists). Could most CS folks actually implement Linux or Chromium from scratch?
  - time_to_smile 4y ago
    I'm clearly talking about quantitative modeling tools. That said, while Linux and Chromium are massive projects each with years of development with thousands of engineers behind them, so of course it would be ridiculous to expect a single engineer to build such a thing. I also wouldn't expect an MLE to build SKLearn entirely as is from scratch on their own. However, I do certainly hope most CS folks could implement an OS or Web browser from scratch.
    - xxpor 4y ago
      >However, I do certainly hope most CS folks could implement an OS or Web browser from scratch. I am way, way less optimistic than you then. I doubt even 10% of CS grads, let alone people who have been out of school for a few years, could tell you what a page table is.

TrackerFF 4y ago

IMO a data scientist should also be a domain expert, in the same way analysts are. But of course, too many view DS as some abstract skill where domain knowledge is not needed, and where the methodology will solve all problems / provide insight.

_Wintermute 4y ago
I agree completely, but if a data scientist should be a domain expert, surely we should just focus more on programming and quantitative skills in these fields?

adamsmith143 4y ago

Can you make pretty graphs in Matplotlib and sound smart in front of Executives?

time_to_smile 4y ago

I think the qualifying term here is "good". I've worked with a surprising number of MLEs that don't really understand gradient descent or how most models really work under the hood. They certainly couldn't implement most things from scratch if they needed to (neither could most data scientists). I used to think an MLE was a solid engineer who also had a strong quantitative and numerical computing background. The kind of engineer that always has a copy of Numerical Recipes handy, and if needed, could reimplement core components of statsmodels and sklearn in javascript. I think after this current contraction in tech is over we'll see that most of the remaining "data scientists/MLEs" will be the type of engineer I imagine an MLE to be.

itissid 4y ago
> I used to think an MLE was a solid engineer who also had a strong quantitative and numerical computing background. Application of Computational Stats/ML Models are not all that hard to aquire but essential. I think we need a fundamental rethink of how applied stats/ML is taught to engineers to make them effective. Here are a few things I can think of: 1. Getting a solid understanding of actually coming up with a simple enough model to do the job 2. Do Power Analysis to figure out how many samples we need. Creating datasets with Hard Negatives and overcoming sampling bias. 3. Using things like Multiple Regression to do EDA. i.e. using models as a tool vs the end goal to understand a problem space.
xxpor 4y ago
>They certainly couldn't implement most things from scratch if they needed to (neither could most data scientists). Could most CS folks actually implement Linux or Chromium from scratch?
- time_to_smile 4y ago
  I'm clearly talking about quantitative modeling tools. That said, while Linux and Chromium are massive projects each with years of development with thousands of engineers behind them, so of course it would be ridiculous to expect a single engineer to build such a thing. I also wouldn't expect an MLE to build SKLearn entirely as is from scratch on their own. However, I do certainly hope most CS folks could implement an OS or Web browser from scratch.
  - xxpor 4y ago
    >However, I do certainly hope most CS folks could implement an OS or Web browser from scratch. I am way, way less optimistic than you then. I doubt even 10% of CS grads, let alone people who have been out of school for a few years, could tell you what a page table is.