4 ms·

I'm struggling to understand what people think is so difficult about all this data science stuff. The maths is very basic, even in "advanced" ml. Nor is it hard

I'm struggling to understand what people think is so difficult about all this data science stuff. The maths is very basic, even in "advanced" ml. Nor is it hard to learn backend software engineering for the purposes of 99% of companies.

visarga 5y ago
By the same logic what is so difficult about programming computers - it's just a bunch of zeroes and ones, very basic operations.
v8dev123 5y ago
I spent 15 years of my damn life to become a dev and you don't know what's it like to be a beginner. If you can re-read what you wrote with a beginner's mind, you will see how wrong you are.
sdenton4 5y ago
It's all about epistemology. How do we know what we think we know? How do we come to know things we didn't know before? And how can we trust those conclusions? Even if the math is basic, it's really, really easy to draw bad conclusions, look at the wrong problems, not realize that your data is more incomplete than you might think, etc etc etc. Guarding against these bad results - figuring out how to actually manufacture new knowledge - is the heart of the problem.
- kortex 5y ago
  > How do we come to know things we didn't know before? And how can we trust those conclusions? IMHO, this is the heart of what discerns scientists from engineers. Yes there is plenty of overlap, but to me this is the principle component. In engineering, the correct answer almost always exists. Enough eyeballs on the problem, sufficient double- and triple-checking converges on high confidence. Scientific problems may not have a right answer, or gaining confidence has diminishing returns, and at some point you decide that's enough sigmas. You can scrutinize in so many ways but there's always blind spots. (Data) scientists generally have to be way more comfortable with uncertainty. And as you mention, the easiest person to fool is yourself.
amcoastal 5y ago
99% of companies? Definitely not. The skills needed to do DS in business or healthcare are not very correlated with doing DS for the physical sciences. Which is the whole point of this comment thread, sure you can understand DL, but you also have to have an understanding of the field to know what type of DL to use. For example, in my role, I came with knowledge of machine learning but had to learn complex fluid physics to be able to know what type of DL techniques to apply or develop.

visarga 5y ago

By the same logic what is so difficult about programming computers - it's just a bunch of zeroes and ones, very basic operations.

v8dev123 5y ago

I spent 15 years of my damn life to become a dev and you don't know what's it like to be a beginner. If you can re-read what you wrote with a beginner's mind, you will see how wrong you are.

sdenton4 5y ago

It's all about epistemology. How do we know what we think we know? How do we come to know things we didn't know before? And how can we trust those conclusions? Even if the math is basic, it's really, really easy to draw bad conclusions, look at the wrong problems, not realize that your data is more incomplete than you might think, etc etc etc. Guarding against these bad results - figuring out how to actually manufacture new knowledge - is the heart of the problem.

kortex 5y ago
> How do we come to know things we didn't know before? And how can we trust those conclusions? IMHO, this is the heart of what discerns scientists from engineers. Yes there is plenty of overlap, but to me this is the principle component. In engineering, the correct answer almost always exists. Enough eyeballs on the problem, sufficient double- and triple-checking converges on high confidence. Scientific problems may not have a right answer, or gaining confidence has diminishing returns, and at some point you decide that's enough sigmas. You can scrutinize in so many ways but there's always blind spots. (Data) scientists generally have to be way more comfortable with uncertainty. And as you mention, the easiest person to fool is yourself.

amcoastal 5y ago

99% of companies? Definitely not. The skills needed to do DS in business or healthcare are not very correlated with doing DS for the physical sciences. Which is the whole point of this comment thread, sure you can understand DL, but you also have to have an understanding of the field to know what type of DL to use. For example, in my role, I came with knowledge of machine learning but had to learn complex fluid physics to be able to know what type of DL techniques to apply or develop.