3 ms·

You nailed where I currently stand. At my company I've been a jack of all trades but mostly software/dba work. My boss and I were very excited about ML when the

You nailed where I currently stand. At my company I've been a jack of all trades but mostly software/dba work. My boss and I were very excited about ML when the hype cycle was taking off several years ago and completed a successful project. Fast forward to today, I got loaned out to another team that lost their data scientist, and for the first time in my career I'm having to say - "I don't think we can do what you want." To me the "science" part really stands out. I have a decent grasp of methodologies and tools, but after weeks of dissecting the issue my conclusion is that they just don't have enough useful data...

throwaway8877 3y ago
The situation is not bad then. Can they collect more data? Can they generate more data?
- coldtea 3y ago
  A more relevant question would be: Is "not enough data" their problem, or the kind of data?
jxramos 3y ago
It can be a very empirical art. If you can't generate more data at the time you can sometimes invest in reviewing the hand labeling ground truth to verify no false classifications slipped by.
j7ake 3y ago
This is where good simulations are useful. If you can show even in ideal simple data scenarios, you encounter inference problems, it’s a strong signal that real data has little chance of doing better. In general these are model identifiability issues.
PeterisP 3y ago
Every data science team should have a wall decoration with John Tukey's quote "The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data."
steve_adams_86 3y ago
> they just don't have enough useful data... I'm not well-versed in this, or not as well as you are, but this has been my conclusion as well about a lot of ML project ideas from teams I've been on. You need so much data to do useful things. Especially the magical kinds of things people tend to want to do. I think these types of datasets are on a scale most software developers typically don't see. Even with the data in hand, it's nothing like trivial to determine how to do something half-way useful with it.

throwaway8877 3y ago

The situation is not bad then. Can they collect more data? Can they generate more data?

coldtea 3y ago
A more relevant question would be: Is "not enough data" their problem, or the kind of data?

jxramos 3y ago

It can be a very empirical art. If you can't generate more data at the time you can sometimes invest in reviewing the hand labeling ground truth to verify no false classifications slipped by.

j7ake 3y ago

This is where good simulations are useful. If you can show even in ideal simple data scenarios, you encounter inference problems, it’s a strong signal that real data has little chance of doing better. In general these are model identifiability issues.

PeterisP 3y ago

Every data science team should have a wall decoration with John Tukey's quote "The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data."

steve_adams_86 3y ago

> they just don't have enough useful data... I'm not well-versed in this, or not as well as you are, but this has been my conclusion as well about a lot of ML project ideas from teams I've been on. You need so much data to do useful things. Especially the magical kinds of things people tend to want to do. I think these types of datasets are on a scale most software developers typically don't see. Even with the data in hand, it's nothing like trivial to determine how to do something half-way useful with it.