3 ms·

> Speaking to Ars, Amazon Web Services Senior Product Manager Danny Smith explained a non-obvious conundrum: Waiting for a data scientist to analyze one's data

> Speaking to Ars, Amazon Web Services Senior Product Manager Danny Smith explained a non-obvious conundrum: Waiting for a data scientist to analyze one's data "is like waiting for an IT project," he said, noting that biz analyst customers have told him things like, "'We have to wait 18 months to just get to the front of the queue.' And then the data scientists, who are all scarce resources, they're like, 'Well, OK, what's the project's ROI?'" > Smith continued: "And the business analyst is like, 'If I knew that, I wouldn't need your help!... I've got this data, I've got a business problem. I've got questions for this data. I don't know what it's worth until we actually do the analysis.' So the business analyst is thinking about just accelerating to the point where they can actually argue for or justify their use of a scarce data scientist." Well as a data scientist I can tell you the ROI of this project is 0 because its a worthless model. Or really negative because it seems like it'd be pretty dangerous to use in practice. The critical error here is having the business analyst scope this out at all They don't know what they're doing. They don't know how to evaluate the results or predict the likely outcomes.

nojito 4y ago
>They don't know how to evaluate the results or predict the likely outcomes. Hard disagree. Interpreting baseline models is very simple these days. Granted our models are based on tabular data so YMMV.
- spywaregorilla 4y ago
  Oh yeah? So what are you going to do with this heart attack model? Because I sure hope the answer isn't "model said heart attack so the answer is heart attack"
  - nojito 4y ago
    We don’t for clinical outcomes. There are way more interesting things to model in healthcare than clinical outcomes.
woeirua 4y ago
This is the major problem with the way data science is managed in most organizations. You typically have business analysts or other management types queuing up work that they _think_ could go somewhere, but they have typically not done the necessary legwork to figure out if a market actually exists for the solution that you're going to build. If a business analyst can't tell you: 1 - What problem you're trying to solve. 2 - What a valuable solution to that problem requires (in gory detail), including model metrics. 3 - What a customer is willing to pay for that solution. 4 - What the timeframe looks like for a valuable solution. 5 - How many resources (people, money) the company is willing to devote to this project. Then the ROI on that project is zero, because there is a very high likelihood that any solution you build will not satisfy the demands of the market.
- vianneychevalie 4y ago
  Can confirm, from experience in Data Management Consulting in large Banking, Insurance, Facilities Management and Food services companies: they build solutions and hope for ROI.

nojito 4y ago

>They don't know how to evaluate the results or predict the likely outcomes. Hard disagree. Interpreting baseline models is very simple these days. Granted our models are based on tabular data so YMMV.

spywaregorilla 4y ago
Oh yeah? So what are you going to do with this heart attack model? Because I sure hope the answer isn't "model said heart attack so the answer is heart attack"
- nojito 4y ago
  We don’t for clinical outcomes. There are way more interesting things to model in healthcare than clinical outcomes.

woeirua 4y ago

This is the major problem with the way data science is managed in most organizations. You typically have business analysts or other management types queuing up work that they _think_ could go somewhere, but they have typically not done the necessary legwork to figure out if a market actually exists for the solution that you're going to build. If a business analyst can't tell you: 1 - What problem you're trying to solve. 2 - What a valuable solution to that problem requires (in gory detail), including model metrics. 3 - What a customer is willing to pay for that solution. 4 - What the timeframe looks like for a valuable solution. 5 - How many resources (people, money) the company is willing to devote to this project. Then the ROI on that project is zero, because there is a very high likelihood that any solution you build will not satisfy the demands of the market.

vianneychevalie 4y ago
Can confirm, from experience in Data Management Consulting in large Banking, Insurance, Facilities Management and Food services companies: they build solutions and hope for ROI.