3 ms·

I'm actually going to take the counter-side of this argument. It's actually very important for people to do the model work first. Yes, you need some amount of d

I'm actually going to take the counter-side of this argument. It's actually very important for people to do the model work first. Yes, you need some amount of decent data, but the first step of a project should always be building a prototype to answer the question "Can we fundamentally do this". It's perfectly reasonable in that case to do a small amount of work picking and cleaning up some data and then really focusing on building the model. Once you've got your model then you can answer the best case scenario of whether the problem is tractable. Once you've got the model then you need to work on operationalise, and that's where work on your data pipeline becomes important. It's always going to be the case though that building your initial prototype is more fun/interesting and probably 1000x easier, than getting your prototype into production. And it's important to remember that operationalizing your model is probably going to involve working on your model, retraining it and refining it to work with your new production environment. The authors have tried to dress their waterfall up as "cascades" but the truth is that they've defined the development process as a waterfall and are trying to fix that rather than adopting modern development practices.

kfk 6y ago
I agree we need to start from an mvp and so with a data model, but does this model really need to be AI or ML? My counter point to your counter point is yes, we need a model, but probably the initial one can be done with simple BI/SQL type aggregation methods. Most of AI/ML I see is optimizing an already existing model, not create something completely new.
- disgruntledphd2 6y ago
  I'd suggest using a lasso, tbh. It's pretty fast, relatively performant and one can interpret the coefficients. That being said, I'd do your simple SQL aggregations first, to ensure that one can actually learn something (if we have 0.0001% positives, a simple approach probably isn't going to work well).
- Ntrails 6y ago
  A few years back the team that do "Business analysis" came to our team asking for help setting up some machine learning. What for, we asked. Well, we'd like to predict hardware expenditures over the next n years. Ok. Well that sure seems like a thing the business might like an estimate of! Sure. Next question: Have you run a basic regression over the data you've got, maybe a simple model would be close enough to throw some error bars up and get going. Of course not.
  - kfk 6y ago
    You see "we'd like to predict hardware expenditures over the next n years" for me is a red flag. When I hear those types of requests I hear "we'd like to shift responsibility and ownership of our financial estimates to someone else".
roytries 6y ago
I don't know. How can you gauge the effectiveness of your model without good data? A model can be wrong, but it can also be changed/redone/rebuilt. Without data you can neither develop nor verify imho.
- quantumofalpha 6y ago
  It gives you a working MVP and a lower bound on what's possible. Tnen you can iterate either on model or data or both - data can also be further cleaned, changed and expanded. How can you gauge ROI of your data cleaning/collection efforts without it? Maybe an extra day you've put into it made a difference, maybe it didn't and you'd have better spent it on modelling or other tasks.

kfk 6y ago

I agree we need to start from an mvp and so with a data model, but does this model really need to be AI or ML? My counter point to your counter point is yes, we need a model, but probably the initial one can be done with simple BI/SQL type aggregation methods. Most of AI/ML I see is optimizing an already existing model, not create something completely new.

disgruntledphd2 6y ago
I'd suggest using a lasso, tbh. It's pretty fast, relatively performant and one can interpret the coefficients. That being said, I'd do your simple SQL aggregations first, to ensure that one can actually learn something (if we have 0.0001% positives, a simple approach probably isn't going to work well).
Ntrails 6y ago
A few years back the team that do "Business analysis" came to our team asking for help setting up some machine learning. What for, we asked. Well, we'd like to predict hardware expenditures over the next n years. Ok. Well that sure seems like a thing the business might like an estimate of! Sure. Next question: Have you run a basic regression over the data you've got, maybe a simple model would be close enough to throw some error bars up and get going. Of course not.
- kfk 6y ago
  You see "we'd like to predict hardware expenditures over the next n years" for me is a red flag. When I hear those types of requests I hear "we'd like to shift responsibility and ownership of our financial estimates to someone else".

roytries 6y ago

I don't know. How can you gauge the effectiveness of your model without good data? A model can be wrong, but it can also be changed/redone/rebuilt. Without data you can neither develop nor verify imho.

quantumofalpha 6y ago
It gives you a working MVP and a lower bound on what's possible. Tnen you can iterate either on model or data or both - data can also be further cleaned, changed and expanded. How can you gauge ROI of your data cleaning/collection efforts without it? Maybe an extra day you've put into it made a difference, maybe it didn't and you'd have better spent it on modelling or other tasks.