4 ms·

If I could add to this: realize that “AI” will be a small fraction of the overall product. And important one, one without which the product would likely not exi

If I could add to this: realize that “AI” will be a small fraction of the overall product. And important one, one without which the product would likely not exist, but in terms of effort spent it will be small. So no, you don’t need an army of ML researchers to ship it. You need maybe one or two, and the rest of your money is better spent on hiring engineers, sales/bizdev people, and a great product manager. People don’t give a shit about models. They do give a shit about whether your product solves a problem. Use AI as a means to an end, not an end in itself. Focus narrowly on a concrete problem, don’t try to do everything at once.

fnbr 9y ago
Yes, exactly. And most of the time, you can get by with a simple model like a random forest, so you just need to have a few analysts/engineers doing feature engineering, and feeding it into your model. Almost everyone using more complicated architectures (e.g. neural networks) are adding unnecessary complexity. [1] [1]: Unless you're doing computer vision.
- e_ameisen 9y ago
  Agreed, we've also written about why the best use of your time is to start with a baseline (even in Computer Vision!) https://blog.insightdatascience.com/always-start-with-a-stupid-model-no-exceptions-3a22314b9aaa https://blog.insightdatascience.com/always-start-with-a-stup...
  - throwaway84742 9y ago
    And before a baseline, start with formulating a problem in a way that’s applicable to a real world product. Not just “image classification” but “image classification to improve QA in underwater basket weaving, a 10 billion dollar industry, from which we could generate $100m/yr in revenue by selling a product that does A, B, and C to an estimated 100k customers each of which would have $1000 LTV”. I’m in an AI startup right now where the founders are hell bent on doing nothing in particular, and I’ll be bailing in a month or so.
    - didgeoridoo 9y ago
      I'd love to hear about your experiences — email in my profile. I've been doing some writing and speaking on why AI seduces founders into pursuing lots of cool ideas instead of marketable products. I'd love another data point. Could do it anonymously if you'd like, or hold off until after you bail.
      - deleted 9y ago
        [deleted]
x_tf2018 9y ago
I used to think this but after working in a large image processing team, I've come around to believe it's something management just rationalizes. In a lot of cases you _do_ need a big team with diverse research skills, otherwise you are deeply unable to pivot your own products and services for changes customer demands or adapt to situations when new models or competition elevate the minimal level of features that customers expect. Building this stuff in a way that won't fall apart is really hard and you need a lot of people who are happy to be focused on the integration, deployment, and maintenance for sure. But you do also need a lot of people who are dedicated to the research, prototype, and model performance understanding side too, without trying to squeeze extra work out of them by trying to sell roles as a "hybrid" between integration and research. Saying that "people don't give a shit about models" is vacuous, because ultimately the stack trace of what they do give a shit about will bottom out at issues with models that require a diverse team of experts to debug and solve. I think companies are still stuck in a dreamworld right now where they want to believe that very junior engineers with tangential interest in machine learning are going to slot in at your standard $130k - $160k job offer (in major urban areas) and do just as adequate of a job as senior experts who might earn twice that. But it doesn't work, and this type of penny pinching on the modeling side, rationalized by a false belief that you only need a tiny bit of research expenditure, I think is responsible for many start-ups failing and many projects being unwound in larger corporations.
newg208g 9y ago
I used to think this but after working in a large image processing team, I've come around to believe it's something management just rationalizes. In a lot of cases you _do_ need a big team with diverse research skills, otherwise you are deeply unable to pivot your own products and services for changing customer demands or adapt to situations when new models or competition elevate the minimal level of features that customers expect. Building this stuff in a way that won't fall apart is really hard and you need a lot of people who are happy to be focused on the integration, deployment, and maintenance for sure. But you do also need a lot of people who are dedicated to the research, prototype, and model performance understanding side too, without trying to squeeze extra work out of them by trying to sell roles as a "hybrid" between integration and research. Saying that "people don't give a shit about models" is vacuous, because ultimately the stack trace of what they do give a shit about will bottom out at issues with models that require a diverse team of experts to debug and solve. I think companies are still stuck in a dreamworld right now where they want to believe that very junior engineers with tangential interest in machine learning are going to slot in at your standard $130k - $160k job offer (in major urban areas) and do just as adequate of a job as senior experts who might earn twice that. But it doesn't work, and this type of penny pinching on the modeling side, rationalized by a false belief that you only need a tiny bit of research expenditure, I think is responsible for many start-ups failing and many projects being unwound in larger corporations.

fnbr 9y ago

Yes, exactly. And most of the time, you can get by with a simple model like a random forest, so you just need to have a few analysts/engineers doing feature engineering, and feeding it into your model. Almost everyone using more complicated architectures (e.g. neural networks) are adding unnecessary complexity. [1] [1]: Unless you're doing computer vision.

e_ameisen 9y ago
Agreed, we've also written about why the best use of your time is to start with a baseline (even in Computer Vision!) https://blog.insightdatascience.com/always-start-with-a-stupid-model-no-exceptions-3a22314b9aaa https://blog.insightdatascience.com/always-start-with-a-stup...
- throwaway84742 9y ago
  And before a baseline, start with formulating a problem in a way that’s applicable to a real world product. Not just “image classification” but “image classification to improve QA in underwater basket weaving, a 10 billion dollar industry, from which we could generate $100m/yr in revenue by selling a product that does A, B, and C to an estimated 100k customers each of which would have $1000 LTV”. I’m in an AI startup right now where the founders are hell bent on doing nothing in particular, and I’ll be bailing in a month or so.
  - didgeoridoo 9y ago
    I'd love to hear about your experiences — email in my profile. I've been doing some writing and speaking on why AI seduces founders into pursuing lots of cool ideas instead of marketable products. I'd love another data point. Could do it anonymously if you'd like, or hold off until after you bail.
    - deleted 9y ago
      [deleted]

x_tf2018 9y ago

I used to think this but after working in a large image processing team, I've come around to believe it's something management just rationalizes. In a lot of cases you _do_ need a big team with diverse research skills, otherwise you are deeply unable to pivot your own products and services for changes customer demands or adapt to situations when new models or competition elevate the minimal level of features that customers expect. Building this stuff in a way that won't fall apart is really hard and you need a lot of people who are happy to be focused on the integration, deployment, and maintenance for sure. But you do also need a lot of people who are dedicated to the research, prototype, and model performance understanding side too, without trying to squeeze extra work out of them by trying to sell roles as a "hybrid" between integration and research. Saying that "people don't give a shit about models" is vacuous, because ultimately the stack trace of what they do give a shit about will bottom out at issues with models that require a diverse team of experts to debug and solve. I think companies are still stuck in a dreamworld right now where they want to believe that very junior engineers with tangential interest in machine learning are going to slot in at your standard $130k - $160k job offer (in major urban areas) and do just as adequate of a job as senior experts who might earn twice that. But it doesn't work, and this type of penny pinching on the modeling side, rationalized by a false belief that you only need a tiny bit of research expenditure, I think is responsible for many start-ups failing and many projects being unwound in larger corporations.

newg208g 9y ago

I used to think this but after working in a large image processing team, I've come around to believe it's something management just rationalizes. In a lot of cases you _do_ need a big team with diverse research skills, otherwise you are deeply unable to pivot your own products and services for changing customer demands or adapt to situations when new models or competition elevate the minimal level of features that customers expect. Building this stuff in a way that won't fall apart is really hard and you need a lot of people who are happy to be focused on the integration, deployment, and maintenance for sure. But you do also need a lot of people who are dedicated to the research, prototype, and model performance understanding side too, without trying to squeeze extra work out of them by trying to sell roles as a "hybrid" between integration and research. Saying that "people don't give a shit about models" is vacuous, because ultimately the stack trace of what they do give a shit about will bottom out at issues with models that require a diverse team of experts to debug and solve. I think companies are still stuck in a dreamworld right now where they want to believe that very junior engineers with tangential interest in machine learning are going to slot in at your standard $130k - $160k job offer (in major urban areas) and do just as adequate of a job as senior experts who might earn twice that. But it doesn't work, and this type of penny pinching on the modeling side, rationalized by a false belief that you only need a tiny bit of research expenditure, I think is responsible for many start-ups failing and many projects being unwound in larger corporations.