2 ms·

> Anecdotally, I've worked in high-performance computing and machine learning for years now and the past few months I've seen a huge spike in the number of mess

> Anecdotally, I've worked in high-performance computing and machine learning for years now and the past few months I've seen a huge spike in the number of messages I get for MLOps positions. I think companies are slowly starting to realize that setting up machine learning at scale isn't as simple as deploying poorly written code by research scientists to managed platforms. This doesn’t bode well for ML in lots of orgs. Obviously machine learning is very powerful and effective in many use cases. But the value proposition already isn’t there for lots of companies. At such places, discovering a hidden requirement for more resources is a great reason to change directions. To be very clear, I’m not talking about the field in general. I’m talking about orgs where management doesn’t see value generated from the ML efforts that suddenly demand more resources to operate.

sitkack 4y ago
Could you explain the statement, "At such places, discovering a hidden requirement for more resources is a great reason to change directions." I don't understand the argument, not feigning confusion. Everyone has to scale at some point and every solution has its limits. If they were successful with Airflow and Pandas/Numpy for a long time and then well, now the fan is spinning. They are going to call for experts that put the pieces in place. Asking for help is a sign of maturity. It really depends on the state of system when the experts arrive. I personally think every org can use ML (it all decays to statistics, then linear algebra).
- nonethewiser 4y ago
  I’m responding to the idea that MLops infrastructure is heavily underestimated. In which case you need more time, expertise, infrastructure, etc. than previously realized. AKA money I agree with everything you said for whatever it’s worth. I’m mostly just making the observation that there is a lot of machine learning endeavors that aren’t generating much value.
  - sitkack 4y ago
    > a lot of machine learning endeavors that aren’t generating much value. I agree with this. From what I have seen, execs want something fancy, when you could give a boring-ish tool that reduces your OODA loop cycle time to 30% of what it was using unsexy techniques. Much of it having to do with tolerancing of answers, being within 5-10% is more than enough to drive the business, but someone somewhere said it had to be exact and that blows out the latency budget. When engaging in consulting gigs, it is super important to know what kind of org you dealing with before you get involved. The myopic penny pinching orgs should be steered well away from, which I think was your point.

sitkack 4y ago

Could you explain the statement, "At such places, discovering a hidden requirement for more resources is a great reason to change directions." I don't understand the argument, not feigning confusion. Everyone has to scale at some point and every solution has its limits. If they were successful with Airflow and Pandas/Numpy for a long time and then well, now the fan is spinning. They are going to call for experts that put the pieces in place. Asking for help is a sign of maturity. It really depends on the state of system when the experts arrive. I personally think every org can use ML (it all decays to statistics, then linear algebra).

nonethewiser 4y ago
I’m responding to the idea that MLops infrastructure is heavily underestimated. In which case you need more time, expertise, infrastructure, etc. than previously realized. AKA money I agree with everything you said for whatever it’s worth. I’m mostly just making the observation that there is a lot of machine learning endeavors that aren’t generating much value.
- sitkack 4y ago
  > a lot of machine learning endeavors that aren’t generating much value. I agree with this. From what I have seen, execs want something fancy, when you could give a boring-ish tool that reduces your OODA loop cycle time to 30% of what it was using unsexy techniques. Much of it having to do with tolerancing of answers, being within 5-10% is more than enough to drive the business, but someone somewhere said it had to be exact and that blows out the latency budget. When engaging in consulting gigs, it is super important to know what kind of org you dealing with before you get involved. The myopic penny pinching orgs should be steered well away from, which I think was your point.