3 ms·

As a researcher applying ML in my work the OP reads like me looking from the outside at web-frameworks: there's a LOT of noise but in practice, people run just

As a researcher applying ML in my work the OP reads like me looking from the outside at web-frameworks: there's a LOT of noise but in practice, people run just the same 3 things. For example, when I first got into ML the ADAM optimizer was the new big thing, since then hundreds of 'better' optimizers have been published. This paper from August '21 shows that most of that is overblown and no optimizer consistently outperforms ADAM: https://arxiv.org/pdf/2007.01547.pdf https://arxiv.org/pdf/2007.01547.pdf I'd even go so far and say that OP's big image of MLOps is misleading. 'Data Science Notebooks' shows Jupyter, binder, colab, I can't see the other logos; but binder and colab both run Jupyter notebooks? Under ML platforms there are a ton of companies in this picture which effectively do the same thing. Some of these logos are tools (Jupyter, R), some of these logos are companies using ML in some way or other (John Deere, Siemens) - and once you go down this path you might as well put any mid-sized company in the world onto this.

awestroke 5y ago
So, what's the same 3 things that everybody runs?
- cnimativ 5y ago
  import torch, tensorflow, sklearn model.fit model.predict
  - kajecounterhack 5y ago
    Except where's the data extraction, warehousing, labeling, deployment, and eval infra here? Are you calling `model.fit` on distributed training infra or just on your local machine? I think that's the OP's point.
kajecounterhack 5y ago
OP doesn't weight the size of each technology by adoption -- if they did that then yeah there are some centroids. But I think their point is correct that parts of the ecosystem remain quite fragmented. > For example, when I first got into ML the ADAM optimizer was the new big thing, since then hundreds of 'better' optimizers have been published. This paper from August '21 shows that most of that is overblown and no optimizer consistently outperforms ADAM: https://arxiv.org/pdf/2007.01547.pdf https://arxiv.org/pdf/2007.01547.pdf FWIW this paper acknowledges that optimizer is application-dependent, yet draws its conclusions from < 10 canned datasets (mostly classification). "no optimizer consistently outperforms ADAM" is a misleading statement -- I think a more correct one is "no optimizer consistently outperforms ADAM on {MNIST, CIFAR-10, SVHN}". If you do work on much larger and more novel architectures / datasets, I think you'll find the advice to "just use adam" to be insufficient. At least at the orgs I've worked at, folks often do a (sometimes smart) param sweep to find best optimizer / settings << if you're lucky enough to have infra that supports this :) That aside and more generally, there certainly are commonly accepted best practices in ML. But that doesn't mean ML organizations all work the same way, and the OP's point is that the dust has not settled on what the best tools are to do X.

awestroke 5y ago

So, what's the same 3 things that everybody runs?

cnimativ 5y ago
import torch, tensorflow, sklearn model.fit model.predict
- kajecounterhack 5y ago
  Except where's the data extraction, warehousing, labeling, deployment, and eval infra here? Are you calling `model.fit` on distributed training infra or just on your local machine? I think that's the OP's point.

kajecounterhack 5y ago

OP doesn't weight the size of each technology by adoption -- if they did that then yeah there are some centroids. But I think their point is correct that parts of the ecosystem remain quite fragmented. > For example, when I first got into ML the ADAM optimizer was the new big thing, since then hundreds of 'better' optimizers have been published. This paper from August '21 shows that most of that is overblown and no optimizer consistently outperforms ADAM: https://arxiv.org/pdf/2007.01547.pdf https://arxiv.org/pdf/2007.01547.pdf FWIW this paper acknowledges that optimizer is application-dependent, yet draws its conclusions from < 10 canned datasets (mostly classification). "no optimizer consistently outperforms ADAM" is a misleading statement -- I think a more correct one is "no optimizer consistently outperforms ADAM on {MNIST, CIFAR-10, SVHN}". If you do work on much larger and more novel architectures / datasets, I think you'll find the advice to "just use adam" to be insufficient. At least at the orgs I've worked at, folks often do a (sometimes smart) param sweep to find best optimizer / settings << if you're lucky enough to have infra that supports this :) That aside and more generally, there certainly are commonly accepted best practices in ML. But that doesn't mean ML organizations all work the same way, and the OP's point is that the dust has not settled on what the best tools are to do X.