5 ms·

The title doesn't really match the article in my mind. To me, it talks about everything but actually deploying a machine learning model in production. In partic

The title doesn't really match the article in my mind. To me, it talks about everything but actually deploying a machine learning model in production. In particular, there are a lot of words around where training data is stored. In my experience, the training data is really more part of the the development process than the actual productionisation of the model. That said, there is a piece here on TFX, which is valuable in this context. I also think the advice about going with proprietary tools that speed up the process is good. Tools like Microsoft's AI tooling, Dataiku and H20 are good in that context. I would have liked to have seen some discussion around when you should deploy a model as an API vs generating batch predictions and storing them - I've done both on a test bench, but I don't really know how well the API scales.

dcl 6y ago
> To me, it talks about everything but actually deploying a machine learning model in production. This seems to be a common theme of a lot of articles about 'how to put ML models in to production'.
- shajznnckfke 6y ago
  “Deploying a model” is sort of a nebulous concept. You probably have some kind of server, which loads a serialized model and runs data from requests or from batch files through the model to get predictions. Which part are we deploying? The service can be deployed like any other service. The model is probably a file. You deploy it by copying it to s3 I guess? You can go more in depth, and that’s what the article is about.
  - shoo 6y ago
    If you don't expect to need to tweak the model parameters very often, and it has a simple form (decision trees, linear regression) and forms a sub component of an existing application, another deployment option is just hard coding the model as a library or even an expression amidst the rest of the application code.
    - shajznnckfke 6y ago
      That’s a good option too. Sometimes the model consists of some training and inference code, which can be loaded as a library, plus a bunch of weights, which may be large enough to warrant a separate file. Either way, I think the deployment of the model isn’t really a hard problem. Validation of model quality, keeping track of what code was used, and what training data, making sure the updated data is where it’s supposed to be before training, tying all these parts together in some kind of comprehensive management system are all harder problems I think.
      - laichzeit0 6y ago
        There are some subtleties to deployment. If you trained a model with say sklearn version X and the component serving requests deserializes it but it’s using sklearn version Y then things can break badly.

dcl 6y ago

> To me, it talks about everything but actually deploying a machine learning model in production. This seems to be a common theme of a lot of articles about 'how to put ML models in to production'.

shajznnckfke 6y ago
“Deploying a model” is sort of a nebulous concept. You probably have some kind of server, which loads a serialized model and runs data from requests or from batch files through the model to get predictions. Which part are we deploying? The service can be deployed like any other service. The model is probably a file. You deploy it by copying it to s3 I guess? You can go more in depth, and that’s what the article is about.
- shoo 6y ago
  If you don't expect to need to tweak the model parameters very often, and it has a simple form (decision trees, linear regression) and forms a sub component of an existing application, another deployment option is just hard coding the model as a library or even an expression amidst the rest of the application code.
  - shajznnckfke 6y ago
    That’s a good option too. Sometimes the model consists of some training and inference code, which can be loaded as a library, plus a bunch of weights, which may be large enough to warrant a separate file. Either way, I think the deployment of the model isn’t really a hard problem. Validation of model quality, keeping track of what code was used, and what training data, making sure the updated data is where it’s supposed to be before training, tying all these parts together in some kind of comprehensive management system are all harder problems I think.
    - laichzeit0 6y ago
      There are some subtleties to deployment. If you trained a model with say sklearn version X and the component serving requests deserializes it but it’s using sklearn version Y then things can break badly.