14 ms·

Docker for data scientists: Introduction and use cases

151 points by raab 9y ago

iamwil 9y ago
How many data scientists are interested in deploying their own models? And how many of you would learn docker in order to do so?
- raab 9y ago
  I would agree, most data scientists are not required to deploy their own models, probably only for those in startups or small companies. I also introduce how Docker can be used for development. I think it is a big enough trend in CS that any data scientist should know some basics, similar to data science from the command line.
  - iamwil 9y ago
    It seems like the baseline of DS skill sets is all over the place, depending on where you started off. I would have thought most DS wouldn't be interested in learning docker just to deploy. But then again, I could be wrong. What made you write the post? Was it because you saw lots of DS wanting to deploy, or something else?
- doug1001 9y ago
  i'm not a data scientist, but i have several on my team. In my experience,(as your question suggests) the interest is quite low, particularly given how much knowledge is required to do so that would otherwise be out of scope building the model and packaging it for deployment are two completely different sets of tasks, are they not? much more useful, again in my experience, is a tool like vagrant, which allows a production environment to be provisioned onto a VM; said VM is then run from the data scientist's desktop (eg, so they can do their work with the same versions of the same tools available in prod)
- minimaxir 9y ago
  There is a strong overlap between data science and devops, which is something not often discussed in all the data science/machine learning MOOCs.
  - claytonjy 9y ago
    I agree this is a big gap in the curricula; "engineering chops" (managing servers, environments, etc.) is what I've seen be the biggest force multiplier for data scientists (esp. at non-huge orgs), not how good at stats they are or how fancy a model they can build.
    - iamwil 9y ago
      Which orgs have you seen this as the biggest force multiplier? From the sibling comments in this thread, it seems like there are some agree, and others don't. I was curious if there's a segment that does, and who they might be.
      - claytonjy 9y ago
        Biased sample for sure, but me and my friends/colleagues in the midwest at companies with DS teams of <10 people, often <5. Companies/teams far too small to separate the builders from the shippers. I also practice data science from a more engineering angle than many which further colors my opinions. Building models is fun, but I also enjoy deployment and maintenance, so I don't _want_ to hand that part off.
- SatvikBeri 9y ago
  At my last job, my (Data Science) team all deployed production code including our models. We were using Scala, which made it fairly easy since we could essentially just build fat JARs and upload them. In contrast, I've found trying to do the same thing with Python nightmarish - simple things like having Pandas .20 on one machine and .21 on another will result in silent changes in how some metrics are calculated. Anaconda mostly, but not entirely, solves that problem with its superior environment management (significantly better than pip IME.) But given that it's imperfect, I'll probably look more into Docker soon.
  - iamwil 9y ago
    How did you manage the dependency versioning problem? Did you set up automated tests for that? Or did you just make sure everyone installed the same version with a requirements.txt file?
    - SatvikBeri 9y ago
      Anaconda has a better tool for cloning identical environments as long as both machines are on the same OS: https://conda.io/docs/user-guide/tasks/manage-environments.html#building-identical-conda-environments https://conda.io/docs/user-guide/tasks/manage-environments.h... In theory, pip lets you do this with pip freeze, but I found that pip often broke on some finicky libraries, while Anaconda failed much less frequently (though still has occasional issues.) The main drawback is the same OS requirement – this makes it harder to e.g. develop on a mac and then launch a pipeline on an Ubuntu machine in AWS.
      - pwang 9y ago
        What do you mean by "same OS requirement"? Can't you take your environment.yml from a Mac and recreate the environment on Linux?
        SatvikBeri 9y ago
        Anaconda has two environment-copying tools, one is environment.yml (which is analogous to pip's requirements.txt), and the other is the "explicit" spec file, which actually points to specific files for specific packages. I've found environment.yml to fail with finicky libraries, same as requirements.txt, while the explicit spec file almost always works.
- JPKab 9y ago
  Many data scientists want to operationalize their models. If you don't work at a hip startup with a killer dev ops team, then you are likely at a big enterprise shop where relying on anyone else to handle your infrastructure for you is a great way to have 90% of your work never see the light of day. I use Docker daily for my needs. It ensures that I can easily prototype a scalable set of services on my local machine, and then deploy it to a multi-node environment on AWS, Gcloud, or our on-prem infrastructure. Good luck developing a Redis-backed ML app without Docker when it comes time for deployment. You'll get it done, but it'll be a lot slower and more difficult than if you had used Docker or containers of some sort. Edit: Make sure to check out Yhat. https://www.yhat.com/ https://www.yhat.com/
  - iamwil 9y ago
    Huh, that's surprising. I would have thought that working at a big enterprise shop, you'd have enough dev ops engineers to help you deploy your model, and it's the small startups that can't afford to do that. I set up Kubernetes to deploy Docker containers, and it took a little while to wrap my head around it. What's the painful part of doing the deploy with Docker like you've been doing?
- vinn124 9y ago
  deployment is not the only reason. distributed ml (not deepmind scale but we're still talking about several instances, usually for something like hyperparameter tuning) is fairly common in practice. without docker, id waste most of my time provisioning and getting things to work - even if it's just for myself!
wyattjoh 9y ago
I'd think that Pachyderm [0] would be mentioned in the article, but it wasn't. It uses Docker under the hood for the data pipelines. I've always wanted an excuse to do some data science... [0]: https://www.pachyderm.io/ https://www.pachyderm.io/
- raab 9y ago
  Cool! I had never heard of that. Starred...
- haraball 9y ago
  Singularity [0] is a related alternative. [0]: http://singularity.lbl.gov/ http://singularity.lbl.gov/
- jdoliner 9y ago
  Thanks for the plug wyattjoh, I'm one of the founders of Pachyderm so I'd like to expand on what you said just a bit to clarify. Pachyderm does use Docker under the hood, but we don't obfuscate it away, so Data Scientists get the full power of Docker (and most of the power of Kubernetes) at their fingertips. This means you can easily grab prefabricated environments such as Jupyter or Tensorflow container images and deploy them directly. Docker is so good for packaging environments we didn't want to conceal it. On the other hand, we felt the data orchestration capabilities of Docker were pretty lacking for Data Science use cases, so that's where we've focused our energy. Volumes are a good basic tool for getting data to your code, but they're a pretty blunt instrument. How do you split data up to parallelize over it? How do you make sure you've got the right version of data? How do you schedule new computations when data becomes available? Those are some of the use cases we solve with our distributed file system PFS.
- claytonjy 9y ago
  +1 for pachyderm; my team and I are using it to build backtesting platforms for financial modeling, and it's been really nice to use. Also great in combination with Gitlab's built-in CI/CD stuff. Product aside, we've appreciated the team's availability on both Slack and Github too.
deleted 9y ago
[deleted]
ganeshkrishnan 9y ago
Pedantic: it's "whet your appetite" not "wet your appetite"
- raab 9y ago
  https://github.com/Raab70/Raab70.github.io/commit/d69addbf124cd35f804d3fe29795816a72472496 https://github.com/Raab70/Raab70.github.io/commit/d69addbf12...
rb808 9y ago
I spent a lot of time learning docker and k8s, thinking it would be really useful. Interest from the rest of the team - is nearly zero. Actual VMs work great, I'm thinking its not really necessary except for if you need scalable/serverless applications.
robszumski 9y ago
As someone dabbling in ML for the first time, setting up the correct Python env has been a nightmare just to run some sample code. I am familiar with Docker and would love if this got more mainstream adoption. Fighting with dependencies sucks when you aren’t intimately familiar with the ecosystem: python, node and other js, and even go.
- smnrchrds 9y ago
  I haven't worked in ML, but I've used Python extensively for scientific computing. Setting up the environment used to be a headache, but Anaconda solved that issue. Are ML tools not available in Anaconda?
  - robszumski 9y ago
    This is on my list of tools to investigate next. Glad I’m headed in the right direction.
    - xapata 9y ago
      I know you said "dabbling" but it seems like Anaconda is recommended by nearly every guide to ML with Python. What did you stumble across that recommended something else?
    - JPKab 9y ago
      Move it to the top of your list. Anaconda is absolutely where you should start if you are trying to mess with ML/Predictive.
  - jimwalsh 9y ago
    ML tools are available in Anaconda and is where we direct our data scientists until they are comfortable customizing further.
- mindhash 9y ago
  Does anyone see value in tool like michaelangelo from uber? What if a data scientist could deploy models right from browser?
bane 9y ago
Does anybody have some good patterns for bundling dependencies for analytics in Docker Containers, but then handling the execution of the analytics on Spark clusters? There seems to be various permutations of this notion, but I've heard that most of them have various issues or don't work quite as expected.
road2stat 9y ago
Personally, I don't think sealing dependencies to certain versions is the best idea, while Docker can be a very valuable addition to a data scientist's toolbox. Reproducibility, just like integration, deployment, and delivery, should be a continuous process. For data scientists who write workflows as R Markdown documents and want to containerize them, you might want to check out our R package liftr: https://liftr.me/ https://liftr.me/.
tbenst 9y ago
I would recommend looking at Singularity as well: http://singularity.lbl.gov/ http://singularity.lbl.gov/. Docker is not supported in the HPC world but Singularity is showing rapid adoption
- jghn 9y ago
  Also udocker for similar reasons
- thebeardedone 9y ago
  It really depends on what you are trying to do with it, for example we have around 15 different integration test configurations that run every night for which VMs may better suited as we want to test installation and deployments automatically for 3 distributions (6 different versions ex. Ubuntu 14.04, ubuntu 16.04, centos 6, centos 7 etc..) and the last 2 windows server. The good thing is that they reproduce the customer environment. But they have large down sides as well which slow us down. They are a pain to maintain as they are somewhat undocumented (you make a poc for 1 and management always wants more without improvments), a lot of edge cases cause issues which are tough to reproduce (locally sometimes impossible and waste a lot of time) and it takes them a while to start, run, etc. This is not too tragic for nightly tests as we get the results in the morning but for tests which are started every hour, you do not want to wait that long to verify your changes work/didnt break anything. You can do these in stages, where you create different images based on the result of a previous job (run basic tests that cover base functionality that should always work, then run more in depth tests, then run performance tests at the end to ensure no significant degradation was introduced, etc..) and send out notifications asap in case of failure. The Dockerfile is essentially the documentation as you can see what is installed/configured. You can run everything locally just as it would in a k8 env. which for some reason every one always struggles with. I am sure there are also edge cases with Docker that are a pain as well but the other selling points show it may be the right direction. You just havevto find use cases and evaluate them.
  - mbreese 9y ago
    I'm not sure what that has to do with Singularity? I know that the HPC clusters I've used in the past few years have all supported Singularity, but none have supported Docker (aside from our small lab cluster). Many HPC admins are (understandably) hesitant to allow non-admins access to start Docker containers (requiring root), but Singularity has no such user permission issues -- and it's faster than initializing a full VM to run a job. I don't expect that to change so long as starting a container requires root-effective permissions. I suspect that many data scientists will be in a similar situation w.r.t HPC clusters (except for those that are using custom clouds like Seven Bridges).
cle 9y ago
How does this compare with the development and deployment functionality in e.g. AWS SageMaker?
- raab 9y ago
  SageMaker has three parts. Hosted notebooks, similar to case #2. API access to ML algorithms which are "optimized" and deployment by providing API access to trained models.
vonnik 9y ago
[Disclosure: Skymind co-founder here.] The Skymind Intelligence Layer helps data scientists operationalize their models on-prem and in the public cloud, and uses Docker to do that. https://skymind.readme.io/v1.0.1/docs/quickstart https://skymind.readme.io/v1.0.1/docs/quickstart https://skymind.readme.io/v1.0.1/docs/docker-image https://skymind.readme.io/v1.0.1/docs/docker-image

151 points by raab 9y ago

How many data scientists are interested in deploying their own models? And how many of you would learn docker in order to do so?

raab 9y ago
I would agree, most data scientists are not required to deploy their own models, probably only for those in startups or small companies. I also introduce how Docker can be used for development. I think it is a big enough trend in CS that any data scientist should know some basics, similar to data science from the command line.
- iamwil 9y ago
  It seems like the baseline of DS skill sets is all over the place, depending on where you started off. I would have thought most DS wouldn't be interested in learning docker just to deploy. But then again, I could be wrong. What made you write the post? Was it because you saw lots of DS wanting to deploy, or something else?
doug1001 9y ago
i'm not a data scientist, but i have several on my team. In my experience,(as your question suggests) the interest is quite low, particularly given how much knowledge is required to do so that would otherwise be out of scope building the model and packaging it for deployment are two completely different sets of tasks, are they not? much more useful, again in my experience, is a tool like vagrant, which allows a production environment to be provisioned onto a VM; said VM is then run from the data scientist's desktop (eg, so they can do their work with the same versions of the same tools available in prod)
minimaxir 9y ago
There is a strong overlap between data science and devops, which is something not often discussed in all the data science/machine learning MOOCs.
- claytonjy 9y ago
  I agree this is a big gap in the curricula; "engineering chops" (managing servers, environments, etc.) is what I've seen be the biggest force multiplier for data scientists (esp. at non-huge orgs), not how good at stats they are or how fancy a model they can build.
  - iamwil 9y ago
    Which orgs have you seen this as the biggest force multiplier? From the sibling comments in this thread, it seems like there are some agree, and others don't. I was curious if there's a segment that does, and who they might be.
    - claytonjy 9y ago
      Biased sample for sure, but me and my friends/colleagues in the midwest at companies with DS teams of <10 people, often <5. Companies/teams far too small to separate the builders from the shippers. I also practice data science from a more engineering angle than many which further colors my opinions. Building models is fun, but I also enjoy deployment and maintenance, so I don't _want_ to hand that part off.
SatvikBeri 9y ago
At my last job, my (Data Science) team all deployed production code including our models. We were using Scala, which made it fairly easy since we could essentially just build fat JARs and upload them. In contrast, I've found trying to do the same thing with Python nightmarish - simple things like having Pandas .20 on one machine and .21 on another will result in silent changes in how some metrics are calculated. Anaconda mostly, but not entirely, solves that problem with its superior environment management (significantly better than pip IME.) But given that it's imperfect, I'll probably look more into Docker soon.
- iamwil 9y ago
  How did you manage the dependency versioning problem? Did you set up automated tests for that? Or did you just make sure everyone installed the same version with a requirements.txt file?
  - SatvikBeri 9y ago
    Anaconda has a better tool for cloning identical environments as long as both machines are on the same OS: https://conda.io/docs/user-guide/tasks/manage-environments.html#building-identical-conda-environments https://conda.io/docs/user-guide/tasks/manage-environments.h... In theory, pip lets you do this with pip freeze, but I found that pip often broke on some finicky libraries, while Anaconda failed much less frequently (though still has occasional issues.) The main drawback is the same OS requirement – this makes it harder to e.g. develop on a mac and then launch a pipeline on an Ubuntu machine in AWS.
    - pwang 9y ago
      What do you mean by "same OS requirement"? Can't you take your environment.yml from a Mac and recreate the environment on Linux?
      - SatvikBeri 9y ago
        Anaconda has two environment-copying tools, one is environment.yml (which is analogous to pip's requirements.txt), and the other is the "explicit" spec file, which actually points to specific files for specific packages. I've found environment.yml to fail with finicky libraries, same as requirements.txt, while the explicit spec file almost always works.
JPKab 9y ago
Many data scientists want to operationalize their models. If you don't work at a hip startup with a killer dev ops team, then you are likely at a big enterprise shop where relying on anyone else to handle your infrastructure for you is a great way to have 90% of your work never see the light of day. I use Docker daily for my needs. It ensures that I can easily prototype a scalable set of services on my local machine, and then deploy it to a multi-node environment on AWS, Gcloud, or our on-prem infrastructure. Good luck developing a Redis-backed ML app without Docker when it comes time for deployment. You'll get it done, but it'll be a lot slower and more difficult than if you had used Docker or containers of some sort. Edit: Make sure to check out Yhat. https://www.yhat.com/ https://www.yhat.com/
- iamwil 9y ago
  Huh, that's surprising. I would have thought that working at a big enterprise shop, you'd have enough dev ops engineers to help you deploy your model, and it's the small startups that can't afford to do that. I set up Kubernetes to deploy Docker containers, and it took a little while to wrap my head around it. What's the painful part of doing the deploy with Docker like you've been doing?
vinn124 9y ago
deployment is not the only reason. distributed ml (not deepmind scale but we're still talking about several instances, usually for something like hyperparameter tuning) is fairly common in practice. without docker, id waste most of my time provisioning and getting things to work - even if it's just for myself!

wyattjoh 9y ago

I'd think that Pachyderm [0] would be mentioned in the article, but it wasn't. It uses Docker under the hood for the data pipelines. I've always wanted an excuse to do some data science... [0]: https://www.pachyderm.io/ https://www.pachyderm.io/

raab 9y ago
Cool! I had never heard of that. Starred...
haraball 9y ago
Singularity [0] is a related alternative. [0]: http://singularity.lbl.gov/ http://singularity.lbl.gov/
jdoliner 9y ago
Thanks for the plug wyattjoh, I'm one of the founders of Pachyderm so I'd like to expand on what you said just a bit to clarify. Pachyderm does use Docker under the hood, but we don't obfuscate it away, so Data Scientists get the full power of Docker (and most of the power of Kubernetes) at their fingertips. This means you can easily grab prefabricated environments such as Jupyter or Tensorflow container images and deploy them directly. Docker is so good for packaging environments we didn't want to conceal it. On the other hand, we felt the data orchestration capabilities of Docker were pretty lacking for Data Science use cases, so that's where we've focused our energy. Volumes are a good basic tool for getting data to your code, but they're a pretty blunt instrument. How do you split data up to parallelize over it? How do you make sure you've got the right version of data? How do you schedule new computations when data becomes available? Those are some of the use cases we solve with our distributed file system PFS.
claytonjy 9y ago
+1 for pachyderm; my team and I are using it to build backtesting platforms for financial modeling, and it's been really nice to use. Also great in combination with Gitlab's built-in CI/CD stuff. Product aside, we've appreciated the team's availability on both Slack and Github too.

deleted 9y ago

[deleted]

ganeshkrishnan 9y ago

Pedantic: it's "whet your appetite" not "wet your appetite"

raab 9y ago
https://github.com/Raab70/Raab70.github.io/commit/d69addbf124cd35f804d3fe29795816a72472496 https://github.com/Raab70/Raab70.github.io/commit/d69addbf12...

rb808 9y ago

I spent a lot of time learning docker and k8s, thinking it would be really useful. Interest from the rest of the team - is nearly zero. Actual VMs work great, I'm thinking its not really necessary except for if you need scalable/serverless applications.

robszumski 9y ago

As someone dabbling in ML for the first time, setting up the correct Python env has been a nightmare just to run some sample code. I am familiar with Docker and would love if this got more mainstream adoption. Fighting with dependencies sucks when you aren’t intimately familiar with the ecosystem: python, node and other js, and even go.

smnrchrds 9y ago
I haven't worked in ML, but I've used Python extensively for scientific computing. Setting up the environment used to be a headache, but Anaconda solved that issue. Are ML tools not available in Anaconda?
- robszumski 9y ago
  This is on my list of tools to investigate next. Glad I’m headed in the right direction.
  - xapata 9y ago
    I know you said "dabbling" but it seems like Anaconda is recommended by nearly every guide to ML with Python. What did you stumble across that recommended something else?
  - JPKab 9y ago
    Move it to the top of your list. Anaconda is absolutely where you should start if you are trying to mess with ML/Predictive.
- jimwalsh 9y ago
  ML tools are available in Anaconda and is where we direct our data scientists until they are comfortable customizing further.
mindhash 9y ago
Does anyone see value in tool like michaelangelo from uber? What if a data scientist could deploy models right from browser?

bane 9y ago

Does anybody have some good patterns for bundling dependencies for analytics in Docker Containers, but then handling the execution of the analytics on Spark clusters? There seems to be various permutations of this notion, but I've heard that most of them have various issues or don't work quite as expected.

road2stat 9y ago

Personally, I don't think sealing dependencies to certain versions is the best idea, while Docker can be a very valuable addition to a data scientist's toolbox. Reproducibility, just like integration, deployment, and delivery, should be a continuous process. For data scientists who write workflows as R Markdown documents and want to containerize them, you might want to check out our R package liftr: https://liftr.me/ https://liftr.me/.

tbenst 9y ago

I would recommend looking at Singularity as well: http://singularity.lbl.gov/ http://singularity.lbl.gov/. Docker is not supported in the HPC world but Singularity is showing rapid adoption

jghn 9y ago
Also udocker for similar reasons
thebeardedone 9y ago
It really depends on what you are trying to do with it, for example we have around 15 different integration test configurations that run every night for which VMs may better suited as we want to test installation and deployments automatically for 3 distributions (6 different versions ex. Ubuntu 14.04, ubuntu 16.04, centos 6, centos 7 etc..) and the last 2 windows server. The good thing is that they reproduce the customer environment. But they have large down sides as well which slow us down. They are a pain to maintain as they are somewhat undocumented (you make a poc for 1 and management always wants more without improvments), a lot of edge cases cause issues which are tough to reproduce (locally sometimes impossible and waste a lot of time) and it takes them a while to start, run, etc. This is not too tragic for nightly tests as we get the results in the morning but for tests which are started every hour, you do not want to wait that long to verify your changes work/didnt break anything. You can do these in stages, where you create different images based on the result of a previous job (run basic tests that cover base functionality that should always work, then run more in depth tests, then run performance tests at the end to ensure no significant degradation was introduced, etc..) and send out notifications asap in case of failure. The Dockerfile is essentially the documentation as you can see what is installed/configured. You can run everything locally just as it would in a k8 env. which for some reason every one always struggles with. I am sure there are also edge cases with Docker that are a pain as well but the other selling points show it may be the right direction. You just havevto find use cases and evaluate them.
- mbreese 9y ago
  I'm not sure what that has to do with Singularity? I know that the HPC clusters I've used in the past few years have all supported Singularity, but none have supported Docker (aside from our small lab cluster). Many HPC admins are (understandably) hesitant to allow non-admins access to start Docker containers (requiring root), but Singularity has no such user permission issues -- and it's faster than initializing a full VM to run a job. I don't expect that to change so long as starting a container requires root-effective permissions. I suspect that many data scientists will be in a similar situation w.r.t HPC clusters (except for those that are using custom clouds like Seven Bridges).

cle 9y ago

How does this compare with the development and deployment functionality in e.g. AWS SageMaker?

raab 9y ago
SageMaker has three parts. Hosted notebooks, similar to case #2. API access to ML algorithms which are "optimized" and deployment by providing API access to trained models.

vonnik 9y ago

[Disclosure: Skymind co-founder here.] The Skymind Intelligence Layer helps data scientists operationalize their models on-prem and in the public cloud, and uses Docker to do that. https://skymind.readme.io/v1.0.1/docs/quickstart https://skymind.readme.io/v1.0.1/docs/quickstart https://skymind.readme.io/v1.0.1/docs/docker-image https://skymind.readme.io/v1.0.1/docs/docker-image