4 ms·

Nvidia-Docker: Build and Run Docker Containers Leveraging Nvidia GPUs

63 points by charlieegan3 10y ago

moondev 10y ago
This looks very interesting. What advantages would there be to running containers on the GPU?
- fscherer 10y ago
  using containers to easily deploy machine learning experiments - thats at least how we use it
  - moondev 10y ago
    That's awesome!
  - flx42_ 10y ago
    Yes, running containerized machine learning workflows is our primary use case of nvidia-docker internally. That's why we provide pre-built images for cuDNN and DIGITS on the DockerHub. Our base cuDNN image is now used by TensorFlow, Caffe, CNTK, Theano, to cite a few.
WhatsName 10y ago
This is brilliant, wished I had this when I was implementing a large scale cloud rendering project. Having a standardised and streamlined way of deploying the application code with CUDA toolkit would have saved me a lot of troubleshooting.
Patrick_Devine 10y ago
I think leveraging GPUs inside of Docker containers is an absolutely fantastic idea, but forking/wrapping Docker to make it happen is just silly. Were there no attempts to talk to the core team to make this happen? (Disclaimer: I work at Docker, but not on the core team)
- machbio 10y ago
  https://github.com/NVIDIA/nvidia-docker/wiki/Why%20NVIDIA%20Docker#yet-another-docker-tool https://github.com/NVIDIA/nvidia-docker/wiki/Why%20NVIDIA%20...
  - andrewstuart2 10y ago
    > However, this solution was brittle: the version of the host driver had to exactly match driver version installed in the container. I'm curious why volume mounting is not an option here. Seems like you'd just ship driverless containers, forcing the user to mount the host drivers before allowing the container to run.
    - flx42_ 10y ago
      That's exactly what we do, the image is indeed driverless and we mount the host driver files as a volume (provided by our volume plugin) when the container is launched. This way, you can launch a CUDA 7.5 container on any machine with driver version 352+ We provide more details here: https://github.com/NVIDIA/nvidia-docker/wiki/NVIDIA-driver https://github.com/NVIDIA/nvidia-docker/wiki/NVIDIA-driver Hope that helps!
- flx42_ 10y ago
  We are only wrapping the Docker CLI, not forking the full code (that would be insane). The wrapper is provided for convenience since it should be enough for most users. If you know what you're doing, you don't need to rely on the wrapper, we explain how you can do that on our wiki: https://github.com/NVIDIA/nvidia-docker/wiki/Internals https://github.com/NVIDIA/nvidia-docker/wiki/Internals Yes, we have been discussing with a few people at Docker, I hope attending DockerCon and discussing with the team will help us move forward.
  - shykes 10y ago
    This is a fantastic use case, we (Docker) would love to help in any way we can to make this usable by all Docker users. Let us know if there's anything more we can to help.
lmeyerov 10y ago
We use this at Graphistry for wrapping our analytics & visuals for cloud & on-premises, super appreciative :)
joshuak 10y ago
This seems like unnecessary additional structure. I was one of the first to build dockerized apps with nvidia drivers[1] and it's as easy as building the drivers and running modprobe in a privileged (driver only) container. All other containers on the platform are then able to run GPU workloads without modification. We haven't found this to be particularly brittle or problematic in any way. It just works(TM). it feels like reaching deeper into the control stack by 'wrapping' Docker CLI or other actions is likely to be an untenable workflow for us. We use a huge amount of GPU. We do it on CoreOS/k8s et al. We're exploring rkt. Doesn't nvidia's approach interfere with this? However, I am always looking for better approaches to help reach as far into the future with our platform as possible. So show me the magic, and I'm yours. 1: https://github.com/Avalanche-io/coreos-nvidia https://github.com/Avalanche-io/coreos-nvidia (Old public repo, Should I push our latest? I don't think anyone is using this so I haven't.)
- exxo_ 10y ago
  I think you are missing the point, it has nothing to do with installing the NVIDIA drivers through Docker. What you are showing[1] is how to install NVIDIA drivers on CoreOS the hackish way (not persistent, no driver libs, no DKMS, no UVM, no KMS...) Regarding rkt, it's not supported at the moment but a similar approach could be taken. As for the Docker CLI wrapper, you can avoid it if you really need to.
  - joshuak 10y ago
    While my code here is definitely hackish, I can't argue with that, I have to say I'm hard pressed to see how running a container to activate a driver is hackish when the comparison at hand is modifying the Docker CLI and requiring a Docker plugin. I run the driver container at startup, and never shut it down. How is this not persistent? DKMS and other build/deployment choices are not obviated by my approach, so I'm not sure that's relevant. Looking more deeply at the "Why NVIDIA Docker" in the repo wiki doesn't provide any enlightenment either. In fact it doesn't really explain why docker itself must be modified. The only explanation really is lack of container portability, but driver containers are portable within the scope of a given kernel version. Certainly modified docker cli and plugin requirements are much less portable. It seems to me like someone at nVidia simply didn't realize that they could run a container in privileged mode and effectively install the driver system wide for all containers.
fapjacks 10y ago
I've been using this for some time specifically for my ML projects.

63 points by charlieegan3 10y ago

moondev 10y ago

This looks very interesting. What advantages would there be to running containers on the GPU?

fscherer 10y ago
using containers to easily deploy machine learning experiments - thats at least how we use it
- moondev 10y ago
  That's awesome!
- flx42_ 10y ago
  Yes, running containerized machine learning workflows is our primary use case of nvidia-docker internally. That's why we provide pre-built images for cuDNN and DIGITS on the DockerHub. Our base cuDNN image is now used by TensorFlow, Caffe, CNTK, Theano, to cite a few.

WhatsName 10y ago

This is brilliant, wished I had this when I was implementing a large scale cloud rendering project. Having a standardised and streamlined way of deploying the application code with CUDA toolkit would have saved me a lot of troubleshooting.

Patrick_Devine 10y ago

I think leveraging GPUs inside of Docker containers is an absolutely fantastic idea, but forking/wrapping Docker to make it happen is just silly. Were there no attempts to talk to the core team to make this happen? (Disclaimer: I work at Docker, but not on the core team)

machbio 10y ago
https://github.com/NVIDIA/nvidia-docker/wiki/Why%20NVIDIA%20Docker#yet-another-docker-tool https://github.com/NVIDIA/nvidia-docker/wiki/Why%20NVIDIA%20...
- andrewstuart2 10y ago
  > However, this solution was brittle: the version of the host driver had to exactly match driver version installed in the container. I'm curious why volume mounting is not an option here. Seems like you'd just ship driverless containers, forcing the user to mount the host drivers before allowing the container to run.
  - flx42_ 10y ago
    That's exactly what we do, the image is indeed driverless and we mount the host driver files as a volume (provided by our volume plugin) when the container is launched. This way, you can launch a CUDA 7.5 container on any machine with driver version 352+ We provide more details here: https://github.com/NVIDIA/nvidia-docker/wiki/NVIDIA-driver https://github.com/NVIDIA/nvidia-docker/wiki/NVIDIA-driver Hope that helps!
flx42_ 10y ago
We are only wrapping the Docker CLI, not forking the full code (that would be insane). The wrapper is provided for convenience since it should be enough for most users. If you know what you're doing, you don't need to rely on the wrapper, we explain how you can do that on our wiki: https://github.com/NVIDIA/nvidia-docker/wiki/Internals https://github.com/NVIDIA/nvidia-docker/wiki/Internals Yes, we have been discussing with a few people at Docker, I hope attending DockerCon and discussing with the team will help us move forward.
- shykes 10y ago
  This is a fantastic use case, we (Docker) would love to help in any way we can to make this usable by all Docker users. Let us know if there's anything more we can to help.

lmeyerov 10y ago

We use this at Graphistry for wrapping our analytics & visuals for cloud & on-premises, super appreciative :)

joshuak 10y ago

This seems like unnecessary additional structure. I was one of the first to build dockerized apps with nvidia drivers[1] and it's as easy as building the drivers and running modprobe in a privileged (driver only) container. All other containers on the platform are then able to run GPU workloads without modification. We haven't found this to be particularly brittle or problematic in any way. It just works(TM). it feels like reaching deeper into the control stack by 'wrapping' Docker CLI or other actions is likely to be an untenable workflow for us. We use a huge amount of GPU. We do it on CoreOS/k8s et al. We're exploring rkt. Doesn't nvidia's approach interfere with this? However, I am always looking for better approaches to help reach as far into the future with our platform as possible. So show me the magic, and I'm yours. 1: https://github.com/Avalanche-io/coreos-nvidia https://github.com/Avalanche-io/coreos-nvidia (Old public repo, Should I push our latest? I don't think anyone is using this so I haven't.)

exxo_ 10y ago
I think you are missing the point, it has nothing to do with installing the NVIDIA drivers through Docker. What you are showing[1] is how to install NVIDIA drivers on CoreOS the hackish way (not persistent, no driver libs, no DKMS, no UVM, no KMS...) Regarding rkt, it's not supported at the moment but a similar approach could be taken. As for the Docker CLI wrapper, you can avoid it if you really need to.
- joshuak 10y ago
  While my code here is definitely hackish, I can't argue with that, I have to say I'm hard pressed to see how running a container to activate a driver is hackish when the comparison at hand is modifying the Docker CLI and requiring a Docker plugin. I run the driver container at startup, and never shut it down. How is this not persistent? DKMS and other build/deployment choices are not obviated by my approach, so I'm not sure that's relevant. Looking more deeply at the "Why NVIDIA Docker" in the repo wiki doesn't provide any enlightenment either. In fact it doesn't really explain why docker itself must be modified. The only explanation really is lack of container portability, but driver containers are portable within the scope of a given kernel version. Certainly modified docker cli and plugin requirements are much less portable. It seems to me like someone at nVidia simply didn't realize that they could run a container in privileged mode and effectively install the driver system wide for all containers.

fapjacks 10y ago

I've been using this for some time specifically for my ML projects.