7 ms·
Nvidia-Docker – Build and Run Docker Containers Leveraging Nvidia GPUs
- stephengillie 9y agoIs there a processing-specific user case for this - are we going to see mining containers or game client containers?
- vqc 9y agoWould this be useful for people who wanted to leverage GPUs for deep learning, but didn't have the wherewithal or the willingness to set up all the dependencies on the host machine?
- fleetwood 9y agoYes, this is a great use case. Search around, there are a few folks putting together dockers to get you out of dl dependency hell
- tomrijntjes 9y agoExactly what we are using this for. However, I found there was little performance benefit versus running on CPU. Can't figure out why.
- minimaxir 9y agoYes. I have made a Docker container containing Keras and the better CNTK backend (+ Jupyter) and it works great: https://github.com/minimaxir/keras-cntk-docker https://github.com/minimaxir/keras-cntk-docker There's another unexpected bonus to nvidia-docker; CNTK leaks memory if you abort training prematurely, causing the GPU to go OOM. But since the memory is owned by the nvidia-docker process, killing-and-restarting the container fixes the issue without having to restart the entire system.
- tiangolo 9y agoIt is used by this kind of popular pre-built Docker image for Deep Learning: https://github.com/floydhub/dl-docker https://github.com/floydhub/dl-docker They actually created this service using that (a Deep Learning "as a service"): https://www.floydhub.com/ https://www.floydhub.com/
- narenst 9y agoAt FloydHub, we use nvidia-docker in production for running DL jobs. It has been very solid for the past 6 months or so. We have also built a collection of open source DL docker images for various frameworks and we actively maintain them. [1]: https://hub.docker.com/r/floydhub/ https://hub.docker.com/r/floydhub/ [2]: https://github.com/floydhub/dockerfiles https://github.com/floydhub/dockerfiles
- sargun 9y agoWe use this in production every day. We've hit a couple bugs, but it's a surprisingly stable piece of software. We hit an issue with user namespaces, and it not correctly chowning things when "exporting" them into Docker containers. I honestly forget how we solved this, but I believe it was just an upgrade. We hit an issue with the first run of a Docker container on a system being extremely slow, to the point where downstream systems would give up, and retry. The issue was fixed by starting a container that leverages the driver at boot time, "warming" it up.
- etaioinshrdlu 9y agoIs it possible that the slowness is caused by disk speed of loading a large file from disk? I've observed the same thing but it seemed to be IO related, not CPU or GPU related.
- deleted 9y ago[deleted]
- tlb 9y agoIt's mainly the speed of creating a huge number of small files on disk. When unpacking a docker image into an AUFS file system, it has to create every file in the shadow system. A robust Ubuntu install has around 100000 files. On AWS with default EBS storage, this can take a few minutes. At OpenAI we pre-built AMI instances with file systems pre-unpacked.
- sipherhex 9y agoGiven the description of your workaround, you may be running into an issue of long GPU driver load times for the nvidia driver. If that is the root cause the recommended solution for this is nvidia-persistenced, though it seems you may have rolled your own equivalent. https://github.com/NVIDIA/nvidia-persistenced https://github.com/NVIDIA/nvidia-persistenced Docs here http://docs.nvidia.com/deploy/driver-persistence/index.html http://docs.nvidia.com/deploy/driver-persistence/index.html
- unixhero 9y agoCan I run regular web apps with this on NVIDIA gpu's?
- mastax 9y agoNo.
- cosinetau 9y agoPorque no?
- neuronexmachina 9y agoI mean, technically I guess you could run it, it just wouldn't make use of the GPU.
- deleted 9y ago[deleted]
- flx42_ 9y agoIt allows you to run GPU-accelerated applications (like machine learning, HPC, video/image processing...) inside a Docker container.
- robeastham 9y agoYes. Ignore the naysayers :-) Check out VirtuaGL and TurboVNC. It is possible to run these two things inside a docker container, install a DE and accelerate a GUI app. I've been doing it for quite a while. Also for all those checking the OPs original lib out. It does work and it is great and also probably the most mature option, but my understanding is that future focus for GPU inside containers is with this newer lib, also looked after by Nvidia: https://github.com/NVIDIA/libnvidia-container https://github.com/NVIDIA/libnvidia-container I'm fairly familiar with this stuff, though not an expert, for my startup http://realityzero.one http://realityzero.one. But, I've not been paying that much attention to this area of late, so if I've got this wrong then please feel free to correct me. Perhaps they're for different things. I know there's been some discussion from the k8n crowd about the newer lib in k8n's github issues.
- ThePhysicist 9y agoJust for information, it's also possible to use NVIDIA GPUs inside Docker containers without using `nvidia-docker`: https://stackoverflow.com/questions/25185405/using-gpu-from-a-docker-container https://stackoverflow.com/questions/25185405/using-gpu-from-... In a nutshell, you can give the container access to the NVIDIA device file via the `--device` flag, so all you need is a container with the NVIDIA drivers. The added benefit of this is that you can use different versions of the drivers side-by-side (in my understanding). I thought this might be relevant as some people might not want to use the `docker-nvidia` CLI to run containers (I'm not sure how you would use this via the Docker API for example).
- flx42_ 9y agoWe document how this on our wiki: https://github.com/NVIDIA/nvidia-docker/wiki/Internals https://github.com/NVIDIA/nvidia-docker/wiki/Internals > The added benefit of this is that you can use different versions of the drivers side-by-side (in my understanding). No, you can only have one driver version, the one that correspond to the loaded kernel modules. Installing the driver inside a Docker image makes it non-portable.
- ThePhysicist 9y agoAh thanks for the clarification, I was not aware of this!
- arnon 9y agoIt's also possible to set up CUDA in LXC https://sqream.com/cuda-in-lxc-containers/ https://sqream.com/cuda-in-lxc-containers/
- zitterbewegung 9y agoI wish they would add support for NVIDIA-docker to run on Windows but I think that is a tall order.
- lmeyerov 9y agoOne of our engineers recently shared an intro-level talk at the Docker meetup about a ~year of (happy) experiences with this: https://www.meetup.com/Docker-Santa-Clara/events/240641246/ https://www.meetup.com/Docker-Santa-Clara/events/240641246/ . It should be recorded somewhere, and may be helpful if you're considering production use, not just personal DL stuff.
- krosaen 9y agoAs and example, we used (nvidia-)docker images to make our object detection results reproducible: https://github.com/umautobots/driving-in-the-matrix https://github.com/umautobots/driving-in-the-matrix docker has been great for this in the lab: only one person now goes through the pain to get latest framework + hack + model tweak work and then the rest can reuse.
- klueska 9y agoIf you're using DC/OS you can also run docker containers that leverage GPUs. We worked with Nvidia to mimic the same functionality provided by nvidia-docker. Here's a talk I gave at this years Nvidia GTC conference explaining how it works: https://drive.google.com/file/d/0B7vZqCY-AJrpMEYyVmZkVGlyOEE/view https://drive.google.com/file/d/0B7vZqCY-AJrpMEYyVmZkVGlyOEE... https://docs.google.com/presentation/d/1q6brajoQkFZVtSocMPGgbmqTLAcrZetcMrdve-WD6x4 https://docs.google.com/presentation/d/1q6brajoQkFZVtSocMPGg...
- etaioinshrdlu 9y agoA collection of deep learning nvidia-docker images to get you started: https://hub.docker.com/r/deepaiorg/ https://hub.docker.com/r/deepaiorg/ Floydhub also curates most frameworks as nvidia-docker images: https://hub.docker.com/r/floydhub/ https://hub.docker.com/r/floydhub/
- aub3bhat 9y agoI use NVidia-Docker extensively in my Open Source project Deep Video Analytics [1] when combined with TensorFlow (which allows explicit GPU memory allocation) its unbeatable in running multiple inference models on a single GPU in a reliable manner. Combining this setup with docker volumes on AWS EFS allows simple multi machine deployments. [1] https://github.com/AKSHAYUBHAT/DeepVideoAnalytics https://github.com/AKSHAYUBHAT/DeepVideoAnalytics
- sandGorgon 9y agothis is pretty cool! why do you use multiple packages like torch and tensorflow ?
- aub3bhat 9y agoCertain algorithms/models are implemented in PyTorch or Caffe and typically it's huge amount of work to convert them to TensorFlow while ensuring correctness / Parity. Also I personally like design of PyTorch.
- sandGorgon 9y agocoming from the facebook/reactjs weaponized patent grant problem.. caffe also has the same revocable patent grant. Tensorflow is apache licensed. I think in general, the perception is that it is far safer to stay away from caffe.
- aub3bhat 9y agoCaffe 1 is developed by Berkeley and I think Apache or BSD licensed. The Patents.txt issue occurs with Caffe 2 which is developed by Facebook.
- random023987 9y agoIsn't AMD leaving a lot of potential revenue on the table without a comparable amdgpu-docker? If I'm building an ML cluster, I'm going to go with the vendor that's easiest to containerize and deploy, and right now it looks like nvidia has a commanding advantage in software. Why doesn't AMD throw a few hundred thousand dollars at some developers to get containerization parity?