4 ms·

Really? I was thinking you just set up a new Docker env for each python project, then you don't need to worry about conflicting with libs from other projects.

by amouat 13y ago

Really? I was thinking you just set up a new Docker env for each python project, then you don't need to worry about conflicting with libs from other projects. What am I missing?

dagw 13y ago
Docker only runs on a small subset of the platforms python runs on is the most obvious one. Can docker easily access the host filesystem or do you have to copy all your data into each docker env? Can I access my GPU for doing CUDA/OpenGL/OpenCL things from withing docker? Does docker play nicely with GUI apps? Calling into a running docker env from an external program (when you are using an app that has python as scripting or plug-in language) doesn't work as far as I know. The overhead of setting up, tearing down an switching between several Docker envs seems to higher than with tools like virtualenvwrapper or conda. Now there may be workarounds for these (and all the other) problems but on the surface they don't seem easier than virtualenv.
- amouat 13y ago
  Ok, thanks for the reply. Good questions. I was mainly thinking in the context of developing webapps. I'm interested in this enough that I intend to investigate and answer all your questions, perhaps in a blog post. I don't know enough right now to answer with certainty. However: - it will currently only work well if you are developing on Linux. MacOS isn't bad but requires vagrant/virtualbox. - I suspect switching between Docker envs may be faster, easier and cleaner than virtualenv which is why I suggested it. - There is definitely support for sharing data.
  - jamtur01 13y ago
    And there's boot2docker for OSX that's even more lightweight than Vagrant - https://github.com/steeve/boot2docker https://github.com/steeve/boot2docker.
- creack 13y ago
  Docker runs on any linux. There is a few dependencies but the list is reducing very fast. You don't even need to have AUFS as you can use devicemapper or even regular copy if you have neither. Yes, docker can easily access host's filesystem with volumes. Yes you can access you GPU. I am using docker for mining. Yes Docker plays nicely with GUI apps, there are plenty of GUI usecases with docker (docker desktop, firefox, etc..) You can do anything you want within a python app and docker. There is a docker-py library that gives access to all docker features. You can take a look at this: https://github.com/jpetazzo/stevedore https://github.com/jpetazzo/stevedore in order to easily switch envs.
  - dagw 13y ago
    Sounds like it's high time I take a closer look at docker
donaldstufft 13y ago
You're also forgetting that Python itself is a core part of most modern Linux distributions. This means that even for a minimal install of an OS there are often times packages already inside your site-packages that your operating system depends on for a minimal set of functionality. Docker won't isolate you from these. It's a special kind of pain when you require one version of something, and your OS itself requires another version.
- amouat 13y ago
  Yeah, that's a very good point. Thankfully I've never ran into this issue.
- markbnj 13y ago
  Interesting. I've only been working with Docker for a couple of weeks, so I probably have some misconceptions, but I thought that only the kernel was shared with the container host. Of course, you will be installing an OS into the container, and that will bring along its own dependencies. But if you run into this situation then there isn't anything to prevent installing virtualenv in the container too. From my perspective working in the context of server-side python deployment, Docker is looking pretty attractive as an alternative to virtualenv. It allows me to construct an image with the specific dependencies I need, and then run it anywhere.

by amouat 13y ago

Really? I was thinking you just set up a new Docker env for each python project, then you don't need to worry about conflicting with libs from other projects. What am I missing?

dagw 13y ago

Docker only runs on a small subset of the platforms python runs on is the most obvious one. Can docker easily access the host filesystem or do you have to copy all your data into each docker env? Can I access my GPU for doing CUDA/OpenGL/OpenCL things from withing docker? Does docker play nicely with GUI apps? Calling into a running docker env from an external program (when you are using an app that has python as scripting or plug-in language) doesn't work as far as I know. The overhead of setting up, tearing down an switching between several Docker envs seems to higher than with tools like virtualenvwrapper or conda. Now there may be workarounds for these (and all the other) problems but on the surface they don't seem easier than virtualenv.

amouat 13y ago
Ok, thanks for the reply. Good questions. I was mainly thinking in the context of developing webapps. I'm interested in this enough that I intend to investigate and answer all your questions, perhaps in a blog post. I don't know enough right now to answer with certainty. However: - it will currently only work well if you are developing on Linux. MacOS isn't bad but requires vagrant/virtualbox. - I suspect switching between Docker envs may be faster, easier and cleaner than virtualenv which is why I suggested it. - There is definitely support for sharing data.
- jamtur01 13y ago
  And there's boot2docker for OSX that's even more lightweight than Vagrant - https://github.com/steeve/boot2docker https://github.com/steeve/boot2docker.
creack 13y ago
Docker runs on any linux. There is a few dependencies but the list is reducing very fast. You don't even need to have AUFS as you can use devicemapper or even regular copy if you have neither. Yes, docker can easily access host's filesystem with volumes. Yes you can access you GPU. I am using docker for mining. Yes Docker plays nicely with GUI apps, there are plenty of GUI usecases with docker (docker desktop, firefox, etc..) You can do anything you want within a python app and docker. There is a docker-py library that gives access to all docker features. You can take a look at this: https://github.com/jpetazzo/stevedore https://github.com/jpetazzo/stevedore in order to easily switch envs.
- dagw 13y ago
  Sounds like it's high time I take a closer look at docker

donaldstufft 13y ago

You're also forgetting that Python itself is a core part of most modern Linux distributions. This means that even for a minimal install of an OS there are often times packages already inside your site-packages that your operating system depends on for a minimal set of functionality. Docker won't isolate you from these. It's a special kind of pain when you require one version of something, and your OS itself requires another version.

amouat 13y ago
Yeah, that's a very good point. Thankfully I've never ran into this issue.
markbnj 13y ago
Interesting. I've only been working with Docker for a couple of weeks, so I probably have some misconceptions, but I thought that only the kernel was shared with the container host. Of course, you will be installing an OS into the container, and that will bring along its own dependencies. But if you run into this situation then there isn't anything to prevent installing virtualenv in the container too. From my perspective working in the context of server-side python deployment, Docker is looking pretty attractive as an alternative to virtualenv. It allows me to construct an image with the specific dependencies I need, and then run it anywhere.