8 ms·

Show HN: Develop your Python applications easily in clean Docker environments

119 points by macostea 10y ago

brudgers 10y ago
If it meets the guidelines, this might make a good 'Show HN'. Show HN guidelines: https://news.ycombinator.com/showhn.html https://news.ycombinator.com/showhn.html
- macostea 10y ago
  Thanks for the suggestion! I'll try it out
rockostrich 10y ago
Pretty simple to set up and use, but I'm not sure if it's better than learning `docker` and `docker-compose`. Also, the default `python <file>.py` doesn't remove the container after the file is done running and the image is tagged as `cage/<name of project>` instead of allowing the user to tag it with their own name. Both of these are easy to fix with `docker` commands, but if this is meant to be a library that helps you avoid `docker` then it's not doing its job.
- gshulegaard 10y ago
  I was going to say something similar. With venv moved into the stdlib in Python3 and ongoing development and improvements to the `docker-compose` workflow I am not sure where a tool like this fits. * venv is lighter and more or less fairly standardized * `docker-compose` is more general and language agnostic So something like `cage` is somewhere between...it's neat, no doubt, but not sure where I would use it personally.
- macostea 10y ago
  Thanks for the feedback! I'll make sure to add the container removal and naming as new features. You can indeed run all the commands easily just with Docker. Cage aims to port all the functionalities from virtualenv to use Docker. After this is achieved I can start working on extending those functionalities.
rch 10y ago
> The current version of Cage only supports dependencies written in a requirements file It might be worth looking at pipfile. Great project though!
crazyhatfish 10y ago
Although I'm not sure I would use this, this is the kind of stuff I've been waiting for. Where nice clean containers are used as part of an active dev cycle rather than just the production push. The ultimate dream of the end user using a desktop comprising only of containers seems to be still a bit far off yet...
- bschwindHN 10y ago
  About a year ago I switched our dev environments from Vagrant to docker-compose environments with a "helper" shell script to run the most common docker-compose operations you'd do in a day. It starts up so much faster than Vagrant, it's easy to add new services, and it's easy to add and remove test data. Here's an example of the kind of setup I have: https://github.com/bschwind/api-starter-kit/blob/master/helper.sh https://github.com/bschwind/api-starter-kit/blob/master/help... What's funny is I don't currently use docker in production, but I love it for dev environments.
0wl3x 10y ago
Could someone please explain why this might be useful? It seems like something that a crypto developer might like but if I'm developing something innocuous why would it matter if it runs inside a container?
- apetresc 10y ago
  Why would a crypto developer be especially interested in this?
  - _joel 10y ago
    I have feeling the OP is thinking of air-gapping or clean room builds, but I could be wrong.
    - 0wl3x 10y ago
      yeah that's exactly the kind of use case I was thinking of.
miloshadzic 10y ago
Docker runs pretty slow on macOS so you'll be paying a pretty high price for cleanliness.
alrs 10y ago
Mac-first development strikes again: pycage requires Python 3.5, which hasn't landed in Debian stable yet. Bummer.
- sandGorgon 10y ago
  can you not use pyenv?
  - alrs 10y ago
    Not if you want to get security updates and run a binary that was compiled against the libraries on your system. If you start compiling Python yourself you need to start following mailing lists to keep up on security updates and running build servers to spit out your artifact, lest you put the entire gcc toolchain in to your production environment.
    - _joel 10y ago
      What about things installed via pip that are not part of the package manager, is that not the same thing as with pyenv? You could use debian's tooling or fpm to turn pip->dpkg but you still have the same problem of maintenance (which tbh isn't that big of a deal if you're developing, it's not a production env) Also, I don't personally see what's wrong with having a compiler on production systems. Granted, if you don't need it and want to reduce bloat but treating it as a security hole is misguided imho. If the actor can't run gcc, then the could still upload their pre-compiled version, otherwise how would they get the source on to compile in the first place. Just seems a pointless measure
- dijit 10y ago
  "It'll get there" But seriously. If there's demand it'll get there. I've been using py3.5 on freebsd, openbsd and Archlinux for months. If enough software is written using the new hotness like Async, there will be pressure and motivation to get it packaged into stable. I hope. For people on RHEL, the IUS repository[0] guys do a great job. [0] https://ius.io/ https://ius.io/
viraptor 10y ago
So apart from having separate disk+network namespace, how is this different from a virtualenv? I feel like configurable sockets and paths should provide everything this can, but with less overhead.
- scott113341 10y ago
  If I understand correctly, the container you develop in can also run a different operating system in a completely separate environment.
- cookiecaper 10y ago
  Less system resource overhead, more human overhead. VirtualEnvs can bleed over if they're not configured properly or if you forget to deactivate. They're harder to fully encapsulate and carry around (have to initiate with a special flag or there's a bunch of manual find+replace when changing paths). They sometimes end up with system dependencies (though `--no-site-packages` has been the default for a long time now). This would be a shortcut that bypasses the need for venvs entirely. I'm not sure I would use it, but that's the difference I see.
  - falsedan 10y ago
    There are a couple of tools to automatically manage virtualenv activation/deactivation, like autoenv[0] & detect_virtualenv[1]; [0]: https://github.com/kennethreitz/autoenv https://github.com/kennethreitz/autoenv [1]: https://github.com/dgholz/detect_virtualenv https://github.com/dgholz/detect_virtualenv
symlinkk 10y ago
Why would you want to take a cross-platform interpreted language and run it under a pseudo-VM like Docker?
- t0mbstone 10y ago
  Security and portability
- Spivak 10y ago
  Docker is less of an pseudo-VM and more like a linker. No matter what the environment is like on the host you can be sure that it's identical to yours within the container.
civodul 10y ago
Docker sounds heavyweight for this purpose, given that it essentially spawns a complete userland separate from the host. I work on Guix; it provides a 'guix environment' command, which is sort of a generic 'virtualenv', and IMO is a simple lightweight alternative: https://gnu.org/s/guix/manual/html_node/Invoking-guix-environment.html https://gnu.org/s/guix/manual/html_node/Invoking-guix-enviro... .
wtvanhest 10y ago
Slightly off-topic, but does anyone develop Django/Python on a windows machine, and may want to offer any general advice or tips? I am using virtualenv, but wondering what else I should be doing to make it easier. Previously I used cloud9 (cloud IDE) for rails development. In both cases, I am pretty much a hobbiest, but don't mind spending some money to make things easier since I'd rather spend my time doing the fun part, rather than developer ops.
- throwawayish 10y ago
  You probably want to use PyCharm. It's a great IDE, and pretty much just works. Personally I find the various CLI tools typically found on a BSD or Linux rather enticing for development, but many people do without that. Apart from that I'm not sure what you could mean / what your issues are. Specify?
- nkantar 10y ago
  My two second-hand cents: At my previous company we used Vagrant for most projects, and it provided a nice abstraction layer that made things work reasonably well across all platforms (macOS, Windows, and Linux). Vagrant on Windows definitely had its challenges, though, but projects themselves ran well once that was sorted out. I've heard the company has since moved to Docker, in search of those same benefits but with smaller overhead (specifically in terms of time spent managing the abstraction layer), and I gather the Windows folks are happier with it than they ever were with Vagrant. My own (macOS-only) experience with both has been mixed, but certainly not worse than just running things locally via virtualenv. Vagrant introduces a full VM into the stack and Docker seems to have a lot of stability issues (at least on macOS 10.12 Sierra), so my next plan is to try combining the two, and running Docker on a Fedora VM. I'm hoping that any Docker nightmares will then at least be confined to the guest system, leaving my main machine mostly out of it, and using Docker to set up the actual project stuff may mean less Linux administration of the VM itself.
  - throwawayish 10y ago
    Personally I try to work as little with VMs as makes sense, they usually just make everything a bit harder and convoluted ("even" with Vagrant which, so far, has been a constant maintenance burden in all projects were I witnessed it's application, especially so if it's not only for Linux). My experiences with docker range from disastrous to annoying; I try to just not use it. It's not installed on any of my systems. If I have to use it, I do that in a VM. (I developed for some eight years on Windows, mostly C/C++, but last two years a lot of Python. At the time I wasn't aware of PyCharm, so Sublime Text "is also an editor" it was, but obviously just an editor; for development PyCharm is infinitely superior.)

119 points by macostea 10y ago

If it meets the guidelines, this might make a good 'Show HN'. Show HN guidelines: https://news.ycombinator.com/showhn.html https://news.ycombinator.com/showhn.html

macostea 10y ago
Thanks for the suggestion! I'll try it out

rockostrich 10y ago

Pretty simple to set up and use, but I'm not sure if it's better than learning `docker` and `docker-compose`. Also, the default `python <file>.py` doesn't remove the container after the file is done running and the image is tagged as `cage/<name of project>` instead of allowing the user to tag it with their own name. Both of these are easy to fix with `docker` commands, but if this is meant to be a library that helps you avoid `docker` then it's not doing its job.

gshulegaard 10y ago
I was going to say something similar. With venv moved into the stdlib in Python3 and ongoing development and improvements to the `docker-compose` workflow I am not sure where a tool like this fits. * venv is lighter and more or less fairly standardized * `docker-compose` is more general and language agnostic So something like `cage` is somewhere between...it's neat, no doubt, but not sure where I would use it personally.
macostea 10y ago
Thanks for the feedback! I'll make sure to add the container removal and naming as new features. You can indeed run all the commands easily just with Docker. Cage aims to port all the functionalities from virtualenv to use Docker. After this is achieved I can start working on extending those functionalities.

rch 10y ago

> The current version of Cage only supports dependencies written in a requirements file It might be worth looking at pipfile. Great project though!

crazyhatfish 10y ago

Although I'm not sure I would use this, this is the kind of stuff I've been waiting for. Where nice clean containers are used as part of an active dev cycle rather than just the production push. The ultimate dream of the end user using a desktop comprising only of containers seems to be still a bit far off yet...

bschwindHN 10y ago
About a year ago I switched our dev environments from Vagrant to docker-compose environments with a "helper" shell script to run the most common docker-compose operations you'd do in a day. It starts up so much faster than Vagrant, it's easy to add new services, and it's easy to add and remove test data. Here's an example of the kind of setup I have: https://github.com/bschwind/api-starter-kit/blob/master/helper.sh https://github.com/bschwind/api-starter-kit/blob/master/help... What's funny is I don't currently use docker in production, but I love it for dev environments.

0wl3x 10y ago

Could someone please explain why this might be useful? It seems like something that a crypto developer might like but if I'm developing something innocuous why would it matter if it runs inside a container?

apetresc 10y ago
Why would a crypto developer be especially interested in this?
- _joel 10y ago
  I have feeling the OP is thinking of air-gapping or clean room builds, but I could be wrong.
  - 0wl3x 10y ago
    yeah that's exactly the kind of use case I was thinking of.

miloshadzic 10y ago

Docker runs pretty slow on macOS so you'll be paying a pretty high price for cleanliness.

alrs 10y ago

Mac-first development strikes again: pycage requires Python 3.5, which hasn't landed in Debian stable yet. Bummer.

sandGorgon 10y ago
can you not use pyenv?
- alrs 10y ago
  Not if you want to get security updates and run a binary that was compiled against the libraries on your system. If you start compiling Python yourself you need to start following mailing lists to keep up on security updates and running build servers to spit out your artifact, lest you put the entire gcc toolchain in to your production environment.
  - _joel 10y ago
    What about things installed via pip that are not part of the package manager, is that not the same thing as with pyenv? You could use debian's tooling or fpm to turn pip->dpkg but you still have the same problem of maintenance (which tbh isn't that big of a deal if you're developing, it's not a production env) Also, I don't personally see what's wrong with having a compiler on production systems. Granted, if you don't need it and want to reduce bloat but treating it as a security hole is misguided imho. If the actor can't run gcc, then the could still upload their pre-compiled version, otherwise how would they get the source on to compile in the first place. Just seems a pointless measure
dijit 10y ago
"It'll get there" But seriously. If there's demand it'll get there. I've been using py3.5 on freebsd, openbsd and Archlinux for months. If enough software is written using the new hotness like Async, there will be pressure and motivation to get it packaged into stable. I hope. For people on RHEL, the IUS repository[0] guys do a great job. [0] https://ius.io/ https://ius.io/

viraptor 10y ago

So apart from having separate disk+network namespace, how is this different from a virtualenv? I feel like configurable sockets and paths should provide everything this can, but with less overhead.

scott113341 10y ago
If I understand correctly, the container you develop in can also run a different operating system in a completely separate environment.
cookiecaper 10y ago
Less system resource overhead, more human overhead. VirtualEnvs can bleed over if they're not configured properly or if you forget to deactivate. They're harder to fully encapsulate and carry around (have to initiate with a special flag or there's a bunch of manual find+replace when changing paths). They sometimes end up with system dependencies (though `--no-site-packages` has been the default for a long time now). This would be a shortcut that bypasses the need for venvs entirely. I'm not sure I would use it, but that's the difference I see.
- falsedan 10y ago
  There are a couple of tools to automatically manage virtualenv activation/deactivation, like autoenv[0] & detect_virtualenv[1]; [0]: https://github.com/kennethreitz/autoenv https://github.com/kennethreitz/autoenv [1]: https://github.com/dgholz/detect_virtualenv https://github.com/dgholz/detect_virtualenv

symlinkk 10y ago

Why would you want to take a cross-platform interpreted language and run it under a pseudo-VM like Docker?

t0mbstone 10y ago
Security and portability
Spivak 10y ago
Docker is less of an pseudo-VM and more like a linker. No matter what the environment is like on the host you can be sure that it's identical to yours within the container.

civodul 10y ago

Docker sounds heavyweight for this purpose, given that it essentially spawns a complete userland separate from the host. I work on Guix; it provides a 'guix environment' command, which is sort of a generic 'virtualenv', and IMO is a simple lightweight alternative: https://gnu.org/s/guix/manual/html_node/Invoking-guix-environment.html https://gnu.org/s/guix/manual/html_node/Invoking-guix-enviro... .

wtvanhest 10y ago

Slightly off-topic, but does anyone develop Django/Python on a windows machine, and may want to offer any general advice or tips? I am using virtualenv, but wondering what else I should be doing to make it easier. Previously I used cloud9 (cloud IDE) for rails development. In both cases, I am pretty much a hobbiest, but don't mind spending some money to make things easier since I'd rather spend my time doing the fun part, rather than developer ops.

throwawayish 10y ago
You probably want to use PyCharm. It's a great IDE, and pretty much just works. Personally I find the various CLI tools typically found on a BSD or Linux rather enticing for development, but many people do without that. Apart from that I'm not sure what you could mean / what your issues are. Specify?
nkantar 10y ago
My two second-hand cents: At my previous company we used Vagrant for most projects, and it provided a nice abstraction layer that made things work reasonably well across all platforms (macOS, Windows, and Linux). Vagrant on Windows definitely had its challenges, though, but projects themselves ran well once that was sorted out. I've heard the company has since moved to Docker, in search of those same benefits but with smaller overhead (specifically in terms of time spent managing the abstraction layer), and I gather the Windows folks are happier with it than they ever were with Vagrant. My own (macOS-only) experience with both has been mixed, but certainly not worse than just running things locally via virtualenv. Vagrant introduces a full VM into the stack and Docker seems to have a lot of stability issues (at least on macOS 10.12 Sierra), so my next plan is to try combining the two, and running Docker on a Fedora VM. I'm hoping that any Docker nightmares will then at least be confined to the guest system, leaving my main machine mostly out of it, and using Docker to set up the actual project stuff may mean less Linux administration of the VM itself.
- throwawayish 10y ago
  Personally I try to work as little with VMs as makes sense, they usually just make everything a bit harder and convoluted ("even" with Vagrant which, so far, has been a constant maintenance burden in all projects were I witnessed it's application, especially so if it's not only for Linux). My experiences with docker range from disastrous to annoying; I try to just not use it. It's not installed on any of my systems. If I have to use it, I do that in a VM. (I developed for some eight years on Windows, mostly C/C++, but last two years a lot of Python. At the time I wasn't aware of PyCharm, so Sublime Text "is also an editor" it was, but obviously just an editor; for development PyCharm is infinitely superior.)