6 ms·

Setting Up Your Development Environment with Docker Compose

76 points by avh02 10y ago

clintonb 10y ago
We use a similar setup for some of our services at edX. It is worth noting that you may need to adjust your development behaviors/environment somewhat to accommodate Docker. For example, PyCharm supports using Docker Machine but not the new Docker for Mac. This means you won't necessarily have access to PyCharm's code inspection and remote debugging if you solely use Docker. This has resulted in my having a virtualenv on my host machine as well as the Docker setup, which is not ideal. Once the tooling is better, things will be great; however, we aren't quite there yet.
- avh02 10y ago
  yeah, the tools have some way to go, I did have issues with virtualenvs like you mentioned but given how fast the docker ecosystem has developed, I expect them to make it there soon enough.
k__ 10y ago
Nice timing. My dev env is deteriorating after 2 years use and I'll set up a new one this weekend. Probably going to use NixOS, but also looking into Docker.
- SmurfJuggler 10y ago
  I wanted to learn some things a while back (can't remember what, I think it was laravel and angular) and set about creating a dev environment for it. Whatever I started out to learn fell by the wayside and building a shit-hot, easy to use, modular, containerised dev environment became my soul focus. I'm getting ready to open it up to the world any day now[1]. I'll post about it once I've ironed out a few more quirks. It makes life so much less unpleasant in so many ways. [1] yeah we'll see how that goes
- swsieber 10y ago
  Use both - Using Nix inside Docker will make the builds truly reproducible.
MobiKid 10y ago
This is how we do development as well, it's been working out great! One thing to note is that volumes can act weird at times if using docker-toolbox (I know the Blog Post was about using a Linux host). https://www.virtualbox.org/ticket/14920 https://www.virtualbox.org/ticket/14920 As a workaround we set up an alias for opening the file in VIM and saving it (without making any changes) that would run from within the container. Like the following: vim <filename> -c 'wq!'
- kayvansylvan 10y ago
  I don't think you'll see these problems with the new Docker 1.12 for Mac (no docker toolbox), which brings the experience on the Mac much closer to what you see on Linux boxes.
jsnk 10y ago
"Mount your code, but don’t forget to restart your services in between changes" This is the part I find unacceptable for a development environment. Is there a way to have live-reloading of docker instance somehow?
- enoch_r 10y ago
  If you're running something that doesn't do live code reloading, it won't work without restarting whether or not you're running it in a container. If you're running something that does do live code reloading, it should work just the same inside a container, assuming you have the volumes mounted.
- jackweirdy 10y ago
  The code does change inside the docker container when it's mounted as a volume, you just have to watch for it. We (at http://www.duecourse.com http://www.duecourse.com, a mostly node + Microservices shop) use nodemon for this
  - was_boring 10y ago
    How do you deal with node_ modules and its insistence of being in the same folder as the source for your app? This has been one of the great challenges for me. Nothing seems to work well with volume mapping.
    - jackweirdy 10y ago
      Yeah, that's a PITA. We currently just don't mount node_modules at all, and run npm install when we build the image, and have to re-build whenever we change package.json. It sounds bad, but once we've got something working we tend not to need to add new dependencies, it's normally just code changes from there on out. As an example, here's a docker compose for one service: ports: - "20003:20003" volumes: - ./APP/src:/opt/app/src - ./APP/test:/opt/app/test command: npm run dev `run dev` is the thing that calls nodemon, which reloads whenever src or test change. Otherwise, everything like node_modules exists inside the image.
blain_the_train 10y ago
> webapp declares an env_file from which to pick up env vars. This is important because we configure a lot of our applications with environment variables so that the same docker image can run in different environments, taking inspiration from 12-factor apps. Keep in mind that your environment variables can be learned by inspecting the image. So if you make that image public, those envs are visible. As far as i understand, this is analogous to committing your passwords to source control and should be a consideration.
- throway21312 10y ago
  What would the correct methodology be?
- jasonmp85 10y ago
  Huh? I mean, sure, you can probably figure out the _names_ of the environment variables expected by the code of the image, but the image itself doesn't have the values for those variables. Environment files are—unless I'm missing something—a run-time consideration. You provide the an env-file you have locally to connect to your development images, etc., and could deploy (possibly) the same image elsewhere, with a different env-file. But the env-file values are not baked into the image.
  - Feuilles_Mortes 10y ago
    This is correct - I just set this up today.
  - girvo 10y ago
    They _can_ be baked into the image via build-args or the "ENV" Dockerfile syntax, but as your Dockerfile is committed to git normally this has the same dramas, of course! You're completely correct, but I wanted to mention that it was possible :)
    - droningparrot 10y ago
      Build-args and ENVs are not the same thing. Build-args use the ARG syntax and are baked in during docker build. ENVs are not baked in; the values are passed in during docker run or docker exec.
girvo 10y ago
Semi-related, but I had issues with Docker Compose and database volumes for web-development work (where the database volume-only container to allow for persistence across containers would be recreated, thus losing the data that was there). This prompted a tiny wrapper around some patterns we were using, exposed via a CLI tool and JSON file: https://github.com/girvo/dup https://github.com/girvo/dup Even more unrelated: it's written in Nim, and a big refactor is coming along that works with the Docker socket directly (either the AF_UNIX or TCP socket, depending on what's available)
tunesmith 10y ago
I was just starting to play with this last night. Frustrating couple of hours because it turns out that IntelliJ's Docker plugin isn't yet compatible with Docker For Mac, and in a way that isn't JetBrains' fault. You have to do some kind of weird workaround with socat.
droningparrot 10y ago
I'm really not a fan of absolute paths in the compose file: services: volumes: - /path/to/my/code/on/my/dev/machine:/path/to/the/code/in/the/container If you're sharing your docker-compose.yml file with other developers, this will get annoying really quickly. Normally, I'd place the docker-compose.yml file in the root of my project, and then replace "/path/to/my/code/on/my/dev/machine" with ".". That way, it doesn't matter where other devs keep code on their machines. In the case where you have a compose file that builds multiple projects (See: double time!), keep the compose file in one git repo, and then include the code for the other two projects (webappone and webapptwo) as submodules.

76 points by avh02 10y ago

We use a similar setup for some of our services at edX. It is worth noting that you may need to adjust your development behaviors/environment somewhat to accommodate Docker. For example, PyCharm supports using Docker Machine but not the new Docker for Mac. This means you won't necessarily have access to PyCharm's code inspection and remote debugging if you solely use Docker. This has resulted in my having a virtualenv on my host machine as well as the Docker setup, which is not ideal. Once the tooling is better, things will be great; however, we aren't quite there yet.

avh02 10y ago
yeah, the tools have some way to go, I did have issues with virtualenvs like you mentioned but given how fast the docker ecosystem has developed, I expect them to make it there soon enough.

k__ 10y ago

Nice timing. My dev env is deteriorating after 2 years use and I'll set up a new one this weekend. Probably going to use NixOS, but also looking into Docker.

SmurfJuggler 10y ago
I wanted to learn some things a while back (can't remember what, I think it was laravel and angular) and set about creating a dev environment for it. Whatever I started out to learn fell by the wayside and building a shit-hot, easy to use, modular, containerised dev environment became my soul focus. I'm getting ready to open it up to the world any day now[1]. I'll post about it once I've ironed out a few more quirks. It makes life so much less unpleasant in so many ways. [1] yeah we'll see how that goes
swsieber 10y ago
Use both - Using Nix inside Docker will make the builds truly reproducible.

MobiKid 10y ago

This is how we do development as well, it's been working out great! One thing to note is that volumes can act weird at times if using docker-toolbox (I know the Blog Post was about using a Linux host). https://www.virtualbox.org/ticket/14920 https://www.virtualbox.org/ticket/14920 As a workaround we set up an alias for opening the file in VIM and saving it (without making any changes) that would run from within the container. Like the following: vim <filename> -c 'wq!'

kayvansylvan 10y ago
I don't think you'll see these problems with the new Docker 1.12 for Mac (no docker toolbox), which brings the experience on the Mac much closer to what you see on Linux boxes.

jsnk 10y ago

"Mount your code, but don’t forget to restart your services in between changes" This is the part I find unacceptable for a development environment. Is there a way to have live-reloading of docker instance somehow?

enoch_r 10y ago
If you're running something that doesn't do live code reloading, it won't work without restarting whether or not you're running it in a container. If you're running something that does do live code reloading, it should work just the same inside a container, assuming you have the volumes mounted.
jackweirdy 10y ago
The code does change inside the docker container when it's mounted as a volume, you just have to watch for it. We (at http://www.duecourse.com http://www.duecourse.com, a mostly node + Microservices shop) use nodemon for this
- was_boring 10y ago
  How do you deal with node_ modules and its insistence of being in the same folder as the source for your app? This has been one of the great challenges for me. Nothing seems to work well with volume mapping.
  - jackweirdy 10y ago
    Yeah, that's a PITA. We currently just don't mount node_modules at all, and run npm install when we build the image, and have to re-build whenever we change package.json. It sounds bad, but once we've got something working we tend not to need to add new dependencies, it's normally just code changes from there on out. As an example, here's a docker compose for one service: ports: - "20003:20003" volumes: - ./APP/src:/opt/app/src - ./APP/test:/opt/app/test command: npm run dev `run dev` is the thing that calls nodemon, which reloads whenever src or test change. Otherwise, everything like node_modules exists inside the image.

blain_the_train 10y ago

> webapp declares an env_file from which to pick up env vars. This is important because we configure a lot of our applications with environment variables so that the same docker image can run in different environments, taking inspiration from 12-factor apps. Keep in mind that your environment variables can be learned by inspecting the image. So if you make that image public, those envs are visible. As far as i understand, this is analogous to committing your passwords to source control and should be a consideration.

throway21312 10y ago
What would the correct methodology be?
jasonmp85 10y ago
Huh? I mean, sure, you can probably figure out the _names_ of the environment variables expected by the code of the image, but the image itself doesn't have the values for those variables. Environment files are—unless I'm missing something—a run-time consideration. You provide the an env-file you have locally to connect to your development images, etc., and could deploy (possibly) the same image elsewhere, with a different env-file. But the env-file values are not baked into the image.
- Feuilles_Mortes 10y ago
  This is correct - I just set this up today.
- girvo 10y ago
  They _can_ be baked into the image via build-args or the "ENV" Dockerfile syntax, but as your Dockerfile is committed to git normally this has the same dramas, of course! You're completely correct, but I wanted to mention that it was possible :)
  - droningparrot 10y ago
    Build-args and ENVs are not the same thing. Build-args use the ARG syntax and are baked in during docker build. ENVs are not baked in; the values are passed in during docker run or docker exec.

girvo 10y ago

Semi-related, but I had issues with Docker Compose and database volumes for web-development work (where the database volume-only container to allow for persistence across containers would be recreated, thus losing the data that was there). This prompted a tiny wrapper around some patterns we were using, exposed via a CLI tool and JSON file: https://github.com/girvo/dup https://github.com/girvo/dup Even more unrelated: it's written in Nim, and a big refactor is coming along that works with the Docker socket directly (either the AF_UNIX or TCP socket, depending on what's available)

tunesmith 10y ago

I was just starting to play with this last night. Frustrating couple of hours because it turns out that IntelliJ's Docker plugin isn't yet compatible with Docker For Mac, and in a way that isn't JetBrains' fault. You have to do some kind of weird workaround with socat.

droningparrot 10y ago

I'm really not a fan of absolute paths in the compose file: services: volumes: - /path/to/my/code/on/my/dev/machine:/path/to/the/code/in/the/container If you're sharing your docker-compose.yml file with other developers, this will get annoying really quickly. Normally, I'd place the docker-compose.yml file in the root of my project, and then replace "/path/to/my/code/on/my/dev/machine" with ".". That way, it doesn't matter where other devs keep code on their machines. In the case where you have a compose file that builds multiple projects (See: double time!), keep the compose file in one git repo, and then include the code for the other two projects (webappone and webapptwo) as submodules.