9 ms·

Local Rails Development with Docker and Docker Compose

34 points by ashconnor 9y ago

ungzd 9y ago
Will auto reload work with this setup? Docker for Mac page claims "Seamless volume mounting for code and data, including file change notifications that unlock fast edit-test cycles", but last time I tried it, it didn't work, it just copied workdir to VM once.
- nikentic 9y ago
  Yes, auto reload works just fine as you mount the directory and not just ADD it.
dankohn1 9y ago
This is a good start, but it requires reinstalling all gems every time any one is updated. Here is a clever way to cache your gems in a separate container https://medium.com/@fbzga/how-to-cache-bundle-install-with-docker-7bed453a5800 https://medium.com/@fbzga/how-to-cache-bundle-install-with-d... I'm working on an example Docker compose setup that makes use of multi-stage builds to copy the gems into the final container, leaving behind all of the build tools. https://docs.docker.com/engine/userguide/eng-image/multistage-build/#use-multi-stage-builds https://docs.docker.com/engine/userguide/eng-image/multistag...
StavrosK 9y ago
Nice writeup! Here's also the equivalent for us Django scrubs: https://www.stavros.io/posts/how-deploy-django-docker/ https://www.stavros.io/posts/how-deploy-django-docker/ Once you've done this, then it becomes trivial to deploy to Dokku, if you have your own VPS or dedicated server. I tried Dokku for the first time a week or so ago and I'm a fan: https://www.stavros.io/posts/deploy-django-dokku/ https://www.stavros.io/posts/deploy-django-dokku/
evbots 9y ago
Over the past week I went through all the motions of setting up docker for mac with docker-compose for a local ruby/rails development environment. I was excited about the prospect of using containers to silo my app and all it's system dependencies, and also make it easier to onboard new developers. My setup had 3 containers. One for app, elasticsearch, and postgres. Unfortunately I have to go back to running my app directly on the system. I am leaving elasticsearch and postgres containerized, however. The docker for mac setup is seriously bottlenecked by slow volume read/write performance. There is an ongoing discussion here on the issue: https://github.com/docker/for-mac/issues/77 https://github.com/docker/for-mac/issues/77. Even after adding the new :cached option to my volume declaration in docker-compose yml file, performance was a huge issue. My app leverages babel, as I write some react components in ES6 and transpile them to ship to the client. Right now transpilation after changing any javascript code results in a minute + of transpilation time as docker for mac has yet to solve the performance problem. This resulted in a totally unacceptable environment to write code and test in. Also, rails boot time on docker for mac was crazy slow. Running unit tests became a huge pain in the ass.
- eadz 9y ago
  Agreed. If you're using webpack, dev-in-docker won't work well at if at all. For an API-only rails app, it does seem to be fast enough.
- s_kilk 9y ago
  The ghastly performance of docker-for-mac was a large factor in my move back to Linux (a few months ago) after a few years on OSX.
  - ty_a 9y ago
    I see a roughly 66% performance hit on docker for Mac in most metrics. Not a deal breaker but I prefer to do development on Linux as well for that reason.
conorh 9y ago
We've attempted this setup a few times with Rails, but we ran into the issue that the Docker OSXFS performance for mounted volumes was too slow when working with the parts of Rails that touch many files (asset compilation for example). It made it pretty much unusable for us. I know there are some new tuning parameters https://docs.docker.com/docker-for-mac/osxfs-caching/ https://docs.docker.com/docker-for-mac/osxfs-caching/ - we haven't tried these yet though, has anyone tried them and do they work well enough for Rails?
- evbots 9y ago
  I just tried :cached. For rails asset compilation I didn't notice a difference. Still unusable in my opinion.
xjx 9y ago
good
dcosson 9y ago
Nice concise write-up. Worth also calling out some issues you'll run into though: - docker for mac's osxfs is incredibly slow and makes the cold rails start time even slower than it already is. Docker-sync is a third party tool that makes it pretty easy to sync code over instead of using a shared file system. - similarly, rails cold start times are so slow on their own that you won't want to 'docker run' your commands. I always 'docker exec' them into the already running rails container to take advantage of the spring cache. - the bundle incremental cache isn't used (and if you have a package.json for us resources, you'll have to solve the same problem for that too). These are arguably downsides of both ruby/rails and docker that combine to make things a little harder to work with than you'd hope. Still, its a useful workflow if you want to deploy to docker in production. It's nice running the app in the same OS with the same exact dependencies in dev/ci as in prod (plus the same configuration between different people on the team).
yagyu 9y ago
Here's a minimal template for local dev in Python, flask, psql if anyone looked for one: https://github.com/jeinarsson/app-template https://github.com/jeinarsson/app-template
vinceguidry 9y ago
Has anyone tried using Docker with something more performant than VirtualBox on OSX? I won't use Vagrant or Docker in my projects for this reason, but I suspect it's VirtualBox that's the problem.
- webo 9y ago
  We use vagrant with VMware Fusion. Haven't had much problem other than OSX upgrades occasionally breaking vagrant itself.
- ty_a 9y ago
  Docker for Mac uses Xhyve, a hypervisor ported from FreeBSD. It could be more performant than Virtualbox but I have not tested them side by side.
- molecule 9y ago
  For VirtualBox alone, using NFS sharing is significantly faster. For Docker, dlite uses NFS and doesn't require VirtualBox. https://github.com/nlf/dlite https://github.com/nlf/dlite (See README on legacy branch https://github.com/nlf/dlite/tree/legacy https://github.com/nlf/dlite/tree/legacy )
hashkb 9y ago
The biggest blocker I've run into is that the Docker build process can't cache bundler/npm/etc runs so every dependency change results in a full rebuild. Is that solved yet? If not, local dev with Docker is still a no-go for me.
- molecule 9y ago
  The solution for that problem is addressed by the author, volume sharing of the app directory: Docker command line: -v $(pwd):/app Docker Compose: app: volumes: ./:/app
gkop 9y ago
I super duper do not recommend this. Read all the caveats in this thread, they are all genuine annoyances. I'll add another one: interactive debugging works, but has quirks. The cost/benefit just isn't there unless your system is quite complicated (think engineering team headcount 100+). It makes everyday front-end development like building a Lego castle while wearing oven mitts. (I still appreciate this article, lots of goood info in it :) )
- michaelrshannon 9y ago
  > It makes everyday front-end development like building a Lego castle while wearing oven mitts. Favourite analogy of the month :D
majewsky 9y ago
@ashconnor: In the "RUN apk add" line, consider adding "--no-cache" to avoid useless temporary files in the image. A quick and nice size improvement without any downsides that I could think of.
- ashconnor 9y ago
  Good catch. Thanks.
hbogert 9y ago
Really eager to know, if the writer would still use it like this, when his project gets bigger. For instance, using Docker on Mac like this is a pure pain. First of all, it's a pain because of Docker's lack of proper UID mapping(and no.. user namespaces is not the answer). Second, sharing files between OSX and containers is extremely slow. A PHP/Symfony project went from a 2 second reload of in dev/debug mode on native install, to 30+ seconds for a reload inside a docker container; unless you won't use host volumes.

34 points by ashconnor 9y ago

Will auto reload work with this setup? Docker for Mac page claims "Seamless volume mounting for code and data, including file change notifications that unlock fast edit-test cycles", but last time I tried it, it didn't work, it just copied workdir to VM once.

nikentic 9y ago
Yes, auto reload works just fine as you mount the directory and not just ADD it.

dankohn1 9y ago

This is a good start, but it requires reinstalling all gems every time any one is updated. Here is a clever way to cache your gems in a separate container https://medium.com/@fbzga/how-to-cache-bundle-install-with-docker-7bed453a5800 https://medium.com/@fbzga/how-to-cache-bundle-install-with-d... I'm working on an example Docker compose setup that makes use of multi-stage builds to copy the gems into the final container, leaving behind all of the build tools. https://docs.docker.com/engine/userguide/eng-image/multistage-build/#use-multi-stage-builds https://docs.docker.com/engine/userguide/eng-image/multistag...

StavrosK 9y ago

Nice writeup! Here's also the equivalent for us Django scrubs: https://www.stavros.io/posts/how-deploy-django-docker/ https://www.stavros.io/posts/how-deploy-django-docker/ Once you've done this, then it becomes trivial to deploy to Dokku, if you have your own VPS or dedicated server. I tried Dokku for the first time a week or so ago and I'm a fan: https://www.stavros.io/posts/deploy-django-dokku/ https://www.stavros.io/posts/deploy-django-dokku/

evbots 9y ago

Over the past week I went through all the motions of setting up docker for mac with docker-compose for a local ruby/rails development environment. I was excited about the prospect of using containers to silo my app and all it's system dependencies, and also make it easier to onboard new developers. My setup had 3 containers. One for app, elasticsearch, and postgres. Unfortunately I have to go back to running my app directly on the system. I am leaving elasticsearch and postgres containerized, however. The docker for mac setup is seriously bottlenecked by slow volume read/write performance. There is an ongoing discussion here on the issue: https://github.com/docker/for-mac/issues/77 https://github.com/docker/for-mac/issues/77. Even after adding the new :cached option to my volume declaration in docker-compose yml file, performance was a huge issue. My app leverages babel, as I write some react components in ES6 and transpile them to ship to the client. Right now transpilation after changing any javascript code results in a minute + of transpilation time as docker for mac has yet to solve the performance problem. This resulted in a totally unacceptable environment to write code and test in. Also, rails boot time on docker for mac was crazy slow. Running unit tests became a huge pain in the ass.

eadz 9y ago
Agreed. If you're using webpack, dev-in-docker won't work well at if at all. For an API-only rails app, it does seem to be fast enough.
s_kilk 9y ago
The ghastly performance of docker-for-mac was a large factor in my move back to Linux (a few months ago) after a few years on OSX.
- ty_a 9y ago
  I see a roughly 66% performance hit on docker for Mac in most metrics. Not a deal breaker but I prefer to do development on Linux as well for that reason.

conorh 9y ago

We've attempted this setup a few times with Rails, but we ran into the issue that the Docker OSXFS performance for mounted volumes was too slow when working with the parts of Rails that touch many files (asset compilation for example). It made it pretty much unusable for us. I know there are some new tuning parameters https://docs.docker.com/docker-for-mac/osxfs-caching/ https://docs.docker.com/docker-for-mac/osxfs-caching/ - we haven't tried these yet though, has anyone tried them and do they work well enough for Rails?

evbots 9y ago
I just tried :cached. For rails asset compilation I didn't notice a difference. Still unusable in my opinion.

xjx 9y ago

good

dcosson 9y ago

Nice concise write-up. Worth also calling out some issues you'll run into though: - docker for mac's osxfs is incredibly slow and makes the cold rails start time even slower than it already is. Docker-sync is a third party tool that makes it pretty easy to sync code over instead of using a shared file system. - similarly, rails cold start times are so slow on their own that you won't want to 'docker run' your commands. I always 'docker exec' them into the already running rails container to take advantage of the spring cache. - the bundle incremental cache isn't used (and if you have a package.json for us resources, you'll have to solve the same problem for that too). These are arguably downsides of both ruby/rails and docker that combine to make things a little harder to work with than you'd hope. Still, its a useful workflow if you want to deploy to docker in production. It's nice running the app in the same OS with the same exact dependencies in dev/ci as in prod (plus the same configuration between different people on the team).

yagyu 9y ago

Here's a minimal template for local dev in Python, flask, psql if anyone looked for one: https://github.com/jeinarsson/app-template https://github.com/jeinarsson/app-template

vinceguidry 9y ago

Has anyone tried using Docker with something more performant than VirtualBox on OSX? I won't use Vagrant or Docker in my projects for this reason, but I suspect it's VirtualBox that's the problem.

webo 9y ago
We use vagrant with VMware Fusion. Haven't had much problem other than OSX upgrades occasionally breaking vagrant itself.
ty_a 9y ago
Docker for Mac uses Xhyve, a hypervisor ported from FreeBSD. It could be more performant than Virtualbox but I have not tested them side by side.
molecule 9y ago
For VirtualBox alone, using NFS sharing is significantly faster. For Docker, dlite uses NFS and doesn't require VirtualBox. https://github.com/nlf/dlite https://github.com/nlf/dlite (See README on legacy branch https://github.com/nlf/dlite/tree/legacy https://github.com/nlf/dlite/tree/legacy )

hashkb 9y ago

The biggest blocker I've run into is that the Docker build process can't cache bundler/npm/etc runs so every dependency change results in a full rebuild. Is that solved yet? If not, local dev with Docker is still a no-go for me.

molecule 9y ago
The solution for that problem is addressed by the author, volume sharing of the app directory: Docker command line: -v $(pwd):/app Docker Compose: app: volumes: ./:/app

gkop 9y ago

I super duper do not recommend this. Read all the caveats in this thread, they are all genuine annoyances. I'll add another one: interactive debugging works, but has quirks. The cost/benefit just isn't there unless your system is quite complicated (think engineering team headcount 100+). It makes everyday front-end development like building a Lego castle while wearing oven mitts. (I still appreciate this article, lots of goood info in it :) )

michaelrshannon 9y ago
> It makes everyday front-end development like building a Lego castle while wearing oven mitts. Favourite analogy of the month :D

majewsky 9y ago

@ashconnor: In the "RUN apk add" line, consider adding "--no-cache" to avoid useless temporary files in the image. A quick and nice size improvement without any downsides that I could think of.

ashconnor 9y ago
Good catch. Thanks.

hbogert 9y ago

Really eager to know, if the writer would still use it like this, when his project gets bigger. For instance, using Docker on Mac like this is a pure pain. First of all, it's a pain because of Docker's lack of proper UID mapping(and no.. user namespaces is not the answer). Second, sharing files between OSX and containers is extremely slow. A PHP/Symfony project went from a 2 second reload of in dev/debug mode on native install, to 30+ seconds for a reload inside a docker container; unless you won't use host volumes.