3 ms·

If you don't use the same Docker Compose file for Production, don't use it for Development. Your two different systems will diverge in behavior, leading you to

If you don't use the same Docker Compose file for Production, don't use it for Development. Your two different systems will diverge in behavior, leading you to troubleshoot two separate sets of problems, and testing being unreliable, defeating the whole "it just runs everywhere" premise. Docker Compose is fine for running smoke tests/unit tests, but if Dev and Prod run differently, there's really no point to using Docker Compose other than developing without internet access. (It goes without saying that Docker Compose only works on a single host, so that won't work for Prod if you need more than one host, but single-host-everything seems to be HN's current fetish)

tjoff 4y ago
That is an absurdly black-and-white view.
- dijksterhuis 4y ago
  I have lost count of the times I've heard "but it works fine on MY machine, it must be the CI/CD that's broken" I then find out that no-one bothered to run the compose file that matches what gets put into prod on their local machines. My point is: your view might become more black and white on this matter after the N-th "but it works fine on MY machine" comment. EDIT: Where N = your personal tolerance level of bullshit.
  - remram 4y ago
    Sounds like this was easily caught by CI, which does run the prod compose file, and everything was fine.
    - dijksterhuis 4y ago
      This wasn't a real example. Just a generic thing that happens everywhere I go. There's always one in the crowd that refuses to give up local development.
      - remram 4y ago
        As long as you have CI or staging with the proper config, what does it matter how the devs write code?
        0xbadcafebee 4y ago
        The two aren't separate. There is a direct link between how devs write code and whether something works in production. And there is a ton of productivity loss and reliability loss when these two things are not in sync.
  - tjoff 4y ago
    That might me your experience, I don't share it. But even so, how you can go from that to: > [...] there's really no point to using Docker Compose other than developing without internet access boggles my mind. EDIT: apologies, mixup on my end and quote is from someone else.
    - dijksterhuis 4y ago
      Not the parent! Made no comment on developing without internet access. EDIT: I usually try to build, test, inspect, push and deploy from the same compose file. You can deploy to ECS direct from compose using an AWS context: https://aws.amazon.com/blogs/containers/deploy-applications-on-amazon-ecs-using-docker-compose/ https://aws.amazon.com/blogs/containers/deploy-applications-...
- deleted 4y ago
  [deleted]
bananaoomarang 4y ago
Compose also works well with Swarm in my experience.
nicoburns 4y ago
Meanwhile here I am developing on macOS (no Docker or VMs involved) and deploying to linux. We do run tests on a linux CI runner, and we have a staging environment that's identical to prod. But really cross-platform support tends to be really good these days. I see no need for identical environments for development.
dijksterhuis 4y ago
> defeating the whole "it just runs everywhere" premise. Straight up 100% hard agree. > It goes without saying that Docker Compose only works on a single host, so that won't work for Prod if you need more than one host, but single-host-everything seems to be HN's current fetish) technically this is only correct for compose file versions up to and including 3.8. Part of the push with the new mainline compose V2 plugin has included the updated compose specification [0] which seems to have unify quite a lot of differences between the "compose" file for compose deployments and the "stack" file for swarm deployments. FYI You can switch over to the new version with the compose V2 plugin by remvoing any `version` key from your compose file. Although the `deploy: restart_policy:` values are currently not unified, much to my sadness. [0]: https://docs.docker.com/compose/compose-file/ https://docs.docker.com/compose/compose-file/
twblalock 4y ago
Also, don't use Docker Compose in dev if you use Kubernetes in prod. Just use the same thing everywhere. Otherwise the fact that something works in dev tells you nothing about whether it would work in prod.
- 0xbadcafebee 4y ago
  ^ 1000%
- frenchman99 4y ago
  Running a local k8s cluster doesn't guarantee that a remote cluster on some cloud provider will work the same either. I've heard of and worked with multiple dev teams that develop with compose and deploy a totally different way (k8s, AWS auto scaling groups, heroku, etc), to great success. Devs know about compose and understand it. As long as the containers behave as planned, the DevOps people can figure out how to deploy them on whatever production environment.
  - twblalock 4y ago
    That's all fine, until the apps that ran fine on a laptop won't run in the real environments, and then the devops people need to loop in the devs to debug it. Quite a lot of the time, it will be a problem with the manifest not providing something the app container expects, like a permission or a port binding or a missing ConfigMap. Everyone loses productivity when that happens.
shroompasta 4y ago
This is a questionable comment >If you don't use the same Docker Compose file for Production, don't use it for Development. Your two different systems will diverge in behavior, leading you to troubleshoot two separate sets of problems, and testing being unreliable, defeating the whole "it just runs everywhere" premise. Anyone ever developing a react project on docker compose would never run a build step for development. In fact for most setups, I probably wouldn't even run react on docker-compose for development, and just use the dev server straight up on my machine. Furthermore, working with any python http servers is substantially easier when working with a dev server in development, rather than running your server through gunicorn or other wsgi servers - not to mention the ability to hot reload for development. I'm sure that there are many other cases where it's necessary and convenient to separate production and development docker compose files. I don't think it's fair to say that there is no point in having separate docker-compose files as the lion's share of dependencies that need to be consistent is inside each container, not on the docker-compose configuration level.
rdsubhas 4y ago
> ... if Dev and Prod run differently, there's really no point to ... "If dev and prod run differently" — Why is there an "if" here at all? Isn't it obvious to verify with two eyes in the real world? Is there any company in the world that runs production from somebody's laptop at home? Or distributes server racks for people to take home? It looks like an extremely disillusioned question out of touch with reality. 1. For development, engineers want to run a local throwaway mysql/redis. In production, engineer want to use a proper managed mysql/redis. The compose file, and envvars/links will be different. This is extremely normal, it makes sense. 2. Development laptops will always be underpowered than servers. The cpu/memory requirements will be different. Again makes sense. 3. For development, engineers will build and run the docker image locally. In production, these are separate, you would build and publish the image separately in CI, and run only published images in prod. Again, makes sense. These are all perfectly logical things that make practical sense. Trying to share/ reuse/ resemble is fine. But saying "dev===prod" is just a disillusion out of touch with reality. Edit: For context, yes I do docker for a living for last 10 years, since v0.4 (lxc) days. I've been to and presented in meetups. If someone says they're trying to get dev as close functionally to prod, by doing X, Y, Z and sharing practical tips — you know you can listen to them. If someone says "dev===prod" and when you ask how they talk abstract principles, you quietly get out of there.
- 0xbadcafebee 4y ago
  > For development, engineers want to run a local throwaway mysql/redis. In production, engineer want to use a proper managed mysql/redis. Bad idea. You develop against a completely different local system, then you push to prod, and "oh no it doesn't work". Yeah - because you're developing against a completely different system! It behaves differently. It causes different bugs. You take a ton of time setting up both different systems differently, duplicating your effort, duplicating your bugs, having to fix things twice for no reason, and causing production issues when expectations from local dev don't match prod reality. Every experienced systems engineer has been repeating this for decades. Listen to them. They all say exactly. the. same. thing. "Make all your systems as identical as possible." That doesn't mean make all your systems different whenever it is more convenient for you. It means that if it's possible for you to be developing using the same tech you use in prod, you should be doing that, unless it is impossible. If you think it's not possible or practical, you simply haven't thought hard enough about it. The whole selling point of the cloud is that you can spin up environments and spin them down in minutes and pay pennies for it. Not a shared environment where everyone's work is constantly conflicting, but dedicated, ephemeral environments that actually match production. Literally the only example I know of where this doesn't work is with physical devices where you have a lab of 10 experimental pieces of test gear and you have to reserve them to test code on them. That can make remote development difficult. Still not impossible though. > The cpu/memory requirements will be different. Unless you're developing using the same servers as prod. There is no reason that you have to use your laptop. If you think "it's more convenient!", that's only because you have done zero effort to make Prod-like ephemeral environments more convenient. > For development, engineers will build and run the docker image locally. In production, these are separate, you would build and publish the image separately in CI, and run only published images in prod. Again, makes sense. That is the antithesis of what Docker was created for. The point of containers was to say "it works on my machine", and use that exact same image in production. Not a different image in CI that nobody developed their code against. It's just going to result in bugs that the developer didn't see locally, so they will need to spend more time to get it working. Your laptop can do the same build CI can on the same branch and push the same image that CI can. What I'm saying is nothing new. Look into cloud-based development environments and the dozen different solutions made solely for rendering your local code in a remote K8s pod as soon as you write to a local file. Imagine a world where you don't waste your time testing something twice and fixing bugs twice.
zoomzoom 4y ago
Totally agree that in practice you often need several configurations for the same systems across different environment types. For example, you probably also need a different docker-compose for running in CI when compared to dev or a deployed environment. The same applies for k8s and terraform, and many others. Managing all these different configurations and the matrix of interactions between them is why we're trying to integrate the SDLC into one configuration at Coherence (withcoherence.com). [disclosure... I'm a cofounder]
strzibny 4y ago
It is. Convenience. I mean you replaced perhaps running staff locally that also didn't match production in the first place... I am now using this setup[0] for development, but it doesn't really match production 1:1. [0] https://nts.strzibny.name/hybrid-docker-compose-rails/ https://nts.strzibny.name/hybrid-docker-compose-rails/

tjoff 4y ago

That is an absurdly black-and-white view.

dijksterhuis 4y ago
I have lost count of the times I've heard "but it works fine on MY machine, it must be the CI/CD that's broken" I then find out that no-one bothered to run the compose file that matches what gets put into prod on their local machines. My point is: your view might become more black and white on this matter after the N-th "but it works fine on MY machine" comment. EDIT: Where N = your personal tolerance level of bullshit.
- remram 4y ago
  Sounds like this was easily caught by CI, which does run the prod compose file, and everything was fine.
  - dijksterhuis 4y ago
    This wasn't a real example. Just a generic thing that happens everywhere I go. There's always one in the crowd that refuses to give up local development.
    - remram 4y ago
      As long as you have CI or staging with the proper config, what does it matter how the devs write code?
      - 0xbadcafebee 4y ago
        The two aren't separate. There is a direct link between how devs write code and whether something works in production. And there is a ton of productivity loss and reliability loss when these two things are not in sync.
- tjoff 4y ago
  That might me your experience, I don't share it. But even so, how you can go from that to: > [...] there's really no point to using Docker Compose other than developing without internet access boggles my mind. EDIT: apologies, mixup on my end and quote is from someone else.
  - dijksterhuis 4y ago
    Not the parent! Made no comment on developing without internet access. EDIT: I usually try to build, test, inspect, push and deploy from the same compose file. You can deploy to ECS direct from compose using an AWS context: https://aws.amazon.com/blogs/containers/deploy-applications-on-amazon-ecs-using-docker-compose/ https://aws.amazon.com/blogs/containers/deploy-applications-...
deleted 4y ago
[deleted]

bananaoomarang 4y ago

Compose also works well with Swarm in my experience.

nicoburns 4y ago

Meanwhile here I am developing on macOS (no Docker or VMs involved) and deploying to linux. We do run tests on a linux CI runner, and we have a staging environment that's identical to prod. But really cross-platform support tends to be really good these days. I see no need for identical environments for development.

dijksterhuis 4y ago

> defeating the whole "it just runs everywhere" premise. Straight up 100% hard agree. > It goes without saying that Docker Compose only works on a single host, so that won't work for Prod if you need more than one host, but single-host-everything seems to be HN's current fetish) technically this is only correct for compose file versions up to and including 3.8. Part of the push with the new mainline compose V2 plugin has included the updated compose specification [0] which seems to have unify quite a lot of differences between the "compose" file for compose deployments and the "stack" file for swarm deployments. FYI You can switch over to the new version with the compose V2 plugin by remvoing any `version` key from your compose file. Although the `deploy: restart_policy:` values are currently not unified, much to my sadness. [0]: https://docs.docker.com/compose/compose-file/ https://docs.docker.com/compose/compose-file/

twblalock 4y ago

Also, don't use Docker Compose in dev if you use Kubernetes in prod. Just use the same thing everywhere. Otherwise the fact that something works in dev tells you nothing about whether it would work in prod.

0xbadcafebee 4y ago
^ 1000%
frenchman99 4y ago
Running a local k8s cluster doesn't guarantee that a remote cluster on some cloud provider will work the same either. I've heard of and worked with multiple dev teams that develop with compose and deploy a totally different way (k8s, AWS auto scaling groups, heroku, etc), to great success. Devs know about compose and understand it. As long as the containers behave as planned, the DevOps people can figure out how to deploy them on whatever production environment.
- twblalock 4y ago
  That's all fine, until the apps that ran fine on a laptop won't run in the real environments, and then the devops people need to loop in the devs to debug it. Quite a lot of the time, it will be a problem with the manifest not providing something the app container expects, like a permission or a port binding or a missing ConfigMap. Everyone loses productivity when that happens.

shroompasta 4y ago

This is a questionable comment >If you don't use the same Docker Compose file for Production, don't use it for Development. Your two different systems will diverge in behavior, leading you to troubleshoot two separate sets of problems, and testing being unreliable, defeating the whole "it just runs everywhere" premise. Anyone ever developing a react project on docker compose would never run a build step for development. In fact for most setups, I probably wouldn't even run react on docker-compose for development, and just use the dev server straight up on my machine. Furthermore, working with any python http servers is substantially easier when working with a dev server in development, rather than running your server through gunicorn or other wsgi servers - not to mention the ability to hot reload for development. I'm sure that there are many other cases where it's necessary and convenient to separate production and development docker compose files. I don't think it's fair to say that there is no point in having separate docker-compose files as the lion's share of dependencies that need to be consistent is inside each container, not on the docker-compose configuration level.

rdsubhas 4y ago

> ... if Dev and Prod run differently, there's really no point to ... "If dev and prod run differently" — Why is there an "if" here at all? Isn't it obvious to verify with two eyes in the real world? Is there any company in the world that runs production from somebody's laptop at home? Or distributes server racks for people to take home? It looks like an extremely disillusioned question out of touch with reality. 1. For development, engineers want to run a local throwaway mysql/redis. In production, engineer want to use a proper managed mysql/redis. The compose file, and envvars/links will be different. This is extremely normal, it makes sense. 2. Development laptops will always be underpowered than servers. The cpu/memory requirements will be different. Again makes sense. 3. For development, engineers will build and run the docker image locally. In production, these are separate, you would build and publish the image separately in CI, and run only published images in prod. Again, makes sense. These are all perfectly logical things that make practical sense. Trying to share/ reuse/ resemble is fine. But saying "dev===prod" is just a disillusion out of touch with reality. Edit: For context, yes I do docker for a living for last 10 years, since v0.4 (lxc) days. I've been to and presented in meetups. If someone says they're trying to get dev as close functionally to prod, by doing X, Y, Z and sharing practical tips — you know you can listen to them. If someone says "dev===prod" and when you ask how they talk abstract principles, you quietly get out of there.

0xbadcafebee 4y ago
> For development, engineers want to run a local throwaway mysql/redis. In production, engineer want to use a proper managed mysql/redis. Bad idea. You develop against a completely different local system, then you push to prod, and "oh no it doesn't work". Yeah - because you're developing against a completely different system! It behaves differently. It causes different bugs. You take a ton of time setting up both different systems differently, duplicating your effort, duplicating your bugs, having to fix things twice for no reason, and causing production issues when expectations from local dev don't match prod reality. Every experienced systems engineer has been repeating this for decades. Listen to them. They all say exactly. the. same. thing. "Make all your systems as identical as possible." That doesn't mean make all your systems different whenever it is more convenient for you. It means that if it's possible for you to be developing using the same tech you use in prod, you should be doing that, unless it is impossible. If you think it's not possible or practical, you simply haven't thought hard enough about it. The whole selling point of the cloud is that you can spin up environments and spin them down in minutes and pay pennies for it. Not a shared environment where everyone's work is constantly conflicting, but dedicated, ephemeral environments that actually match production. Literally the only example I know of where this doesn't work is with physical devices where you have a lab of 10 experimental pieces of test gear and you have to reserve them to test code on them. That can make remote development difficult. Still not impossible though. > The cpu/memory requirements will be different. Unless you're developing using the same servers as prod. There is no reason that you have to use your laptop. If you think "it's more convenient!", that's only because you have done zero effort to make Prod-like ephemeral environments more convenient. > For development, engineers will build and run the docker image locally. In production, these are separate, you would build and publish the image separately in CI, and run only published images in prod. Again, makes sense. That is the antithesis of what Docker was created for. The point of containers was to say "it works on my machine", and use that exact same image in production. Not a different image in CI that nobody developed their code against. It's just going to result in bugs that the developer didn't see locally, so they will need to spend more time to get it working. Your laptop can do the same build CI can on the same branch and push the same image that CI can. What I'm saying is nothing new. Look into cloud-based development environments and the dozen different solutions made solely for rendering your local code in a remote K8s pod as soon as you write to a local file. Imagine a world where you don't waste your time testing something twice and fixing bugs twice.

zoomzoom 4y ago

Totally agree that in practice you often need several configurations for the same systems across different environment types. For example, you probably also need a different docker-compose for running in CI when compared to dev or a deployed environment. The same applies for k8s and terraform, and many others. Managing all these different configurations and the matrix of interactions between them is why we're trying to integrate the SDLC into one configuration at Coherence (withcoherence.com). [disclosure... I'm a cofounder]

strzibny 4y ago

It is. Convenience. I mean you replaced perhaps running staff locally that also didn't match production in the first place... I am now using this setup[0] for development, but it doesn't really match production 1:1. [0] https://nts.strzibny.name/hybrid-docker-compose-rails/ https://nts.strzibny.name/hybrid-docker-compose-rails/