7 ms·

In my opinion development environments should be as close to production as humanly possible. Configuration of services, network policies, build systems that are

In my opinion development environments should be as close to production as humanly possible. Configuration of services, network policies, build systems that are used in CI, etc. The closer you get to sharing tooling and configuration between your development and prod environments the better. Does it make some things more difficult as this author says? At first, yes. As time moves forward you'll end up with many nock on effects that you can't get any other way. 1. Developers can now help you debug that one issue in prod that no one can figure out. 2. Deployment is as easy as adding a line to a config (load dev yaml + patch) 3. Good devs will start automating the boring stuff. Good operators will look at, learn to use, and love these tools. 4. Developers will make sure containers and roll out systems work lightening fast. 5. "We need to run this migration in production. Tested it locally" actually means something now. 6. Demos can be done locally with absolutely no network connection. Demoing to the security minded? Worried about showing off your, often down, production env? No problem! Bring a dev laptop. 7. Developers can test QPS quotas, build dashboards for metric collection, benchmark performance, etc I think it's odd to get hung up on prefixing your commands with "docker-compose exec" or putting "docker-compose up -d --build" under watchman/inotify or investing in setting up minikube + (tilt | skaffold). Sure it's more stuff for your developers to know but it's very powerful to see your change as it effects the entire living system.

yahyaheee 7y ago
Amen, like you mention there are a couple of hurdles to get around but the reproducibility far outweighs that cost. Before docker trying to get virtually anything running on a variety of machines was a total nightmare
- alfiedotwtf 7y ago
  This. Every company I’ve ever worked for, to setup a dev environment and get the product passing test suites has been days to weeks. And without the automation and standard operating environment, every release was filled with missing dependencies because developers only tested on their local precious system. A side benefit to moving to containers is not contaminating your own machine with libraries or even breaking it because of versioning or config issues. Docker gets developers and production machines on the same page. ... I would never go back to bare metal development
  - nitrogen 7y ago
    Another downside of the containers-for-dev approach is that your company's stack might be too big to fit in a 16GB laptop when run as containers in a VM with dedicated memory.
    - alfiedotwtf 7y ago
      Sure, but do you need production data for development? Most people would just synthesizer what’s needed to run
      - nitrogen 7y ago
        I mean the code -- having to spin up 30 JVMs for a medium sized org's working stack takes a while and brings a MacBook Pro to its knees. This could probably be improved, if it were deemed sufficiently organizationally important.
      - winrid 7y ago
        Well if your environment is 100 services and each one takes 512mb of ram (that's being generous w/ what I've seen for the typical Spring Java app) at idle...
  - speedplane 7y ago
    > Docker gets developers and production machines on the same page. ... I would never go back to bare metal development Instead of reading this article as "docker containers are bad" (wrong), try reading it as "there are problems with docker containers that can be improved" (right). Containers fix a large number of issues, and many of these fixes directly benefit developers. But the article is right to point out inefficiencies that docker introduces in developer workflows. Docker may be taking developers and ops three steps forward, but for some, it's also one step backwards.
    - alfiedotwtf 7y ago
      Sure there may be some issues, but as you say, we're moving two steps forward :)
geofft 7y ago
The problem we have with that at my workplace is that it's hard to do things like run IDEs/set breakpoints, run gdb or strace, get a REPL (with useful REPL tooling), etc. inside a production-style container. I understand the arguments that you don't want to do this in actual production, you want to kill the offending container and figure out the problem from logs - but once you've got an idea of where the problem lies, for local dev you do actually want to debug problems this way. Even editing a random config file or editing source code to add a printf is more difficult inside a production-style container than in a local checkout. Have you found good approaches for this?
- crooked-v 7y ago
  For me, the other big complication is that we do backend stuff in Java... and God help you if you want a self-contained image that can intelligently and quickly reload Java code for anything that's not a very specific arrangement of frameworks. It's night and day compared to the Javascript world, where it's nearly a given that the bare minimum should be watchers that recompile code and restart a server before you finish switching window focus.
  - geofft 7y ago
    Yeah, I wonder to what extent "Just put it all in Docker" works for specific language communities, and if you're not in those communities the advice falls flat. (My employer is pretty polyglot, but primarily Java; my team is mostly Python; we also build our own C code as well as third-party C code and need to debug it every so often.)
    - crdrost 7y ago
      I mean, what Docker does is, it turns a network of executables and scripts into a single executable with no outside dependencies. It works extremely well for language communities that need this, in particular it caught on because you would build something and test it locally and it would work just fine, but then you are running the wrong version of Python or pip installed the wrong version of a library, or you have the wrong version of PHP or Node.js relative to Prod. For Java and Go, I believe they have already settled these problems, no? You might shell out to other commands and have a weak dependency on your runtime environment that way, but if you stay within your lane, then you already have an executable with all of its dependencies bundled in. They just have a lot less to gain from Docker. I can't see why they would have more to lose.
debaserab2 7y ago
There's nothing particularly unique about docker that allows you to achieve this - many of us have been doing this for over a decade with a plain ol' VM. Which docker is using anyways if you're on windows or mac. I've found that given the many docker specific issues I've ran into in development, the "matches your production" argument doesn't always bear completely true unless you're also using containers in production.
hrktb 7y ago
If your goal is detecting earlier configuration issues or limitations induced by the prod environment, it can be enough to have well maintained integration/test environments. And I guess it’s basically the situation the author ended up with. You can have these running on the same stack as your prod (for instance in a similar kubernetes cluster, or at least on a mirrored config by the same provider if you’re on plain VMs), getting that much closer to test/prod parity. Most companies can afford one virtual env per dev, so the remote test environment becomes an extension of their local one. Demoing features offline should still be OK as well, you need everything setup locally for dev anyway. If the goal is to get people used to what’s running in production and train them to access the containers and debug them remotely, it’s another beast altogether, and reminds me of all the discussion about learning vim. It brings tremendous value to be proficient in coding in CLI, and not just for simple edits, but to actually be able to use it as the main coding tool. But then is it an efficient time investment for most people ? It’s pretty close to living your life in dim light because one day you’ll have to deal with a blackout and you’ll be super efficient then. It’s good to have a few people doing that, I’m not convinced it’s good for a whole team.
speedplane 7y ago
> Does [Docker] make some things more difficult as this author says? At first, yes. As time moves forward you'll end up with many nock on effects that you can't get any other way. I agree that docker provides many efficiency improvements, especially with respect to testing, reproducing bugs, and deployment. But what about pure development of new features? In a bare-metal development environment, you just compile and run or launch a debugger. With docker, you need to compile, build a docker image, run the docker image, then launch your app/debugger. Would love to hear solutions on how to eliminate/ameliorate these additional steps.
- hrktb 7y ago
  You can mount your working source folder in the running container. If you are working with a scripting language that can be enough to have your changes reflected on the running instance, otherwise you can set a hook on these changes to recompile and/or redeploy the new code automatically.
ClikeX 7y ago
> I think it's odd to get hung up on prefixing your commands with "docker-compose exec" or ... We're already doing "bundle exec rails s" or "bundle exec capistrano production deploy" commands. So a "docker-compose up" shouldn't matter at all. To be fair, I've had frontenders complain that it was hard for them to remember the capistrano deploy commands. And that they'd love a GUI butten for deployments. Rather than just look at our wiki or write it down.
- pas 7y ago
  It's not like providing a GUI is impossible, a single file python server outputting HTML with buttons would do it. :)
  - ClikeX 7y ago
    It's not impossible. But we're a small dev team with several active clients. So most of us are swamped with client work. For the few frontend devs that need to deploy the applications it's way easier for them to just save the command to their notes.
    - pas 7y ago
      Hm. Actual deploys? How about a GitLab CI pipeline with "when: manual" tasks for PROD/LIVE deploy? But, yes, putting the how to deploy into the README.md can be done in 5 seconds whereas setting up a CI/CD pipeline is a lot more effort. So I guess frontend devs just have to be tamed :)

yahyaheee 7y ago

Amen, like you mention there are a couple of hurdles to get around but the reproducibility far outweighs that cost. Before docker trying to get virtually anything running on a variety of machines was a total nightmare

alfiedotwtf 7y ago
This. Every company I’ve ever worked for, to setup a dev environment and get the product passing test suites has been days to weeks. And without the automation and standard operating environment, every release was filled with missing dependencies because developers only tested on their local precious system. A side benefit to moving to containers is not contaminating your own machine with libraries or even breaking it because of versioning or config issues. Docker gets developers and production machines on the same page. ... I would never go back to bare metal development
- nitrogen 7y ago
  Another downside of the containers-for-dev approach is that your company's stack might be too big to fit in a 16GB laptop when run as containers in a VM with dedicated memory.
  - alfiedotwtf 7y ago
    Sure, but do you need production data for development? Most people would just synthesizer what’s needed to run
    - nitrogen 7y ago
      I mean the code -- having to spin up 30 JVMs for a medium sized org's working stack takes a while and brings a MacBook Pro to its knees. This could probably be improved, if it were deemed sufficiently organizationally important.
    - winrid 7y ago
      Well if your environment is 100 services and each one takes 512mb of ram (that's being generous w/ what I've seen for the typical Spring Java app) at idle...
- speedplane 7y ago
  > Docker gets developers and production machines on the same page. ... I would never go back to bare metal development Instead of reading this article as "docker containers are bad" (wrong), try reading it as "there are problems with docker containers that can be improved" (right). Containers fix a large number of issues, and many of these fixes directly benefit developers. But the article is right to point out inefficiencies that docker introduces in developer workflows. Docker may be taking developers and ops three steps forward, but for some, it's also one step backwards.
  - alfiedotwtf 7y ago
    Sure there may be some issues, but as you say, we're moving two steps forward :)

geofft 7y ago

The problem we have with that at my workplace is that it's hard to do things like run IDEs/set breakpoints, run gdb or strace, get a REPL (with useful REPL tooling), etc. inside a production-style container. I understand the arguments that you don't want to do this in actual production, you want to kill the offending container and figure out the problem from logs - but once you've got an idea of where the problem lies, for local dev you do actually want to debug problems this way. Even editing a random config file or editing source code to add a printf is more difficult inside a production-style container than in a local checkout. Have you found good approaches for this?

crooked-v 7y ago
For me, the other big complication is that we do backend stuff in Java... and God help you if you want a self-contained image that can intelligently and quickly reload Java code for anything that's not a very specific arrangement of frameworks. It's night and day compared to the Javascript world, where it's nearly a given that the bare minimum should be watchers that recompile code and restart a server before you finish switching window focus.
- geofft 7y ago
  Yeah, I wonder to what extent "Just put it all in Docker" works for specific language communities, and if you're not in those communities the advice falls flat. (My employer is pretty polyglot, but primarily Java; my team is mostly Python; we also build our own C code as well as third-party C code and need to debug it every so often.)
  - crdrost 7y ago
    I mean, what Docker does is, it turns a network of executables and scripts into a single executable with no outside dependencies. It works extremely well for language communities that need this, in particular it caught on because you would build something and test it locally and it would work just fine, but then you are running the wrong version of Python or pip installed the wrong version of a library, or you have the wrong version of PHP or Node.js relative to Prod. For Java and Go, I believe they have already settled these problems, no? You might shell out to other commands and have a weak dependency on your runtime environment that way, but if you stay within your lane, then you already have an executable with all of its dependencies bundled in. They just have a lot less to gain from Docker. I can't see why they would have more to lose.

debaserab2 7y ago

There's nothing particularly unique about docker that allows you to achieve this - many of us have been doing this for over a decade with a plain ol' VM. Which docker is using anyways if you're on windows or mac. I've found that given the many docker specific issues I've ran into in development, the "matches your production" argument doesn't always bear completely true unless you're also using containers in production.

hrktb 7y ago

If your goal is detecting earlier configuration issues or limitations induced by the prod environment, it can be enough to have well maintained integration/test environments. And I guess it’s basically the situation the author ended up with. You can have these running on the same stack as your prod (for instance in a similar kubernetes cluster, or at least on a mirrored config by the same provider if you’re on plain VMs), getting that much closer to test/prod parity. Most companies can afford one virtual env per dev, so the remote test environment becomes an extension of their local one. Demoing features offline should still be OK as well, you need everything setup locally for dev anyway. If the goal is to get people used to what’s running in production and train them to access the containers and debug them remotely, it’s another beast altogether, and reminds me of all the discussion about learning vim. It brings tremendous value to be proficient in coding in CLI, and not just for simple edits, but to actually be able to use it as the main coding tool. But then is it an efficient time investment for most people ? It’s pretty close to living your life in dim light because one day you’ll have to deal with a blackout and you’ll be super efficient then. It’s good to have a few people doing that, I’m not convinced it’s good for a whole team.

speedplane 7y ago

> Does [Docker] make some things more difficult as this author says? At first, yes. As time moves forward you'll end up with many nock on effects that you can't get any other way. I agree that docker provides many efficiency improvements, especially with respect to testing, reproducing bugs, and deployment. But what about pure development of new features? In a bare-metal development environment, you just compile and run or launch a debugger. With docker, you need to compile, build a docker image, run the docker image, then launch your app/debugger. Would love to hear solutions on how to eliminate/ameliorate these additional steps.

hrktb 7y ago
You can mount your working source folder in the running container. If you are working with a scripting language that can be enough to have your changes reflected on the running instance, otherwise you can set a hook on these changes to recompile and/or redeploy the new code automatically.

ClikeX 7y ago

> I think it's odd to get hung up on prefixing your commands with "docker-compose exec" or ... We're already doing "bundle exec rails s" or "bundle exec capistrano production deploy" commands. So a "docker-compose up" shouldn't matter at all. To be fair, I've had frontenders complain that it was hard for them to remember the capistrano deploy commands. And that they'd love a GUI butten for deployments. Rather than just look at our wiki or write it down.

pas 7y ago
It's not like providing a GUI is impossible, a single file python server outputting HTML with buttons would do it. :)
- ClikeX 7y ago
  It's not impossible. But we're a small dev team with several active clients. So most of us are swamped with client work. For the few frontend devs that need to deploy the applications it's way easier for them to just save the command to their notes.
  - pas 7y ago
    Hm. Actual deploys? How about a GitLab CI pipeline with "when: manual" tasks for PROD/LIVE deploy? But, yes, putting the how to deploy into the README.md can be done in 5 seconds whereas setting up a CI/CD pipeline is a lot more effort. So I guess frontend devs just have to be tamed :)