7 ms·
I’ve worked with a few developers that have been adamant about developing inside of containers. What I’ve noticed: Terrible performance. One of my engineers re
by mailslot 6y ago
I’ve worked with a few developers that have been adamant about developing inside of containers. What I’ve noticed:
Terrible performance. One of my engineers recently thought that 150ms was terrific for a HTTP request. Break out of the container and it was <10ms. YMMV.
Fragile everything: Because one expects a “pristine” environment, often any slight change causes the entire stack to fall apart. This doesn’t happen at the start, but creeps in over time, until you can’t even update base images. I’ve seen it a lot. It ends up only adding an additional layer of complication.
Etc.
There are definitely reasons to do this... But when a pedantic developer that needs everything to be “just right” does it, it often becomes a disaster, leading to shortcuts and a lack of adaptability.
There’s also the developer that has no idea WTF is going on. They use a standard Rails/PHP/NodeJS/etc container and don’t understand how it works. Sometimes, they don’t even know that their system can run their stack natively. I’ve been on teams that have said “Let’s just use Docker because X doesn’t know how to install Y.”
Docker is fantastic for many things, but let’s stop throwing it at everything.
- paloaltokid 6y agoThis feels like very much a YMMV situation. I think my own personal thinking is mostly the same as yours. But for the OP it may be perfect. On the blog he indicates that he's a CS professor. I could imagine that in a research environment maybe he gets better mileage out of this than someone coding in a for-money work environment.
- throwaway_pdp09 6y agoDan Lemire is a prof but also a very hands-dirty type. He measures stuff down to CPU IPC levels. Check other stuff on his blog, he's not an ivory tower type[0] [0] Not that there's anything wrong with that at all, it's just a different kind of person with different strengths, but DL is not one.
- paloaltokid 6y agoAh, that makes sense. It sounds like he balances being academic with trying to work through things in a real-world way.
- mrec 6y agoOh, I thought the name looked familiar. I've used his JavaFastPFOR library; very performant and a pleasure to work with.
- jeffbee 6y agoBoth 10ms and 150ms are bad results for trivial HTTP requests to localhost.
- als0 6y ago> often any slight change causes the entire stack to fall apart Arguably this is one reason why containers are popular in the first place. Devs don't want to spend time dealing with dirty environments.
- TheChaplain 6y agoDevs want to spend even less time with fragile environments that breaks from the slightest change.
- rumanator 6y agoThankfully containers are the exact opposite then.
- mailslot 6y agoWhat’s a dirty environment? One that a dev doesn’t clean up? Makes a mess like a filthy hoarder?
- infogulch 6y ago> often any slight change causes the entire stack to fall apart Yes, but this is true generally; it's not specific to containers. Any dev environment naturally tends disorder with unsynchronized versions, implicit dependencies, platform-specific quirks, etc. It takes an effort to keep chaos at bay. At least with containers you have a chance of fully capturing the complete list of dev dependencies & installed software. I'm interested in how CodeSpaces/Coder.com solves these issues.
- dahfizz 6y agoSure, keeping a stack clean is always difficult. But I think OPs point was that programming in a container encourages a more fragile setup. On a native setup, you get a feel for the fact that X config file might be in different places, or that Y lib is more robust and more widely available than lib Z. You end up with a more robust application because you have been "testing" it on a wide range of systems from day one.
- rumanator 6y ago> But I think OPs point was that programming in a container encourages a more fragile setup. I don't see how that point can be argued at all, particularly if the project is expected to be deployed with Docker.
- dahfizz 6y agoI just argued that point. When developing inside docker, you are fooled into thinking that various things about your environment are constants. When it comes time to update your base image, all these constants change, and your application breaks.
- jfim 6y agoIs that a problem in practice though? Updating libraries or the base image that one's code depends on always has the risk of breaking from API changes or regressions, and in a container, at least it's easy to reproduce the issue.
- jayd16 6y agoHow do you maintain build machine environments if not with docker? The CI script updates all the tools or something?
- nemetroid 6y agoThere's no conflict between using Docker for CI (and deployment) and not using Docker for development.
- jayd16 6y agoOh ok, so they still build and maintain a (fragile?) docker image with up to date build tooling for CI machines to use?
- nemetroid 6y agoI cannot speak for the OP, but this is more or less how we use Docker at my workplace. > (fragile?) The Docker image isn't fragile, it's your software that risks becoming fragile if it's too strongly reliant on a specific environment.
- jayd16 6y agoOh I see what they mean now. Lack of varied developer preference leads to things like hard coded paths instead of configuration files, for example.
- mailslot 6y agoIt’s been rare, but this has definitely been a problem: “Why do I need an ENV var, the path is always /app?” And then not supporting symlinks... ugh.
- blondin 6y agocan i also add that containers were not really built with multiple services or apps running inside them in mind? there are workarounds but, honestly, they are not worth it. also, author seems to be using ubuntu. i wonder if he has considered multipass? https://multipass.run https://multipass.run
- casept 6y agoDocker containers were not meant to be used that way. LXD containers on the other hand are excellent for running multiple services.
- throwaway888abc 6y ago>>Terrible performance. One of my engineers recently thought that 150ms was terrific for a HTTP request. Break out of the container and it was <10ms. YMMV. Try Linux and all those lags,spikes,inconsistencies are magically gone.
- mailslot 6y agoEh. Worked for a big CDN. Using containers would have decimated profits. There IS overhead. At scale, you feel it. But for dev? Yeah. Linux on Linux is less impactful if you’re building something simple like a blog.
- cjvirtucio 6y agoAgreed.. there's ways to have a "pristine" environment on the host machine, anyway. Our team uses ansible.
- mailslot 6y agoI prefer chef locally, but that totally works!
- alexgartrell 6y agoMaybe you're talking about non-native containers (i.e. not Linux), but there's no technical merit to the idea that a container by itself could introduce 15x latency on a Linux host for something like a web request, unless something like network namespaces, tc, etc was being used very improperly. You also point to a lot of problems that are container-independent and lay them at the feet of docker, which is unfair. Upgrading the OS is always hard unless you have some awesome, declarative config and you managed to depend on zero of the features that have changed. It doesn't matter if you're in a container or not, switching from iptables in Centos 7 to nftables in Centos 8 is going to introduce some pain. And somehow we get mad at people for not knowing how to install things, but the complexity of installing them is itself a problem. More steps means more inconsistency, which means it's more likely that "it works on my machine, but breaks on yours."
- cortesoft 6y agoThey were probably running docker on Mac
- rumanator 6y ago> Terrible performance. One of my engineers recently thought that 150ms was terrific for a HTTP request. Break out of the container and it was <10ms. YMMV. Did you happened to develop with Macs? Because Docker for Mac has a known network performance issue. https://github.com/docker/for-mac/issues/3497 https://github.com/docker/for-mac/issues/3497
- mailslot 6y agoYep. macOS for what I mentioned :) ... but latency is still an issue regardless. It’s why there’s a premium to go bare metal with cloud providers.
- cyphar 6y ago> Terrible performance. One of my engineers recently thought that 150ms was terrific for a HTTP request. Break out of the container and it was <10ms. YMMV. If you're not using Linux (presumably you're using MacOS), your "containers" are actually VMs so it's unsurprising that the performance suffers somewhat (not to mention that file accesses are especially slow with the Docker-on-Linux setup). The performance impact of being inside a container on Linux is as close to zero as you can get.
- ajcodez 6y agoWorking with 150ms handicap is not necessarily a bad thing for web developers.
- m463 6y agoNone of these problems are really a result of containers. People now call native installs "bare metal" installs. I've developer for linux on the machine, and in a container. Once you get the container mentality and start writing dockerfiles it creates a pretty predictable organized haven.
- nickjj 6y ago> Terrible performance. One of my engineers recently thought that 150ms was terrific for a HTTP request. Break out of the container and it was <10ms. YMMV. It's not that bad for everyone. For example on my Windows dev box, I have HTTP endpoints in volume mounted Flask and Phoenix applications that respond in microsends (ie. less than 1 millisecond). This is on 6 year old hardware and the source code isn't even mounted from an SSD (although Docker Desktop is installed on an SSD). On Linux, I have not noticed any runtime differences in speed, except that starting a container with Docker takes quite a bit longer than starting the same process without Docker. Apparently there's a regression: https://github.com/moby/moby/issues/38077 https://github.com/moby/moby/issues/38077
- zarkov99 6y agoDocker is fantastic at precisely this use case: capturing tool and build dependencies in a reproducible way. I am not sure what performance issues you are complaining about. We run high speed trading services with single digit microsecond latency on docker just fine.