5 ms·

I'm not a devops person at all, but this pile of abstractions seems like it's going the wrong way. Like, this seems almost like a parody? You have this problem

I'm not a devops person at all, but this pile of abstractions seems like it's going the wrong way. Like, this seems almost like a parody? You have this problem where you want to set up infrastructure in an abstract way, so you come up with a way to write some infrastructure requirements and now you have a program that does all the actions necessary to take your setup from wherever it is, to what you specified. Then you have multiple of those systems, and it gets confusing. So you come up with a generic way to specify the abstractions that can be sort of compiled into any of the flavors, so that you can set up infrastructure using any of the sub-abstractions available, instead of only a specific one. This has the smell of a FactoryFactoryFactory sort of thing. Like, if this whole cottage industry were my architecture project I would be saying to myself "This is too much, a rickety, foot-gun-laden tower, destined to crumble. I've obviously chosen a poor abstraction, lemme back way the hell up." Am I just totally wrong here? I feel like I'm taking crazy pills.

radus 7y ago
Maybe you have a point but from my perspective you're overthinking it. Lots of developers use Docker Compose and will continue to do so because it's a great product. The news here is that there is now an open specification that others can contribute to.
carlisle_ 7y ago
I think you generally have the right sentiment, but some of these solutions are born out of necessity. Using containers at scale is really hard when you're trying to solve for optimal resource usage and resilience. These abstractions may seem crazy but other solutions are probably much crazier. The other part of this problem that isn't applicable generally to other things, is this is a fairly unique problem to cgroups. By design cgroups don't have a singular implementation or solution, so there are a lot of "competing" solutions. I think some amount of unix philosphy is at play here, to do one thing (or really, fewer) and that's why it seems insane how many systems you have to hook up together to make the thing work the way it should. I'm fairly devops centric, but I'd still take this with a grain of salt b/c containers aren't my area of expertise.
jayd16 7y ago
>I'm not a devops person at all You should try it before you knock it. There are a lot of flavors but the abstractions of any single flavor isn't that deep. Containers themselves aren't even an abstraction, they're just namespaced apps.
jaredklewis 7y ago
Sometimes the proof is in the pudding. I’ve had similarly adverse reactions to the seemingly “unnecessary” complexity of many things: docker, webpack, autotools, react, AWS, just to name a few. But I’ve found that, upon closer inspection, the software is usually complicated because the problem domain is complicated. Further, as an inviolable rule, the most popular tools for any problem are the ones that don’t make you rewrite all your code; popular tools build on top of the mess we already have. This can give the illusion that the tool is somehow responsible for all the preexisting complexity. Tools which attempt a “clean slate” approach are mostly doomed for obscurity. In the case of docker-compose, I can say after years of resistance, I tried docker compose and was an immediate convert. Setting up, documenting, and sharing development environments has never been simpler.
- mumblemumble 7y ago
  I would add a caveat: Docker compose is nice for setting up development environments as long as everyone's developing on the same platform. I work in a shop where various developers work on Linux, OS X and Windows (useful for dogfooding - we support deploying to multiple platforms), and, there, Docker Compose has ironically turned out to be our chief source of cross-platform headaches, due to annoying differences in how it works on each host platform. Particularly with the networking subsystem.
  - williamdclt 7y ago
    > Docker Compose has ironically turned out to be our chief source of cross-platform headaches Which is the whole point of it. If it's not, it's just a bad abstraction. Docker-compose itself becomes the only variable
    - mumblemumble 7y ago
      Perhaps I should rephrase: Since moving to Docker, we now have more cross-platform problems than we used to. We continue to use it anyway, because there are some other benefits that we feel outweigh shortcomings like these. But still, it's annoying.
jjoonathan 7y ago
Docker is how you put your widgets into a container, docker-compose is how you put a few containers on a ship. How is separating those responsibilities overengineering?
baq 7y ago
it is but it solves the distribution problem. golang aimed to solve the same problem by compiling to a single statically linked binary and it was one of its biggest selling points. i feel docker blunted this part considerably, but it also benefited it nevertheless: development environments, shipping runable packages with assets, etc. is easier when you literally have to only do docker run whatever.
hinkley 7y ago
Everyone is so obsessed with never repeating themselves that they've forgotten to ask whether they are still removing duplicated effort instead of just duplicated text. Every tech cycle we get people trying to compress the text long after the power of the system has already been maximized. Particularly, as a species, we discount the future heavily, so as developers we do not accurately estimate the variance introduced by something that works by magic until it doesn't. So on some unnamed day in the future when you had other plans, you will be completely blocked while having to learn a bunch of things we were avoiding learning in the first place.
avip 7y ago
> Am I just totally wrong here? Yes. I think so. docker-compose is just a tool to run multiple containers together. It is extremely simple, readable and straightforward (if you're familiar with docker). It is not an inf. abstraction at all.
- stevebmark 7y ago
  It's not extremely simple. Do you know when to use v2 vs v3?
  - williamdclt 7y ago
    Looking into it for a minute I don't see any particular gotcha. v3 is a breaking change compatible with Docker Swarm, so it's just like any versionned tool: if you're already on v2 and don't care about the new feature stay there, if you're on a greenfield project or do care about new feature use v3. Am I missing something?
    - deleted 7y ago
      [deleted]
  - avip 7y ago
    It's like asking when to use python2. Use v2 if you have no choice. If you're asking, apparently you have the choice, so there's your answer.
    - stevebmark 7y ago
      Nope, entirely wrong! v3 isn't an upgrade, it has features specific to orchestration. v3 is a fork, not an upgrade. v2 is better for local development, it has things like depends_on which v3 doesn't have.
      - avip 7y ago
        depends_on was removed because it was an inconsistent and problematic directive. You can abuse links instead.
lookdangerous 7y ago
It is just a matter of time. Right now, we’re in the thick of things and as pointed out by others, the problem domain is very complex. As we move forward, though, our solutions will become simpler and our container tools will work more elegantly.
csomar 7y ago
You're not wrong. What you are missing is how much complicated web development (and software development in general has become). Even if you are doing a simple Static website it's more complicated than yesterday. If you are using Git to push your changes, you already need to push to Github and then SSH into your machine, make a pull request. Oups. Now Nginx is still serving the old version through cache. So need to clear cache, restart, shut this, reconfigure this, etc... After a while this gets tiring. Docker is not the perfect solution (one can wonder if it's actually a solution) but it's a symptom of the real issue rather than the issue itself.

radus 7y ago

Maybe you have a point but from my perspective you're overthinking it. Lots of developers use Docker Compose and will continue to do so because it's a great product. The news here is that there is now an open specification that others can contribute to.

carlisle_ 7y ago

I think you generally have the right sentiment, but some of these solutions are born out of necessity. Using containers at scale is really hard when you're trying to solve for optimal resource usage and resilience. These abstractions may seem crazy but other solutions are probably much crazier. The other part of this problem that isn't applicable generally to other things, is this is a fairly unique problem to cgroups. By design cgroups don't have a singular implementation or solution, so there are a lot of "competing" solutions. I think some amount of unix philosphy is at play here, to do one thing (or really, fewer) and that's why it seems insane how many systems you have to hook up together to make the thing work the way it should. I'm fairly devops centric, but I'd still take this with a grain of salt b/c containers aren't my area of expertise.

jayd16 7y ago

>I'm not a devops person at all You should try it before you knock it. There are a lot of flavors but the abstractions of any single flavor isn't that deep. Containers themselves aren't even an abstraction, they're just namespaced apps.

jaredklewis 7y ago

Sometimes the proof is in the pudding. I’ve had similarly adverse reactions to the seemingly “unnecessary” complexity of many things: docker, webpack, autotools, react, AWS, just to name a few. But I’ve found that, upon closer inspection, the software is usually complicated because the problem domain is complicated. Further, as an inviolable rule, the most popular tools for any problem are the ones that don’t make you rewrite all your code; popular tools build on top of the mess we already have. This can give the illusion that the tool is somehow responsible for all the preexisting complexity. Tools which attempt a “clean slate” approach are mostly doomed for obscurity. In the case of docker-compose, I can say after years of resistance, I tried docker compose and was an immediate convert. Setting up, documenting, and sharing development environments has never been simpler.

mumblemumble 7y ago
I would add a caveat: Docker compose is nice for setting up development environments as long as everyone's developing on the same platform. I work in a shop where various developers work on Linux, OS X and Windows (useful for dogfooding - we support deploying to multiple platforms), and, there, Docker Compose has ironically turned out to be our chief source of cross-platform headaches, due to annoying differences in how it works on each host platform. Particularly with the networking subsystem.
- williamdclt 7y ago
  > Docker Compose has ironically turned out to be our chief source of cross-platform headaches Which is the whole point of it. If it's not, it's just a bad abstraction. Docker-compose itself becomes the only variable
  - mumblemumble 7y ago
    Perhaps I should rephrase: Since moving to Docker, we now have more cross-platform problems than we used to. We continue to use it anyway, because there are some other benefits that we feel outweigh shortcomings like these. But still, it's annoying.

jjoonathan 7y ago

Docker is how you put your widgets into a container, docker-compose is how you put a few containers on a ship. How is separating those responsibilities overengineering?

baq 7y ago

it is but it solves the distribution problem. golang aimed to solve the same problem by compiling to a single statically linked binary and it was one of its biggest selling points. i feel docker blunted this part considerably, but it also benefited it nevertheless: development environments, shipping runable packages with assets, etc. is easier when you literally have to only do docker run whatever.

hinkley 7y ago

Everyone is so obsessed with never repeating themselves that they've forgotten to ask whether they are still removing duplicated effort instead of just duplicated text. Every tech cycle we get people trying to compress the text long after the power of the system has already been maximized. Particularly, as a species, we discount the future heavily, so as developers we do not accurately estimate the variance introduced by something that works by magic until it doesn't. So on some unnamed day in the future when you had other plans, you will be completely blocked while having to learn a bunch of things we were avoiding learning in the first place.

avip 7y ago

> Am I just totally wrong here? Yes. I think so. docker-compose is just a tool to run multiple containers together. It is extremely simple, readable and straightforward (if you're familiar with docker). It is not an inf. abstraction at all.

stevebmark 7y ago
It's not extremely simple. Do you know when to use v2 vs v3?
- williamdclt 7y ago
  Looking into it for a minute I don't see any particular gotcha. v3 is a breaking change compatible with Docker Swarm, so it's just like any versionned tool: if you're already on v2 and don't care about the new feature stay there, if you're on a greenfield project or do care about new feature use v3. Am I missing something?
  - deleted 7y ago
    [deleted]
- avip 7y ago
  It's like asking when to use python2. Use v2 if you have no choice. If you're asking, apparently you have the choice, so there's your answer.
  - stevebmark 7y ago
    Nope, entirely wrong! v3 isn't an upgrade, it has features specific to orchestration. v3 is a fork, not an upgrade. v2 is better for local development, it has things like depends_on which v3 doesn't have.
    - avip 7y ago
      depends_on was removed because it was an inconsistent and problematic directive. You can abuse links instead.

lookdangerous 7y ago

It is just a matter of time. Right now, we’re in the thick of things and as pointed out by others, the problem domain is very complex. As we move forward, though, our solutions will become simpler and our container tools will work more elegantly.

csomar 7y ago

You're not wrong. What you are missing is how much complicated web development (and software development in general has become). Even if you are doing a simple Static website it's more complicated than yesterday. If you are using Git to push your changes, you already need to push to Github and then SSH into your machine, make a pull request. Oups. Now Nginx is still serving the old version through cache. So need to clear cache, restart, shut this, reconfigure this, etc... After a while this gets tiring. Docker is not the perfect solution (one can wonder if it's actually a solution) but it's a symptom of the real issue rather than the issue itself.