8 ms·

I'm very happy to see you better support for docker compose. I think about 50% of the time I find Kubernetes used in production/development when it could have j

by jadbox 3y ago

I'm very happy to see you better support for docker compose. I think about 50% of the time I find Kubernetes used in production/development when it could have just used Compose along with a simple Terraform or Pulumi deploy script.

Already__Taken 3y ago
ah the 1% of the time you do want to use something in kubernetes and you have a pile of compose is pretty demoralising though
- cpitman 3y ago
  I haven't run into the need to do that, but there is the Kompose project that exists to help with the conversion (https://kompose.io/ https://kompose.io/)!
nonameiguess 3y ago
I guess they stopped doing this and now consider the product mature from looking at their website, but Docker Compose used to tell you that it wasn't suitable or intended for production use. That seemed like a pretty good reason not to do it.
- geerlingguy 3y ago
  Compose has its own footguns, but I've run a number of services with just Compose on a VM or direct on bare metal and the few little issues I've had were easy to debug and fix. Very much unlike Kubernetes, where even a small deployment can sometimes be maddening to debug. Great for larger deployments where complexity is basically guaranteed, but anything that could run on one server with a hot spare is a great fit for Compose!
  - erulabs 3y ago
    Surprised to hear you think debugging in kubernetes is maddening. It’s absolutely very different, but if I had to debug a system I knew nothing about, I’d rather debug a kubernetes based system rather than any other. Standards and all that. Anyways - for a single container or two obviously you’re right - I’m just not sure that’s so common outside of a side project anymore.
    - derefr 3y ago
      Especially because "production" implies an ingress controller, monitoring, etc. Does a Docker Compose-based production setup contain run its own Nginx with its own manually-wired rules? Its own Prometheus?
      - fulafel 3y ago
        I'd wager most production systems don't have those.
    - geerlingguy 3y ago
      Compared to Compose on a single server, Kubernetes deployments can have multiple layers (control plane, containers, usually some sidecars) where little quirks occur. It almost necessitates deeper monitoring and/or extra tools to give a dashboard/'single pane of glass', whereas I can run Compose and just log into the individual server and jump around logs.
      - vbezhenar 3y ago
        Kubernetes control plane is the same as docker daemon. Sidecars do not exist in kubernetes, unless you install something that adds them. Containers are... containers? The same containers you'd see in docker. Kubernetes does not need deeper monitoring compared to docker. You can jump around kubernetes logs just like you do with docker. Except you don't even need to log into individual server. I don't use any kind of dashboard for my kubernetes cluster. I installed kubernetes-dashboard, so developers and managers can enjoy their dashboards but I've found no value with it. kubectl is more than enough.
brazzledazzle 3y ago
Docker compose falls apart when you need something clustered across nodes or any other multi-host deployment. Once you have to rip it apart it becomes way less appealing. Though I haven't tried it in a while and perhaps the ergonomics have changed.
- flatline 3y ago
  Docker swarm is supposed to be the migration target for these cases, but I’ve never actually used it.
  - tommica 3y ago
    I run it on a personal server, so there is no multi server setup, but I enjoy the fact that it's just an extension of docker compose.
  - agumonkey 3y ago
    I lost track about it, I thought swarm was canceled
    - Cfu288 3y ago
      Docker swarm mode (still around) != docker swarm (cancelled). Poorly named so the confusion is normal
  - e12e 3y ago
    Swarm works, but has poor support for volumes - which means it's tricky to run legacy applications on swarm (which eg uploads files to local disk, not s3 - or keeps state/cache on disk, not a database). Ingress is also more complicated/bespoke - the best I've found is traefik with labels for routing/config. My advice today would be to scale Docker compose vertically (eg: on a dedicated server) - then move to Kubernetes. The swarm middle ground isn't really worth it IMNHO.
    - nwienert 3y ago
      Traefik is really simple to set up and I’d bet setting up s3 is 100x easier than kubernetes no? Unless I’m missing something. Fwiw I found swarm lovely and just so much easier to work with than anything else solving the same problems.
vbezhenar 3y ago
Docker compose is extremely basic. Even on a single node Kubernetes is more capable than docker compose. Ingresses, Services, Deployments, Cronjobs, Statefulsets. Awesome operators. Cert-manager. There're so many things I could easily do with Kubernetes that would require countless unmaintainable bash scripts and ad-hoc approaches with docker-compose.
- Demiurge 3y ago
  Yes, but 50% of the projects the parent poster sees, and 90% of the projects that I see, do not require any of that.
- preya2k 3y ago
  For me, I really enjoy how basic it is. I’ve never missed any of these things for my simple single-host scenarios.
  - vbezhenar 3y ago
    Simple setups will be as simple with Kubernetes. However with Kubernetes your infrastructure will be ready to scale. You need to expose both front-end and back-end services under the same host? No need to tinker with nginx configs, you just create two ingresses and do it in a standardized way. You need to run your service with two replicas and rolling zero-downtime update? Kubernetes has it out of the box. Good luck implementing all the necessary dances with custom docker-compose setup without dropped requests. You want to customize healthcheck? Kubernetes got you covered with all the imaginable scenarios. The only missing block in Kubernetes is support for building containers. This is implemented with docker-compose extremely elegantly and simple. That I admit.
    - mardifoufs 3y ago
      I wonder if there are any plans to add a docker compose like api to kubernetes? I know, kubernetes should not be used for that but there must be a better way than to use docker compose or manually building containers? Don't know a lot about DevOps so I sometimes just build my containers manually with a docker file but it's not ideal for up to date dependencies.
badrequest 3y ago
I've never understood using K8s in local dev over docker-compose, but I can absolutely see why someone would use it production.
- rzmmm 3y ago
  If someone is working on complex kubernetes configuration, it's beneficial to have kubernetes in local dev environment, since it's trivial to test changes.

by jadbox 3y ago

Already__Taken 3y ago

ah the 1% of the time you do want to use something in kubernetes and you have a pile of compose is pretty demoralising though

cpitman 3y ago
I haven't run into the need to do that, but there is the Kompose project that exists to help with the conversion (https://kompose.io/ https://kompose.io/)!

nonameiguess 3y ago

I guess they stopped doing this and now consider the product mature from looking at their website, but Docker Compose used to tell you that it wasn't suitable or intended for production use. That seemed like a pretty good reason not to do it.

geerlingguy 3y ago
Compose has its own footguns, but I've run a number of services with just Compose on a VM or direct on bare metal and the few little issues I've had were easy to debug and fix. Very much unlike Kubernetes, where even a small deployment can sometimes be maddening to debug. Great for larger deployments where complexity is basically guaranteed, but anything that could run on one server with a hot spare is a great fit for Compose!
- erulabs 3y ago
  Surprised to hear you think debugging in kubernetes is maddening. It’s absolutely very different, but if I had to debug a system I knew nothing about, I’d rather debug a kubernetes based system rather than any other. Standards and all that. Anyways - for a single container or two obviously you’re right - I’m just not sure that’s so common outside of a side project anymore.
  - derefr 3y ago
    Especially because "production" implies an ingress controller, monitoring, etc. Does a Docker Compose-based production setup contain run its own Nginx with its own manually-wired rules? Its own Prometheus?
    - fulafel 3y ago
      I'd wager most production systems don't have those.
  - geerlingguy 3y ago
    Compared to Compose on a single server, Kubernetes deployments can have multiple layers (control plane, containers, usually some sidecars) where little quirks occur. It almost necessitates deeper monitoring and/or extra tools to give a dashboard/'single pane of glass', whereas I can run Compose and just log into the individual server and jump around logs.
    - vbezhenar 3y ago
      Kubernetes control plane is the same as docker daemon. Sidecars do not exist in kubernetes, unless you install something that adds them. Containers are... containers? The same containers you'd see in docker. Kubernetes does not need deeper monitoring compared to docker. You can jump around kubernetes logs just like you do with docker. Except you don't even need to log into individual server. I don't use any kind of dashboard for my kubernetes cluster. I installed kubernetes-dashboard, so developers and managers can enjoy their dashboards but I've found no value with it. kubectl is more than enough.

brazzledazzle 3y ago

Docker compose falls apart when you need something clustered across nodes or any other multi-host deployment. Once you have to rip it apart it becomes way less appealing. Though I haven't tried it in a while and perhaps the ergonomics have changed.

flatline 3y ago
Docker swarm is supposed to be the migration target for these cases, but I’ve never actually used it.
- tommica 3y ago
  I run it on a personal server, so there is no multi server setup, but I enjoy the fact that it's just an extension of docker compose.
- agumonkey 3y ago
  I lost track about it, I thought swarm was canceled
  - Cfu288 3y ago
    Docker swarm mode (still around) != docker swarm (cancelled). Poorly named so the confusion is normal
- e12e 3y ago
  Swarm works, but has poor support for volumes - which means it's tricky to run legacy applications on swarm (which eg uploads files to local disk, not s3 - or keeps state/cache on disk, not a database). Ingress is also more complicated/bespoke - the best I've found is traefik with labels for routing/config. My advice today would be to scale Docker compose vertically (eg: on a dedicated server) - then move to Kubernetes. The swarm middle ground isn't really worth it IMNHO.
  - nwienert 3y ago
    Traefik is really simple to set up and I’d bet setting up s3 is 100x easier than kubernetes no? Unless I’m missing something. Fwiw I found swarm lovely and just so much easier to work with than anything else solving the same problems.

vbezhenar 3y ago

Docker compose is extremely basic. Even on a single node Kubernetes is more capable than docker compose. Ingresses, Services, Deployments, Cronjobs, Statefulsets. Awesome operators. Cert-manager. There're so many things I could easily do with Kubernetes that would require countless unmaintainable bash scripts and ad-hoc approaches with docker-compose.

Demiurge 3y ago
Yes, but 50% of the projects the parent poster sees, and 90% of the projects that I see, do not require any of that.
preya2k 3y ago
For me, I really enjoy how basic it is. I’ve never missed any of these things for my simple single-host scenarios.
- vbezhenar 3y ago
  Simple setups will be as simple with Kubernetes. However with Kubernetes your infrastructure will be ready to scale. You need to expose both front-end and back-end services under the same host? No need to tinker with nginx configs, you just create two ingresses and do it in a standardized way. You need to run your service with two replicas and rolling zero-downtime update? Kubernetes has it out of the box. Good luck implementing all the necessary dances with custom docker-compose setup without dropped requests. You want to customize healthcheck? Kubernetes got you covered with all the imaginable scenarios. The only missing block in Kubernetes is support for building containers. This is implemented with docker-compose extremely elegantly and simple. That I admit.
  - mardifoufs 3y ago
    I wonder if there are any plans to add a docker compose like api to kubernetes? I know, kubernetes should not be used for that but there must be a better way than to use docker compose or manually building containers? Don't know a lot about DevOps so I sometimes just build my containers manually with a docker file but it's not ideal for up to date dependencies.

badrequest 3y ago

I've never understood using K8s in local dev over docker-compose, but I can absolutely see why someone would use it production.

rzmmm 3y ago
If someone is working on complex kubernetes configuration, it's beneficial to have kubernetes in local dev environment, since it's trivial to test changes.