4 ms·

With time, I discovered something interesting: for us, techies, using container orchestration is about reliability, zero-downtime deployments, limiting blast ra

by benterix 11mo ago

With time, I discovered something interesting: for us, techies, using container orchestration is about reliability, zero-downtime deployments, limiting blast radius etc. But for management, it's completely different. It's all about managing complexity on an organizational level. It's so much easier to think in terms "Team 1 is in charge of microservice A". And I know from experience that it works decently enough, at least in some orgs with competent management.

embedding-shape 11mo ago
> using container orchestration is about reliability, zero-downtime deployments I think that's the first time I've heard any "techie" say we use containers because of reliability or zero-downtime deployments, those feel like they have nothing to do with each other, and we've been building reliable server-side software with zero-downtime deployments long before containers became the "go-to", and if anything it was easier before containers.
- benterix 11mo ago
  It would be interesting to hear your story, mine is that containers in general start an order of magnitude faster than vms (in general! we can easily find edge cases) and hence e.g. horizontal scaling is faster. You say it was easier before containers, I say k8s in spite of its complexity is a huge blessing as teams can upgrade their own parts independently and do things like canary releases easily with automated rollbacks etc. It's so much faster than VMs or bare metal (which I still use a lot and don't plan to abandon anytime soon but I understand their limitations).
  - embedding-shape 11mo ago
    In general, my experience is "the more moving parts == less reliable", if I were to generalize across two decades of running web services. The most reliable platforms I've helped manage has been platforms that tried to avoid adding extra complexity until they really couldn't avoid it, and when I left still deployed applications by copy a built binary to a Linux host, reload the systemd service, switch the port in the proxy and let traffic hit the new service while healtchecking, and when green, switch over and stop the old service. Deploys usually took minutes (unless something was broken), scaling worked the same as if you were using anything else, increase a number and redeploy, and no Kubernetes, Docker or even containers as far as the eye could see.
torben-friis 11mo ago
It’s not a management thing. I’m an engineer and I think it’s THE main advantage micro services actually provide: they split your code hard and allow a team to actually get ownership of the domain. No crossing domain boundaries, no in between shared code, etc. I know: it’s ridiculous to have an architectural barrier for an organizational reason, and the cost of a bad slice multiplies. I still think in some situations, that is better to the gas-station-bathroom effect of shared codebases.
- strken 11mo ago
  I don't see why it's ridiculous to have an architectural barrier for org reasons. Requiring every component to be behind a network call seems like overkill in nearly all cases, but encapsulating complexity into a library where domain experts can maintain it is how most software gets built. You've got to lock those demons away where they can't affect the rest of the users.
  - vbezhenar 11mo ago
    The problem is, that library usually does not provide good enough boundaries. C library can just shit over your process memory. Java library can cause all the hell over your objects with reflection, can just call System.exit(LOL). Minimal boundary to keep demons at bay is process boundary and you need some way for processes to talk to each other. If you're separating components into processes, it's very natural to put them to different machines, so you need your IPC to be network calls. One more step and you're implementing REST, because infra people love HTTP.
    - pjmlp 11mo ago
      And then we're back to 1980's UNIX process model before wide adoption of dynamic loading, but because we need to be cool we call them microservices.
    - sevensor 11mo ago
      > it's very natural to put them to different machines, so you need your IPC to be network calls But why is this natural? I’m not saying we shouldn’t have network RPC, but it’s not obvious to me that we should have only network RPC when there are cheap local IPC mechanisms.
Towaway69 11mo ago
As soon there is more than one container to organise, it becomes a management task for said techies. Then suddenly one realises that techies can also be bad at management. Management of a container environment not only requires deployment skills but also documentational and communication skills. Suddenly it’s not management rather the techie that can't manage their tech stack. This pointing of fingers at management is rather repetitive and simplistic but also very common.
deleted 11mo ago
[deleted]

embedding-shape 11mo ago

> using container orchestration is about reliability, zero-downtime deployments I think that's the first time I've heard any "techie" say we use containers because of reliability or zero-downtime deployments, those feel like they have nothing to do with each other, and we've been building reliable server-side software with zero-downtime deployments long before containers became the "go-to", and if anything it was easier before containers.

benterix 11mo ago
It would be interesting to hear your story, mine is that containers in general start an order of magnitude faster than vms (in general! we can easily find edge cases) and hence e.g. horizontal scaling is faster. You say it was easier before containers, I say k8s in spite of its complexity is a huge blessing as teams can upgrade their own parts independently and do things like canary releases easily with automated rollbacks etc. It's so much faster than VMs or bare metal (which I still use a lot and don't plan to abandon anytime soon but I understand their limitations).
- embedding-shape 11mo ago
  In general, my experience is "the more moving parts == less reliable", if I were to generalize across two decades of running web services. The most reliable platforms I've helped manage has been platforms that tried to avoid adding extra complexity until they really couldn't avoid it, and when I left still deployed applications by copy a built binary to a Linux host, reload the systemd service, switch the port in the proxy and let traffic hit the new service while healtchecking, and when green, switch over and stop the old service. Deploys usually took minutes (unless something was broken), scaling worked the same as if you were using anything else, increase a number and redeploy, and no Kubernetes, Docker or even containers as far as the eye could see.

torben-friis 11mo ago

It’s not a management thing. I’m an engineer and I think it’s THE main advantage micro services actually provide: they split your code hard and allow a team to actually get ownership of the domain. No crossing domain boundaries, no in between shared code, etc. I know: it’s ridiculous to have an architectural barrier for an organizational reason, and the cost of a bad slice multiplies. I still think in some situations, that is better to the gas-station-bathroom effect of shared codebases.

strken 11mo ago
I don't see why it's ridiculous to have an architectural barrier for org reasons. Requiring every component to be behind a network call seems like overkill in nearly all cases, but encapsulating complexity into a library where domain experts can maintain it is how most software gets built. You've got to lock those demons away where they can't affect the rest of the users.
- vbezhenar 11mo ago
  The problem is, that library usually does not provide good enough boundaries. C library can just shit over your process memory. Java library can cause all the hell over your objects with reflection, can just call System.exit(LOL). Minimal boundary to keep demons at bay is process boundary and you need some way for processes to talk to each other. If you're separating components into processes, it's very natural to put them to different machines, so you need your IPC to be network calls. One more step and you're implementing REST, because infra people love HTTP.
  - pjmlp 11mo ago
    And then we're back to 1980's UNIX process model before wide adoption of dynamic loading, but because we need to be cool we call them microservices.
  - sevensor 11mo ago
    > it's very natural to put them to different machines, so you need your IPC to be network calls But why is this natural? I’m not saying we shouldn’t have network RPC, but it’s not obvious to me that we should have only network RPC when there are cheap local IPC mechanisms.

Towaway69 11mo ago

As soon there is more than one container to organise, it becomes a management task for said techies. Then suddenly one realises that techies can also be bad at management. Management of a container environment not only requires deployment skills but also documentational and communication skills. Suddenly it’s not management rather the techie that can't manage their tech stack. This pointing of fingers at management is rather repetitive and simplistic but also very common.

deleted 11mo ago

[deleted]