4 ms·

Isn't a mixed fleet always the case once you have more than one server and do rolling updates?

• taeric 2y ago
  Yes. And if you structure your code to explicitly do this, it is a lot easier to reason about.
• xyzzy123 2y ago
  Sort of; at medium scale you can blue/green your whole system out of the monorepo (even if its say 20 services) in k8s and flip the ingresses to cut over during release. Of course k8s not required, you can do it in straight IaC etc (i.e deploy a whole parallel system and switch). It's still "mixed fleet" in terms of any shared external resources (queues, db state, etc) but you can change service interfaces etc with impunity and not worry about compatibility / versioning between services. Throwing temporary compute at the problem can save a lot of busywork and/or thinking about integration problems. This stops being practical if you get _very_ big but at that point you presumably have more money and engineers to throw at the problem.
  • chipdart 2y ago
    > Sort of; at medium scale you can blue/green your whole system out of the monorepo (even if its say 20 services) in k8s and flip the ingresses to cut over during release. That's overall a bad idea, and negates the whole point of blue-green deployments. This is particularly bad if you have in place any form of distributed transaction. There are very good reasons why deployment strategies such as rolling deployments and one-box deployments were developed. You need to be able to gradually roll out a change to prevent problems from escalating and causing downtime. If you use all that infrastructure to then flip global switches then you're building up all this infrastructure to the negate any of that with practices that invariably cause problems. And this is being done just because someone thinks it's a good idea to keep all code in a single repo?
    • xyzzy123 2y ago
      > You need to be able to gradually roll out a change to prevent problems from escalating and causing downtime. From my perspective I prefer it to rolling updates, it just costs a lot of resources. It gives you a chance to verify all the parts of your system together during release. You deploy an entire new parallel copy of your system and test it. This can be manual or automated. Once you have confidence, you can flip traffic. Alternatively you can siphon off a slice of production traffic and monitor metrics as you gradually shift traffic over to the new release. Note that you can also nearly "instantly" roll back at any point, unlike with a rolling update. Again, this is only "sort of" a mixed fleet situation because you can maintain the invariant that service A at v1.23 only talks to service B at v1.23. This means you can atomically refactor the interface between two services and deploy them without thinking too hard about what happens when versions are mixed. Distributed transactions I'm not so sure about, we would have to talk specific situations. > And this is being done just because someone thinks it's a good idea to keep all code in a single repo? It's more like, if you have a monorepo this is one option that becomes simpler for you to do, should you happen to want to do it.