4 ms·
One of our projects has ~15 backend engineers and ~30 microservices. It also requires a separate platform team. A k8s cluster with several nodes. It has inciden
by kgeist 3y ago
One of our projects has ~15 backend engineers and ~30 microservices. It also requires a separate platform team. A k8s cluster with several nodes. It has incidents like every week because the whole thing is like a house of cards. A small feature requires changes in ~5 microservices and a lot of coordination between the teams. They always need to think about things like circuit breakers, timeouts, APIs (schema synchronization, versioning), event queues, distributed transactions, sagas, distributed locking, k8s configs, Git repository management, distributed tracing etc. etc.
On the other hand, there's another project, which is a monolith. Around ~15 engineers. Incidents happen very rarely. The infrastructure is simple: just one beefy server without k8s (but there's a standby replica of course), so the team often does maintenance of the server on their own instead of a separate SRE/production team. The monolith is modular so it enjoys same advantages as microservices when it comes to module isolation etc. Highload functionality is moved to separate Go services only when needed (currently 2 or 3 satellite microservices). All changes happen in the same codebase so features are released faster and it's easier to reason about.
I'm currently on the second project and whenever I attend architecture reviews of the first project, I'm shocked how overcomplicated the first project has become. A simple addition of a new object property can end up in 5-10 engineers arguing for hours whose microservices are affected and how they should best communicate.
- f4c39012 3y agoHas the microservice + backend project team ever observed the monolith project team?