3 ms·

> Separate containers for multiple services owned by the same team sounds like a lot of overhead. If I want to work on something do I have to spin up a bunch of

> Separate containers for multiple services owned by the same team sounds like a lot of overhead. If I want to work on something do I have to spin up a bunch of containers, with all the overhead that brings? Can I easily use a debugger across all of the code that implements a given feature? Most of the services just ran normally on a developers system, or we had a docker script that could spin things up locally, or you used one of the development sandboxes running on the system. BTW, we had a system that allowed very low cost creation and management of containers (VMs in our case). I think that is an important detail. Could you use a debugger for a feature? Most likely not, because a lot of our features already pulled data from external sources which you had no visibility into. > You're talking like there was a single "owner" for each service? What did you do when they went on holiday / left? How did people share knowledge about the system? Same problem every development team has. Who setup up the CI server? Who setup the front-end proxy? Who configured the AWS setup? Everything needs redundancy. We had a list of all tools and services and had multiple names assigned to each one. Even without services a team should have this. > Dealing with an outside service brings a lot of overhead. It's harder to debug, harder to test. When things are maintained by a separate team it makes sense to treat them as an outside service, but within the same team why would you make all that extra work for yourself? Because it worked really well and we didn't see that extra work. I disagree that external services are hard to interface to. They are hard to work with when they are buggy, but if they hit the specs, they generally are not an issue. I have no problem interfacing to S3. Because of the loose coupling, things were very extensible and easy to modify. A few times we would replace a service with a new one by adding the new one and slowly migrating everything over to use it. In some cases, that migration took over a year, but that was fine. > Sounds like you need a better language. I agree that anything that might segfault belongs in a separate service, but most of the time the solution is to not use stuff that might segfault. LOL, it was Python. Bugs happen, things break. Everything can eventually segfault for all kinds of reasons. Run a complex system for a decade and you will see all types of issues. I remember running OpenVZ before switching to KVM and occasionally we would even see kernel panics. > How does that fit with "services only had to talk to other services from the same commit"? You conveniently left out "or branch". Listen, I am not trying to sell anything, I don't care what you think. I am just sharing my teams experience. It was overwhelmingly positive using services to build a complex system with real revenue from Fortune 100 customers for almost a decade.

lmm 6y ago
> Same problem every development team has. Who setup up the CI server? Who setup the front-end proxy? Who configured the AWS setup? Everything needs redundancy. Sure, so all those things should be owned by a team rather than an individual. But that makes the whole "When a container/vm would run out of memory, we all didn't have to scramble to figure out who broke something. The person who owned the code for that service dealt with it. If a service started kicking out errors nobody but the owner had to be distracted." kind of pointless; the whole team should know about the service and be aware of any recent changes to it, and everyone should be able to debug it. > I disagree that external services are hard to interface to. They are hard to work with when they are buggy, but if they hit the specs, they generally are not an issue. Well of course programming is easy as long as there are no bugs! But the most important and hardest part of programming is debugging, and every time you hit a service boundary you have to figure out which side of it the bug lies on; that ends up being a significant amount of work. Practically speaking you need to implement a stub version of the service for testing with, monitoring that can confirm whether that service is up and running, maybe some kind of tracing/replay that lets you track specific problematic requests... and that's even more work for internal services, for something like S3 at least some of the bits and pieces for doing that have been built already. > Because of the loose coupling, things were very extensible and easy to modify. Absolutely, but you don't need a network boundary to achieve that. (I mean, maybe you do in Python because it has no visibility rules/enforcement and incredibly poor dependency management, but that's not a general problem). > You conveniently left out "or branch". Different commits on the same branch can be just as different as different branches. You can't have it both ways, either you keep all your services in lockstep which means you have to restart everything to fix a bug, or you have to worry about different services being on different versions. (Actually, since you normally can't restart a group of microservices atomically, you have to worry about version compatibility even if you try to always deploy new versions of everything together, IME). > Listen, I am not trying to sell anything, I don't care what you think. I am just sharing my teams experience. It was overwhelmingly positive using services to build a complex system with real revenue from Fortune 100 customers for almost a decade. Your post sounds like a salesperson/cultist with the "not a single engineer that ever voiced a concern over how things worked" emphasis. I've worked in multiple microservice-oriented environments including those that claimed success for microservices, and they've all either had the same problems I'm talking about, or actually not been talking about microservices but rather reasonably sized services (i.e. having teams - groups of ~10 people who worked together and had standups together - maintain one or two services each).

lmm 6y ago

> Same problem every development team has. Who setup up the CI server? Who setup the front-end proxy? Who configured the AWS setup? Everything needs redundancy. Sure, so all those things should be owned by a team rather than an individual. But that makes the whole "When a container/vm would run out of memory, we all didn't have to scramble to figure out who broke something. The person who owned the code for that service dealt with it. If a service started kicking out errors nobody but the owner had to be distracted." kind of pointless; the whole team should know about the service and be aware of any recent changes to it, and everyone should be able to debug it. > I disagree that external services are hard to interface to. They are hard to work with when they are buggy, but if they hit the specs, they generally are not an issue. Well of course programming is easy as long as there are no bugs! But the most important and hardest part of programming is debugging, and every time you hit a service boundary you have to figure out which side of it the bug lies on; that ends up being a significant amount of work. Practically speaking you need to implement a stub version of the service for testing with, monitoring that can confirm whether that service is up and running, maybe some kind of tracing/replay that lets you track specific problematic requests... and that's even more work for internal services, for something like S3 at least some of the bits and pieces for doing that have been built already. > Because of the loose coupling, things were very extensible and easy to modify. Absolutely, but you don't need a network boundary to achieve that. (I mean, maybe you do in Python because it has no visibility rules/enforcement and incredibly poor dependency management, but that's not a general problem). > You conveniently left out "or branch". Different commits on the same branch can be just as different as different branches. You can't have it both ways, either you keep all your services in lockstep which means you have to restart everything to fix a bug, or you have to worry about different services being on different versions. (Actually, since you normally can't restart a group of microservices atomically, you have to worry about version compatibility even if you try to always deploy new versions of everything together, IME). > Listen, I am not trying to sell anything, I don't care what you think. I am just sharing my teams experience. It was overwhelmingly positive using services to build a complex system with real revenue from Fortune 100 customers for almost a decade. Your post sounds like a salesperson/cultist with the "not a single engineer that ever voiced a concern over how things worked" emphasis. I've worked in multiple microservice-oriented environments including those that claimed success for microservices, and they've all either had the same problems I'm talking about, or actually not been talking about microservices but rather reasonably sized services (i.e. having teams - groups of ~10 people who worked together and had standups together - maintain one or two services each).