5 ms·

Yes, microservices and containers are a way for developers to feel important and grant cover to an unabashed manifestation of Conway's Law. This particular fad

Yes, microservices and containers are a way for developers to feel important and grant cover to an unabashed manifestation of Conway's Law. This particular fad is wreaking ridiculous havoc that is going to have the computing world in quite a mess soon. In the real world, distributed systems are a complication that is better left alone if they're not necessary for a problem space. You very likely don't need Kubernetes (watch out, this sentiment will get your HN account throttled; I've already been punished for it so it's safe for me to say). The reason Google et al push these distributed systems is because it gets you to buy more cloud instances. Very predictable and straightforward behavior there. Like all technology that proliferates, this is definitely more about politics than merit, but it doesn't really move the duties around practically; anyone who thinks that a group so undisciplined and lackluster as your average software developer will willingly take on new responsibilities is detached. As a sysadmin, you just get developers delivering garbage to your doorstep and expecting you to deal with it, same as always. It's just packaged in a stupid way now, that makes the developers say "Well I sent you a container, so this should be up and running in like, 5 minutes, right?" In practice, most developers won't even write their own Dockerfiles; my experience has been that they develop locally as they always have, and then expect the sysadmins/"devops" (now a term for sysadmins) to do all that mumbo-jumbo. But yes, we've been through this song and dance before. It's static v. dynamic linking. It's shared tenancy v. single tenancy. None of it is new, it just gets a different name and oscillates from position A to position B. It's an embarrassment and a testament against our industry that we continue to thrash wildly and be so thoroughly dominated by fads.

_jal 9y ago
Looks like someone else has been around the block a couple times. There are going to be massive opportunities for good consultants in this muddle. I'm currently watching one train wreck, thankfully at a distance. Nothing novel - they moved ops under the engineering umbrella. The engineering manager not only has no clue what the differences in managing the two are, but seems unaware that there are any. So of course the good system people quickly left, the engineers are slowly discovering that containers don't magically erase the need for understanding how things work, and they're spending six figures a month on K8s consultants. We seem to have a generational cycle on IT/sysadmin/ops hate. As you get older, it becomes amusing watching every new batch of engineers learn why they exist.
- devonkim 9y ago
  This kind of cycle is part of why I’m regretting 13+ years in different incarnations of ops. Developers have always gotten business priority over ops unless it comes to security and I’ve seen far more places brush aside even security in the name of more features than should be healthy to one’s sanity. Even at the peak of ops when everyone was scrambling to get security engineers and such, I simply never heard of some bright kid graduating from school and getting $200k+ comp packages... for being in ops, even for DoD cyber security BS. Ops is a cost center even in the cutting edge of organizations in this respect. Few companies will make more money by throwing more money at ops that is. Combined with the serious health hazards of being on-call I’m a bit worried when I hear about companies asking their developers to get more involved with ops.
  - cookiecaper 9y ago
    Yeah. "DevOps" originated as a naive attempt to fix this issue by eliminating the "wall" between the roles, but it was quickly misunderstood. The old-school, master of the universe-style sysadmin who had to use advanced wizardry developed out of deep knowledge and intuition about how the computer worked at a fundamental level is still the ideal developer OR operator. But it's hard to find people who even understand this role, let alone people who are hiring for it. People only seem to understand pigeonholes, despite occasional well-intended attempts to unify, that generally have disastrous results. You just need a special breed to get someone who can really function up and down the stack, and every company should seek out, train, and develop such highly-empowered "tech special forces", for lack of a better term. It's tragic that so many companies don't realize this.
crdoconnor 9y ago
>Yes, microservices and containers are a way for developers to feel important and grant cover to an unabashed manifestation of Conway's Law. Microservices are a more sensible idea in big companies because they can be used (provided you have a sane architect) to create fairly easy to use abstractions that code in any language can use. Where communication and coordination costs are very high but there's lots of resources and programmers (of varying flavors) available it makes sense. Within individual teams and particularly in small companies, where communication and coordination costs are very low and there are not a lot of people or resources available, "microservicing" everything is a brain dead idea. Introducing a network endpoint does not necessarily decouple things any better than a function interface would and it imposes complications and costs that a function interface doesn't.
- cookiecaper 9y ago
  I think the problem is partially in the terminology, which drives people to extremes. What we need are "ReasonableServices" (TM), but it's not such a catchy name. I've seen several attempts at "microservices" that I like to describe, rather, as "puny-services": weak, random little one-offs that get enlisted as a "service" because whoever needed that function didn't feel like seeing if anyone else had already provided it through a different endpoint, etc. So yes, there is a way to do "microservices" right, provided your understanding of "micro" is sufficiently macro and it doesn't just become an excuse for every little self-selecting group of developers to go off in their own world and ignore everyone and everything they'd rather not worry about all day. But in my experience, it almost always does. If you have a team disciplined and experienced enough to do microservices well, you probably have a team that is dubious of the hype and will just start building reasonable software without worrying about how micro-servicey it is.
  - crdoconnor 9y ago
    I think the hype, groupthink and a propensity to view this as a technical, rather than a largely an economic problem drives people to extremes. I think Martin Fowler is largely at fault for that - I think he saw it work really well in his environment, but I think he really didn't understand why. He was utterly oblivious to the fact that: * In a different environment with lower coordination costs it is net harmful * "decoupling" and "microservices" are entirely orthogonal problems and if decoupling is not done while you microservice the shit out of everything your problems are brutally magnified. Tight coupling over network end points is the worst. My time wasted with this problem actually still makes me rage. Before I worked in a big company I was a MAJOR microservices skeptic because my experience was pretty much just with seeing them grossly misapplied on small teams. Now I am working in a big company I find myself recommending the approach simply because I often see them as a working solution to often extreme people and resource coordination costs that in my previous lives in a small company, simply didn't exist. I actually kind of shocked myself and had a mini identity crisis when I started doing that. I'm not the microservices guy! I'm the skeptic! Nonetheless, I had that problem and the solution made sense... In a small company you tend to just have a never ending series of problems, people working at close to peak efficiency and your main problem isn't coordination but just not having enough time, people or resources to solve your problems. The introduction of unnecessary network endpoints everywhere is just going to slow you down and cause problems. If you also have an architect without a deep appreciation of loose coupling that's a recipe for a disastrous rube goldbergesque pile of steaming shit. In a big company there's often tons of developers all over the place, mostly working very inefficiently and a lot of them will speak different programming languages. Your main problem isn't a lack of resources or time - both are plentiful, it's trying to prevent the rigid and slow moving bureaucracy from preventing you getting anything done and breaking down work into clean chunks so that individuals and teams have enough (but not too much) to work on and the problem gets solved. In that situation a few more network endpoints that are strictly necessary from a technical standpoint is a minor price to pay. Strictly speaking this isn't 100% a big company / small company thing (my big company environment isn't necessarily like yours) but I'm convinced it's 100% about coordination costs.

_jal 9y ago

Looks like someone else has been around the block a couple times. There are going to be massive opportunities for good consultants in this muddle. I'm currently watching one train wreck, thankfully at a distance. Nothing novel - they moved ops under the engineering umbrella. The engineering manager not only has no clue what the differences in managing the two are, but seems unaware that there are any. So of course the good system people quickly left, the engineers are slowly discovering that containers don't magically erase the need for understanding how things work, and they're spending six figures a month on K8s consultants. We seem to have a generational cycle on IT/sysadmin/ops hate. As you get older, it becomes amusing watching every new batch of engineers learn why they exist.

devonkim 9y ago
This kind of cycle is part of why I’m regretting 13+ years in different incarnations of ops. Developers have always gotten business priority over ops unless it comes to security and I’ve seen far more places brush aside even security in the name of more features than should be healthy to one’s sanity. Even at the peak of ops when everyone was scrambling to get security engineers and such, I simply never heard of some bright kid graduating from school and getting $200k+ comp packages... for being in ops, even for DoD cyber security BS. Ops is a cost center even in the cutting edge of organizations in this respect. Few companies will make more money by throwing more money at ops that is. Combined with the serious health hazards of being on-call I’m a bit worried when I hear about companies asking their developers to get more involved with ops.
- cookiecaper 9y ago
  Yeah. "DevOps" originated as a naive attempt to fix this issue by eliminating the "wall" between the roles, but it was quickly misunderstood. The old-school, master of the universe-style sysadmin who had to use advanced wizardry developed out of deep knowledge and intuition about how the computer worked at a fundamental level is still the ideal developer OR operator. But it's hard to find people who even understand this role, let alone people who are hiring for it. People only seem to understand pigeonholes, despite occasional well-intended attempts to unify, that generally have disastrous results. You just need a special breed to get someone who can really function up and down the stack, and every company should seek out, train, and develop such highly-empowered "tech special forces", for lack of a better term. It's tragic that so many companies don't realize this.

crdoconnor 9y ago

>Yes, microservices and containers are a way for developers to feel important and grant cover to an unabashed manifestation of Conway's Law. Microservices are a more sensible idea in big companies because they can be used (provided you have a sane architect) to create fairly easy to use abstractions that code in any language can use. Where communication and coordination costs are very high but there's lots of resources and programmers (of varying flavors) available it makes sense. Within individual teams and particularly in small companies, where communication and coordination costs are very low and there are not a lot of people or resources available, "microservicing" everything is a brain dead idea. Introducing a network endpoint does not necessarily decouple things any better than a function interface would and it imposes complications and costs that a function interface doesn't.

cookiecaper 9y ago
I think the problem is partially in the terminology, which drives people to extremes. What we need are "ReasonableServices" (TM), but it's not such a catchy name. I've seen several attempts at "microservices" that I like to describe, rather, as "puny-services": weak, random little one-offs that get enlisted as a "service" because whoever needed that function didn't feel like seeing if anyone else had already provided it through a different endpoint, etc. So yes, there is a way to do "microservices" right, provided your understanding of "micro" is sufficiently macro and it doesn't just become an excuse for every little self-selecting group of developers to go off in their own world and ignore everyone and everything they'd rather not worry about all day. But in my experience, it almost always does. If you have a team disciplined and experienced enough to do microservices well, you probably have a team that is dubious of the hype and will just start building reasonable software without worrying about how micro-servicey it is.
- crdoconnor 9y ago
  I think the hype, groupthink and a propensity to view this as a technical, rather than a largely an economic problem drives people to extremes. I think Martin Fowler is largely at fault for that - I think he saw it work really well in his environment, but I think he really didn't understand why. He was utterly oblivious to the fact that: * In a different environment with lower coordination costs it is net harmful * "decoupling" and "microservices" are entirely orthogonal problems and if decoupling is not done while you microservice the shit out of everything your problems are brutally magnified. Tight coupling over network end points is the worst. My time wasted with this problem actually still makes me rage. Before I worked in a big company I was a MAJOR microservices skeptic because my experience was pretty much just with seeing them grossly misapplied on small teams. Now I am working in a big company I find myself recommending the approach simply because I often see them as a working solution to often extreme people and resource coordination costs that in my previous lives in a small company, simply didn't exist. I actually kind of shocked myself and had a mini identity crisis when I started doing that. I'm not the microservices guy! I'm the skeptic! Nonetheless, I had that problem and the solution made sense... In a small company you tend to just have a never ending series of problems, people working at close to peak efficiency and your main problem isn't coordination but just not having enough time, people or resources to solve your problems. The introduction of unnecessary network endpoints everywhere is just going to slow you down and cause problems. If you also have an architect without a deep appreciation of loose coupling that's a recipe for a disastrous rube goldbergesque pile of steaming shit. In a big company there's often tons of developers all over the place, mostly working very inefficiently and a lot of them will speak different programming languages. Your main problem isn't a lack of resources or time - both are plentiful, it's trying to prevent the rigid and slow moving bureaucracy from preventing you getting anything done and breaking down work into clean chunks so that individuals and teams have enough (but not too much) to work on and the problem gets solved. In that situation a few more network endpoints that are strictly necessary from a technical standpoint is a minor price to pay. Strictly speaking this isn't 100% a big company / small company thing (my big company environment isn't necessarily like yours) but I'm convinced it's 100% about coordination costs.