6 ms·

Microservices failure stories? “All of them. The End”

tnolet 8y ago
I'm not convinced all microservices ventures are failures. Having worked in the space a bit as a founder/CTO of a vendor in that space I've just seen many examples of misguided attempts due to fashion / CV-driven development / hype driven development. The pattern is valid, just not for everyone at every time.
- hjacobs 8y ago
  ThoughtWorks even has a term for it: "Microservice envy" https://www.thoughtworks.com/radar/techniques/microservice-envy https://www.thoughtworks.com/radar/techniques/microservice-e...
- ex_amazon_sde 8y ago
  The good part of "microservices" is "services". The bad part is "micro".
  - tyldum 8y ago
    When the definition turns to some kind of dogma, then it will fail.
lucidone 8y ago
I'm consulting on a micro services back end right now with mostly prior experience with monoliths. What is the selling point that drives companies down this direction? It's insane, and my client keeps trying to hire new developers and bring on more consultants to build this thing, but the amount of knowledge required is more than any one person can handle. I have similar issues with their choice of db (nosql) and its inflexibility.
- stingraycharles 8y ago
  The biggest reason for choosing microservices _should_ be scaling development teams: microservices allow multiple teams to work on different code bases, without stepping on each other's toes. What actually happens, tough, is that (uninformed) people choose it because they think it brings them scalability (wrong), it's more cloud compatible (wrong) or the worst offender, it's more modern.
  - jordanbeiber 8y ago
    If you achieve this scale of teams suggested, it means you have a lot of implicit gains here as well, right? Such as higher quality, higher "velocity", separation of concern and hopefully a clear sense of ownership. These are some of the things in my opinion that allows you to scale.
    - lmm 8y ago
      > If you achieve this scale of teams suggested, it means you have a lot of implicit gains here as well, right? > Such as higher quality, higher "velocity", separation of concern and hopefully a clear sense of ownership. No, it means you pay a huge overhead. Quality and velocity both drop as your day-to-day development requires a lot more setup and faff to do anything, and counterintuitively so does separation of concerns as your interfaces become more rigid. Small organisations should do things that don't scale, turn their size into an advantage. If you think of your overhead as ax + bx^2 where x is the number of developers, microservices are a way to reduce b, but at the cost of a big increase to a. It makes sense when x is huge but not before. My litmus test would be: do you need to do multiple (unrelated) deployments of different services at once? If your organisation is small enough that you can get away with only deploying one thing at a time, you'll probably have less overhead if you work without microservices.
      - jordanbeiber 8y ago
        Which is why you’ll need the input and help from experienced sysadmins that like to do development, which actually is ”automation”. This done right reduces overhead and gives you an edge in repeatability and quality. Both are required to scale.
- jordanbeiber 8y ago
  Microservices requires proper, structured systems management. Most seem to think "devops" and microservices are ways to ignore this, when in fact the complete opposite applies. Make sure you have a true service delivery and service management pipeline in place where it is real easy for dev and ops to deploy and decommission services. Service metadata is key in my experience. Not too much though, just enough (such as service owner, deployment scopes, a release database, etc...). Nothing gets deployed without this metadata present, and make sure to automate the creation and keeping of these records. Make sure convention before configuration applies, in most cases this is doable. This stuff, in my experience, takes somewhere 6-12 months to put into place, including the actual automations/orchestration (be it the hashicorp stack, kubernetes, dc/os, triton, whatever). Do not even think about starting to move to prod before everyone have agreed upon the conventions and operational situation surrounding the stack and services. Other than this, it's just code and integrations. Business logic. =) In my last project we moved from 0 to 200 fully managed microservices using the hashicorp stack (plus a bunch of other stuff and homegrown things as well such as service metadata and release database/apis) and the biggest challenge was having everyone agree on the conventions. Convincing the developers that this type of house-keeping will be necessary, perhaps not today, or next month, but down the line took a few months. In the end it was a massive success that really accelerated the way we could deliver value to the business. When it comes to the data and ETL part you get a lot for free if the above is done decently. From my perspective the data scientists and ML guys need this stuff as well to be able to deploy and modify ETL flows and data pipelines at will. They'll most likely want to deploy some python, R and/or shiny apps as well! =) They (can) benefit greatly from getting integrated in the same automations Selling points achieved in above mentioned "transformation"/project: - 14.000 production deployments with 45 devs and 3 "ops" guys (dev, ops, devops?)" in about a year. ("But why!?" someone will ask! And I'll be happy to respond.) - Things started to happen, such as "could we just not provide this [insert awesome thing] to end users?" And what would have taken 6 months before could be pushed to canary days or even hours later. - One specific feature I can think about increased revenue with more than a million dollars per month, and it took one of the teams exactly two days to build and release. Granted, this was within a multibillion turnaround enterprise. Most businesses would benefit from this transformation, but it requires a blend of people and technology that probably is difficult to attain outside the proper "tech" industry. And my experience from within "small" tech is that focus sometimes is, understandably so, not situated around what I call systems management and service delivery. Phew, that went on... sorry! =)
somethingle234 8y ago
I worked in a company where micro services succeed. To succeed, the ratio has to make sense. We had more than 2,000 engineers with fewer than a few hundred services. So, of course, it made perfect sense. I've also seen a startup where it had 30 people and used the micro-service architecture. That didn't work very well. To make it worse, they used Node.js. It's like using Micro service wasn't difficult enough... they had to go with a dynamic-typed language.

I'm not convinced all microservices ventures are failures. Having worked in the space a bit as a founder/CTO of a vendor in that space I've just seen many examples of misguided attempts due to fashion / CV-driven development / hype driven development. The pattern is valid, just not for everyone at every time.

hjacobs 8y ago
ThoughtWorks even has a term for it: "Microservice envy" https://www.thoughtworks.com/radar/techniques/microservice-envy https://www.thoughtworks.com/radar/techniques/microservice-e...
ex_amazon_sde 8y ago
The good part of "microservices" is "services". The bad part is "micro".
- tyldum 8y ago
  When the definition turns to some kind of dogma, then it will fail.

lucidone 8y ago

I'm consulting on a micro services back end right now with mostly prior experience with monoliths. What is the selling point that drives companies down this direction? It's insane, and my client keeps trying to hire new developers and bring on more consultants to build this thing, but the amount of knowledge required is more than any one person can handle. I have similar issues with their choice of db (nosql) and its inflexibility.

stingraycharles 8y ago
The biggest reason for choosing microservices _should_ be scaling development teams: microservices allow multiple teams to work on different code bases, without stepping on each other's toes. What actually happens, tough, is that (uninformed) people choose it because they think it brings them scalability (wrong), it's more cloud compatible (wrong) or the worst offender, it's more modern.
- jordanbeiber 8y ago
  If you achieve this scale of teams suggested, it means you have a lot of implicit gains here as well, right? Such as higher quality, higher "velocity", separation of concern and hopefully a clear sense of ownership. These are some of the things in my opinion that allows you to scale.
  - lmm 8y ago
    > If you achieve this scale of teams suggested, it means you have a lot of implicit gains here as well, right? > Such as higher quality, higher "velocity", separation of concern and hopefully a clear sense of ownership. No, it means you pay a huge overhead. Quality and velocity both drop as your day-to-day development requires a lot more setup and faff to do anything, and counterintuitively so does separation of concerns as your interfaces become more rigid. Small organisations should do things that don't scale, turn their size into an advantage. If you think of your overhead as ax + bx^2 where x is the number of developers, microservices are a way to reduce b, but at the cost of a big increase to a. It makes sense when x is huge but not before. My litmus test would be: do you need to do multiple (unrelated) deployments of different services at once? If your organisation is small enough that you can get away with only deploying one thing at a time, you'll probably have less overhead if you work without microservices.
    - jordanbeiber 8y ago
      Which is why you’ll need the input and help from experienced sysadmins that like to do development, which actually is ”automation”. This done right reduces overhead and gives you an edge in repeatability and quality. Both are required to scale.
jordanbeiber 8y ago
Microservices requires proper, structured systems management. Most seem to think "devops" and microservices are ways to ignore this, when in fact the complete opposite applies. Make sure you have a true service delivery and service management pipeline in place where it is real easy for dev and ops to deploy and decommission services. Service metadata is key in my experience. Not too much though, just enough (such as service owner, deployment scopes, a release database, etc...). Nothing gets deployed without this metadata present, and make sure to automate the creation and keeping of these records. Make sure convention before configuration applies, in most cases this is doable. This stuff, in my experience, takes somewhere 6-12 months to put into place, including the actual automations/orchestration (be it the hashicorp stack, kubernetes, dc/os, triton, whatever). Do not even think about starting to move to prod before everyone have agreed upon the conventions and operational situation surrounding the stack and services. Other than this, it's just code and integrations. Business logic. =) In my last project we moved from 0 to 200 fully managed microservices using the hashicorp stack (plus a bunch of other stuff and homegrown things as well such as service metadata and release database/apis) and the biggest challenge was having everyone agree on the conventions. Convincing the developers that this type of house-keeping will be necessary, perhaps not today, or next month, but down the line took a few months. In the end it was a massive success that really accelerated the way we could deliver value to the business. When it comes to the data and ETL part you get a lot for free if the above is done decently. From my perspective the data scientists and ML guys need this stuff as well to be able to deploy and modify ETL flows and data pipelines at will. They'll most likely want to deploy some python, R and/or shiny apps as well! =) They (can) benefit greatly from getting integrated in the same automations Selling points achieved in above mentioned "transformation"/project: - 14.000 production deployments with 45 devs and 3 "ops" guys (dev, ops, devops?)" in about a year. ("But why!?" someone will ask! And I'll be happy to respond.) - Things started to happen, such as "could we just not provide this [insert awesome thing] to end users?" And what would have taken 6 months before could be pushed to canary days or even hours later. - One specific feature I can think about increased revenue with more than a million dollars per month, and it took one of the teams exactly two days to build and release. Granted, this was within a multibillion turnaround enterprise. Most businesses would benefit from this transformation, but it requires a blend of people and technology that probably is difficult to attain outside the proper "tech" industry. And my experience from within "small" tech is that focus sometimes is, understandably so, not situated around what I call systems management and service delivery. Phew, that went on... sorry! =)

somethingle234 8y ago

I worked in a company where micro services succeed. To succeed, the ratio has to make sense. We had more than 2,000 engineers with fewer than a few hundred services. So, of course, it made perfect sense. I've also seen a startup where it had 30 people and used the micro-service architecture. That didn't work very well. To make it worse, they used Node.js. It's like using Micro service wasn't difficult enough... they had to go with a dynamic-typed language.