33 ms·

I've been merging microservices back into the monolith

282 points by rmason 6y ago

proverbialbunny 6y ago
The bit about Conway's Law >Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure. is mind blowing. How did I not notice this the whole time? Awesome article. <3
- eightysixfour 6y ago
  A lot of the job of effective large organizational software/management consultants is understanding where there are mismatches between org structure and software architecture and helping to drive changes to get those in alignment. A big part of this is finding better places to put the interfaces (whether they are human or API), because they're usually holdovers from the past and no longer related to the business structure or architecture.
- steerpike 6y ago
  It's colloquially described as "shipping the org chart". Which I always liked as a way to describe it to a wider audience.
- ako 6y ago
  You can use this to change the architecture of your software: change the structure of your teams, and your software architecture will follow.
- layoric 6y ago
  Here is a post by Martin Fowler James Lewis in 2014 pointing out the same thing [0]. A lot of the thoughts expressed there I think are still relevant but I guess we had to wait for this hype cycle to play out? [0] https://martinfowler.com/articles/microservices.html#OrganizedAroundBusinessCapabilities https://martinfowler.com/articles/microservices.html#Organiz...
- baq 6y ago
  note the law doesn't specify if this is by necessity/on purpose - but designing a system which doesn't do that is usually setting yourself up for failure.
- vishnugupta 6y ago
  It takes one to "experience" this phenomenon from the inside of an org to appreciate it. Analogous to how one would have to work on largish codebase/architectures to appreciate design patterns. I have a corollary to Conway's Law. The number of network hops an end customer's request goes through before it is served is directly proportional to the number of teams in that org. A native English speaker could help me simplify that sentence :-)
suyash 6y ago
Glad to see a reality check post around the hype of Microservices.
eightysixfour 6y ago
This article absolutely nails it. Finally, recognition that microservices are a technical tool for solving a people and organizational problem. We need to understand that a lot of "new" technology paradigms (especially those coming down from FAANG or other large organizations) are often designed to solve the organizational problems of operating at scale, not to provide some kind of technical panacea we should all aspire to built towards.
- burade 6y ago
  People are godamn hopeless. I just saw a guy trying to argue that writing functional-style Java is the "new" way to write Java, and that not using functional interfaces means you're writing "old" style code.
  - akmarinov 6y ago
    I don’t know about Java, but we’re seeing a shift to functional in the mobile space, where non functional code is definitely “old” code. In Swift with Combine and in Kotlin with their thing.
    - burade 6y ago
      "Old" in what sense though? In the style/fashion sense?
      - akmarinov 6y ago
        In the “writing non-functional code in 2 years will be the same as writing Objective-C today” sense.
      - rualca 6y ago
        > "Old" in what sense though? In the style/fashion sense? If you take the time to look into it, you'll discover that the functional style is only a change in the "fashion sense" to those who are totally oblivious to their advantages. I get how the pedantic take on monads turns people away from the functional side of programming, but you'd be hard pressed to find anything to criticize how returning either/result monads from promises is not a huge improvement over the boilerplate-rich/pure OO approach to Java.
Pamar 6y ago
From what I understand (never actually worked with MS, so no hands-on experience) a "proper" MS architecture will also include separate data persistence layers for each service. I.e. if your service manages persisted data, it will access (and own) a dedicated storage instance which no other service has access to. Assuming it is really true, folding a microservice back into a monolith should also mean that the data are ported back into the main DB? The article does not seem to spend much time discussing this, though. So maybe it is one of those ideas that everyone conveniently decides to ignore when they actually start dealing with a real world problem?
- majormajor 6y ago
  Some services don't own any of their own data, so for them the question wouldn't arise. You could split them out without migrating data, and you could fold them in without it as well. Otherwise, yes, moving a service out of a monolith without moving its data leaves you in a pretty tricky and problematic place, though the reverse is much less true: a single service having multiple data layers for different things isn't really a big operational or organizational problem.
  - nthj 6y ago
    > a single service having multiple data layers for different things isn't really a big operational or organizational problem. Your statement is true, and I’d like to consider the product/feature side as well. Enterprise SAAS customers expect their Single Source of Truth to be, well, a single source. They want reports across all of the data they have put into a product. A feature that should be a couple of JOINs can quickly become pretty dicey when data is sliced across multiple data stores.
    - Pamar 6y ago
      This is exactly what always made me very skeptical about microservices (as I said, no actual experience so I might just change my mind if I start using them) - for any non-trivial problem reasoning about your data in a integrated and coherent way seems pretty daunting.
fourseventy 6y ago
Seems like microservices has moved past the "peak of inflated expectations" and into the "Trough of Disillusionment" phase of the hype lifecycle. Thank god. I'm tired of semi-technical product managers throwing around the term.
- DamnYuppie 6y ago
  I agree this is a good thing. Now that knowledge just needs to be accelerated up the corporate ladder to CTO's/CEO's.
- rualca 6y ago
  > Seems like microservices has moved past the "peak of inflated expectations" and into the "Trough of Disillusionment" phase of the hype lifecycle. The only change I've been witnessing with regards to microservices is where their critics place their personal goalpost. Microservices is a buzzword that is used as synonym for distributed systems and the evolution of service-oriented architectures after removing the constraints of rigid interface-related XML-based technologies like UDDI and WISDL in favour of ad-hoc interfaces. Some responsibilities are moved to dedicated services when the operations side justifies it, and services are developed based on key design principles to reflect lessons that have been learned throughout the past couple of decades. But even if the hype associated with a buzzword comes and goes, the concepts and its uses are still the same.
- pjmlp 6y ago
  Unfortunately containers are still into the "peak of inflated expectations".
bob1029 6y ago
Monolithic software maintained in a single computer/repository is so much better than software spread across many computers and repositories. There are some first-order arguments against a monolithic codebase, but the higher-order points will win out every time. I.e. the reason you want a separate codebase for each "concern" is probably because no one figured out how to work together as a team toward a common objective. This lack of teamwork will ultimately kill your project if you do not resolve it, so whether or not you do microservices in the expected way (i.e. HTTP RPC) will make no difference either way. If you still want "micro" services because it actually makes technological sense, you can build these at a lower level of abstraction under some /Services folder. Many languages have namespaces, which allow for boundless organizational schemes. Only for a lack of imagination would you fail to organize even the most complex of applications in this manner.
- paperpunk 6y ago
  Out of curiosity, what approaches do you use to manage zero-downtime deployments during system operation with a monolith?
  - gizzlon 6y ago
    Not parent, but how about a load ballancer and VM's? =) Or, if you have 1 machine I believe you can do things in Linux / BSD to start a new binary without killing ongoing connections to the old process. Edit: Oh, or you know, just don't? Maybe no-one will notice 5 secs of downtime :P depends ..
    - antoinealb 6y ago
      Even if you have only 1 machine, you could deploy a reverse proxy in front of it (think nginx, haproxy and co). Then you can start your application on two different endpoints, and instruct your load balancer to go from one to the other. Nginx supports live reloading of configuration, I think that should work.
    - spockz 6y ago
      You can use SO_REUSEADDR and SO_REUSEPORT to bind multiple instances to the same port. This means you will be using the Linux network stack as load balancer though.
posharma 6y ago
Oh the amount of time our industry wastes behind technology for the sake of technology! First, let's write tons of articles extolling the virtues of microservices. Then let's counter that with tons extolling the virtues of using microservices the right way. Hype followed by wisdom and then go back to hype again! When are we going to mature! Boring is good and predictable.
- nindalf 6y ago
  Evolution through natural selection depends on mutation. If we stopped all mutation, we would stop evolution too. We would be stuck in local optima, never trying to find a better global optimum. Not every new practice will work out. The bad ones will be weeded out by natural selection. But if we try to stop evolution by discouraging mutation, we will stagnate.
- hackerfromthefu 6y ago
  This century has been programming's Eternal September. So many programmers are too new to understand the huge amount of context for our industry, and it will probably not improve while the number of programmers continues in hypergrowth mode. Another way to think of it is that half the industry is still in the stage of not even knowing what they don't know.
  - onethought 6y ago
    Huge amount of context of our industry!? All 60 years is it! How do we manage! (Meanwhile over in the 1000s of years old industries...)
ajsharp 6y ago
The author correctly identifies the problem as part technical, part people. My shot-in-the-dark estimation is probably 70% of teams doing microservices do it to solve the people problem before (if ever) they need it to solve the technical problem. Technical solutions are _usually_ bad solutions for people problems, and architectural patterns are probably even worse solutions at solving the problem of human collaboration. It doesn't help that microservices are mostly a better-sounding name for "SOA, but smaller", that has grown in prominence mostly to sell you hosting for your very many microservices. Microservices takes one of the hardest and most important parts of of SOA (service boundaries) and replaces it with...smaller. Glad to see someone at a larger company publishing about this.
axegon_ 6y ago
"Future thinking is future trashing". Though I've had mixed feelings about this quote, in the case of the author this is probably valid. Yes, microservices are not a bad idea. They are in fact a great idea and they are needed SOMETIMES. But there is a time and place for everything. They are unnecessary and completely counterproductive and cumbersome unless you have a very large and complex system which needs to be distributed. If that is the case, of course, god speed. But if you can work without them you should not go for a microservice architecture: let's face it, they are hard to design, develop and in many cases a nightmare to debug when something goes wrong. Annoyingly the term "microservice" is yet another PR campaign gone horribly wrong. As a consequence it has become a buzzword like blockchain, AI, agile, etc. Just because FAANG is doing it, does not mean that your online shop for selling socks needs microservices in any shape or form. A short while ago a friend dragged me in as a side consultant for one of his clients. He owned a site which is basically craigslist for musical instruments and he was in the process of hiring a company to rebuild and subsequently modernize his site. Realistically the site has around 5k users/day tops and no more than a GB of traffc, and a mysql database which over the course of 15 years is less than 50GB. Basically a tiny site. The company he was about to hire had designed an architecture, which involved a large EKS cluster, 15 microservices, mysql, postgeres, elastic, memcache and redis, complicated grpc communication and their claims were that this architecture would make it the best in the business. OK, let's give them the benefit of a doubt, I advised him to ask them how much more performance would he gain out of that and if there are any other advantage over the typical vps with a webserver. Their response was "Infinitely more performant, you'll be able to scale to 100 million users per day and the system wouldn't feel a thing. That is what Google, Facebook and all other large companies are doing". Yeah... And does he expect to have 100 million users pour in at any point in time? Take a wild guess... Mind you, they asked for the same amount of money for a simple "buy now" website.
Animats 6y ago
A real issue with microservices is that you can no longer use the database system for transactional integrity. Microservices are fine for things you never have to back out. But if something can go wrong that requires a clean cancel of an entire transaction, now you need all the logic for that in the microservice calls. Which is very hard to get right for the cases where either side fails. (Today's interaction with phone support: Convincing Smart and Final that when their third-party delivery company had their systems go down, they didn't actually deliver the order, even though they generated a receipt for it.)
- gizzlon 6y ago
  Yeah, and while things like the Saga pattern[1] is interesting, it hasn't impressed me with its elegance or cleverness :[ (also, there seems to many real life cases where it would not work) 1: https://www.youtube.com/watch?v=xDuwrtwYHu8 https://www.youtube.com/watch?v=xDuwrtwYHu8
  - yeneek 6y ago
    I personally don't like saga pattern. For example when some operation fails, it should be reverted ob every service it went through. But what to do, when that revert fails? It's just matter of time when the system ends up in inconsistent state.
    - nicoburns 6y ago
      Is there a better option? And what is it? (genuinely curious, I've never had to deal with scale beyond where a single RDBMS could handle atomicity for me, and I'm curious about the best practice for handling this in bigger systems)
      - throwaway189262 6y ago
        Two phase commit is the traditional answer. Only extreme "enterprise" environments support it like JTA transaction managers in Java land. But 2PC is slow. The modern answer is to move scalability to your DB layer and have X instances of your monolith. Some NewSQL databases like CockroachDB and TiDB are reasonable choices as long as you avoid complex SQL
dpix 6y ago
This is a great article - One thing I always try and vouch for is that you don't need to go "all-in" on microservices. There is nothing wrong with having a main monolithic application with a few parts separated out into microservices that are necessary for performance or team management. The author hits the nail on the head at the end: If I could go back and redo our early microservice attempts, I would 100% start by focusing on all the "CPU bound" functionality first: image processing and resizing, thumbnail generation, PDF exporting, PDF importing, file versioning with rdiff, ZIP archive generation. I would have broken teams out along those boundaries, and have them create "pure" services that dealt with nothing but Inputs and Outputs (ie, no "integration databases", no "shared file systems") such that every other service could consume them while maintaining loose-coupling.
- bakedbeanz 6y ago
  > One thing I always try and vouch for is that you do need to go "all-in" on microservices Did you mean "don't need"? I agree with the rest of your post, but this first sentence confuses me
  - dpix 6y ago
    You are right! Sorry typo fixed
- gizzlon 6y ago
  yeah, but is that microservices? This is what we did the first place I worked, 15 years ago.
  - rusticpenn 6y ago
    It was called modular design. I think the main difference with micro services is the accessibility on the network. I would like to be corrected if I am wrong.
    - ArloL 6y ago
      Microservices are indepently deployable modules. Thus the same principles of modular design in a monolith apply. So, yes, you're correct with the network part.
loopz 6y ago
This is about a legacy system, and scaling back to a monolith, because of that need. Not holding my breath in anticipation this won't be misapplied and cargo culted.
throwaway9870 6y ago
We used services to great effect at the company I previously worked at. There was not a single engineer that ever voiced a concern over how things worked. A couple important notes: * We used a mono repo and services only had to talk to other services from the same commit (or branch). We didn't version service APIs, I think that would have been a huge waste of time for us. * Our tests did unit tests and integration tests. * We could use different languages as the problems required. We started one data processing service in Python and migrated it to Go when it became more CPU bound than we were comfortable with. We have an old but extremely reliable C++ service that had been running for almost a decade. * Each service ran in its own container. When a container/vm would run out of memory, we all didn't have to scramble to figure out who broke something. The person who owned the code for that service dealt with it. If a service started kicking out errors nobody but the owner had to be distracted. * We had a good team of architects that understood how to build things. We didn't split services across atomic boundaries. We knew how to build interfaces that handled errors robustly. If you don't know how to do these things, you service APIs can quickly become a mess. * The world is services. Just think about AWS and how it works. There is no such thing as a monolith, eventually you interface to services. Be it a weather API, a CI server, deployment system, a central logger, etc. Just because you didn't write all those services doesn't mean you don't have a service architecture. * We had really disjoint services that would have made zero sense to put together. For example, the web server that handled the front-end vs a back-end data system that refreshed a product database and built PDF assets based on the data in overnight batch runs. We actually had many of the later for different customers. We had services for central logging, for monitoring, for metric storage, etc. No way would I ever want to push all that into a single service and debug a GC issue or segfault. * If we had to hotfix a service, we could deploy just that service with zero concern the other services would have an issue. For us, services were a key part of building a very reliable system that we could update rapidly with minimal risk. We did this for almost a decade. We had a very good group of engineers and never did I hear one of them say services were holding us back. After this experience, I would say everyone one of them is an advocate for the appropriate use services.
- gizzlon 6y ago
  > * We used a mono repo and services only had to talk to other services from the same commit (or branch) Huh, that was a new one, can you expand? How did that work in practice? Did all the services update to the latest commit automatically?
3bodyProblem 6y ago
I don't really see what this article nails. So, if I understand the argument correctly. Microservices were used to increase development speed. While old pieces were left behind (legacy) new systems have come up to modernize (these are still microservices)? His team, now responsible for many, legacy unloved old microservices, are being merged back into a monolith. The real question is, is remerging all the code back into 1 application the right solution for their problems they had with stability. I think mental models are important, and having a huge blob of unrelated roles, makes sense to the current development team. But won't, just like the old situation, to the new developers. Perhaps it's just the clickbait article, but a better title would have been. "Homogenizing our wild-west legacy microservices". For me personally microservices was a god send, working on getting stuff done, instead of dealing with ancient code that doesn't reflect the current business anymore. I still buy in the thought of, if you can't develop a great monolith, you sure won't be great at building microservices. modular-monolith is the cool thing currently. Create a Monolith, without the shortcuts that create problems in the long run. Public interfaces are your most valuable pieces in the system. Worship them, code-review them, fight about them. Currently I could care less about the implementation itself. Does it solve our problems, is it fast enough, is it tested great, ship it. What language you used, architecture. database, I don't care. Just make sure it's a joy to use from the outside. If more developers would spend longer on thinking about the problems and less with throwing large amount of code that makes them feel smart. Making a microservice doesn't fix that problem. Think that what is missing is the stability that microservice are able to give in its most optimal form. Each service being the main source, the second it leaves the system it is stale reference data. Is stability important? use the old reference data, how fresh does your data really needs to be.
waynesonfire 6y ago
I really like the energy you bring to working on maintenance / legacy code. I also have some bananas that need straightening. Maybe next year you can show us one more time how to break up a monolith???
louwrentius 6y ago
This article should be No. 1 on HN. For a week.
wly_cdgr 6y ago
Because it's the closest you can come to something that feels like meaningful agency in that dreary an environment
ChicagoDave 6y ago
This article isn't about microservices. It's about a monolith that microservices were built on top of, then that code pulled back into the monolith. If you built serviced with its own data, built translation layers to handle deltas, you could move away from the monolith over time and have discreet services for your domain and sub-domains.
avl999 6y ago
It's insane to me that for a few years so many organizations were somehow convinced that MONSTROSITIES like trying to do distributed transactions using things like 2 phase commit just to make the almighty microservices god happy was an acceptable way of doing things. The first law of Microservice architecture should have been- do the microservice boundaries you have defined result in the need for cross service transactions? If so, slam on those breaks hard.
NicoJuicy 6y ago
Microservices taught me how to correctly apply DDD in a monolith. That's probably the most useful thing I picked up on the last year's.
mintyc 6y ago
There is an alternate view of what microservices should be. Check out the sample chapter for Monolith to microservice by Sam Newman. As described in this article, microservices seem like just a way to horizontally scale development and deployed implementations. Nothing like independently deployable network functions corresponding to business needs.
kovac 6y ago
"Not only were the various teams all competing for the same deployment resources, it meant that every time an "Incident" was declared, several teams' code had to get rolled-back; and, no team could deploy while an incident was being managed." So, how exactly did switching back to monolith solved the above problems? Seems to me like your actual problem is in QA and managing releases than application architecture whether it's towards or away from microservices. What I'm hearing is that you decided to fallback on relying on people to not push broken code out to production. Which is fine and all but what are you going to do if your team grows again? Rearchitect the system into microservices? A little heavy handed don't you think? In your particular case, I think, be it monolith or microservice architecture you need to work more on improving QA and streamlining releases. These, once automated, don't depend much on the number of developers. I've experienced this with a team of 3 as well as a team of about 15 (not 30 though). I don't think that the application architecture has anything to do with your problems one way or the other. Having said that, if the late night calls stopped after this exercise, then that's fine because no one should have to go through that!
- jdlshore 6y ago
  > So, how exactly did switching back to monolith solved the above problems? It seems that you didn’t read the whole article, but instead rushed to express your superiority. Switching to a monolith helped because they are a single team maintaining multiple services. Combining the services reduces their overhead. The deployment problems haven’t recurred because their team is the only one deploying the monolith (or at least, that’s what’s implied.)
  - kovac 6y ago
    I did read the article. Saying your now small team to be "careful" is not a real solution which I mentioned in my comment. What do you mean by expressing my superiority? Expressing my opinion backed by my own professional experience which happened to disagree with your own?
    - bobthebuilders 6y ago
banq 6y ago
microservices are a technical tool for solving business problem , business problem => people and organizational problem
slad 6y ago
I have worked at several companies with SOA. In fact, I was lead at one of the companies where we were breaking monolith to smaller services. We were having lots of issues with scalability with monolith. First we tried breaking to scale it horizontally by creating shards and routing users to different shards. That helped but with the growth we were seeing, we were back to scaling issue in 14 months. We broke it further into 4 shards and started working on SOA. After year and half we had dozens of smaller services and scaling was really smaller problem as it boiled down to scaling specific service. Over all, few points to add for without regards to SOA that I didn’t see here in threads - - smaller blast radius: every change is small and specific to service so easy to rollback and understand the impact - load tests: capacity management was relatively easy; small services with small dependencies - easier to update dependencies: java version updates was not huge project with every feature development on hold - autonomy: team had more autonomy as it didn’t require committee approvals - prolific design patterns: services could use different architectural patterns This obviously came with lot of other issues - latencies, cross service atomocity, logs correlation. But at the end I believe pros outweigh the cons and I would continue to use SOA pattern wherever I could. Industry has been trending towards microservices/lambdas which in my opinion take it too far. Finding that balance between Monolith and micro service is what works in my opinion.
- bertjk 6y ago
  In your opinion what is the difference between an SOA and microservice architecture? When does a microservice become just an SOA service or vice versa?
  - mr_nobody 6y ago
    I think the biggest difference is as defined by Martin Fowler: " When building communication structures between different processes, we've seen many products and approaches that stress putting significant smarts into the communication mechanism itself. A good example of this is the Enterprise Service Bus (ESB), where ESB products often include sophisticated facilities for message routing, choreography, transformation, and applying business rules. The microservice community favours an alternative approach: smart endpoints and dumb pipes. Applications built from microservices aim to be as decoupled and as cohesive as possible - they own their own domain logic and act more as filters in the classical Unix sense - receiving a request, applying logic as appropriate and producing a response. These are choreographed using simple RESTish protocols rather than complex protocols such as WS-Choreography or BPEL or orchestration by a central tool." https://martinfowler.com/articles/microservices.html#SmartEndpointsAndDumbPipes https://martinfowler.com/articles/microservices.html#SmartEn...
TeeWEE 6y ago
It's not about microservices or not, it's about domain boundaries, application architecture, coupling. Whether it runs in a service or in a monolith isn't the deciding factor of wether its good or bad. Architecture and design is. A 100 services app with 5000 lines of code each could be really badly designed and entangled, and a monolith with 500.0000 lines of code could be very well designed... It's about code quality, right abstractions, decoupling, cohesion. Correct domain modelling Etc. Microservices are just a tool, but you have to have a good design first.
- globular-toast 6y ago
  Yeah. One thing I've noticed is that people are really bad at working towards high-level, nebulous goals like "code quality, right abstractions, decoupling, cohesion". Keeping these goals in mind requires constant thought, analysis and decision making as each case is different and there's no universal solution. So people jump at simple, single-faceted goals like "normalise the database", "spin out microservices", "don't use HTML tables", "rewrite using Rust" etc. These involve no further thought, no continuous evaluation of nebulous targets like quality, no keeping up with moving goals. It explains quite a lot of problems with politics too. Choose from red or blue. No further thinking required. If even that's too difficult, just pick the one your parents/friends picked. Easy!
- konschubert 6y ago
  Yea. Once you have good domain boundaries you can use microservices to make the split more visible for less thoughtful devs and use the associated tooling (contract testing, ... ) to reduce operational coupling around deployments and tests. But aside from that, anything you can do with microservices you can also do in monoliths.
  - pydry 6y ago
    Some people think that if you start creating micro services you will magically just "get" those boundaries.
    - 6y ago
lmilcin 6y ago
I have been working in couple organizations where I have thoroughly explained why we should undo the "microservices" approach and go back to monolythic application. Basically, the problem is that in most cases going to microservices is being pushed by management to complete exclusion of understanding whether the organization is ready to implement it. Microservices require that you have mature approach to many topics. You need to really have nailed down automated deployments and you really need to have nailed down automatically creating development infrastructure. In one company I had a tool where I could log in, give a new project name, click couple parameters like namespace and a completely new microservice would be generated with BitBucket, Jira, automated deployments, confluence space, etc. If you need to spend time to configure anything individually for a project, to do deployments, etc., you are not yet ready to do microservices. So in all those cases where we have scaled back on microservices the developers switched from developing the application to being full time engaged with managing the application and its numerous configurations. In one of the applications we had 140 services rolled back to one. Before, preparing the deployment would take 2 weeks as the developers would be sending and compiling emails with information what needed to be done where. Then the deployment would take one day as an engineer would be executing all those runbook instructions. Each of the 140 services had its own version which made ensuring correct versions a separate problem. After the change where we have rolled it all into a single service under a single version, the entire thing took 2h. Still manually, but way cheaper and more reliably.
- emj 6y ago
  It would be nice to have some of those basic ideas that led to your thorough explaination of the pit falls of microservice development. What you write in this post is too general, sure if you have to release your microservices as one mega monolith, then it feels like an easy choice. When your application is not suited for microservices there are still things that can be easier: updating dependencies because of security is usually easier if the whole code base needs it. So it all depends on the application, what kind of development work does it need in a ten year frame, etc. I really do not recommend big monoliths, but the most important this is that it should be easy to start development, that is usually easier with microservices. I feel your pain the automation really only starts to matter when you are above 40 services in my view.
02020202 6y ago
an application that takes messages from a queue and sends them as emails is a microservice. anything more complicated thant that is just SoA and SoA is evil. stay mono my friends.
bullen 6y ago
I think most microservice implementations get it wrong. You're not supposed to run one service per machine or even process (that leads to orchestration hell and fragmentation cost at the loss of redundancy): The way you get all benefits of a microservice architecture is if you run all services on all machines: by VM sandboxing them all on many distributed machines f.ex. like this does: http://host.rupy.se http://host.rupy.se This requires your entire solution to be async.-to-async. end-to-end though. But performance, scalability and strength is unmatched: it becomes anti-fragile!
rahoulb 6y ago
My take on this (which I've not really thought through). With O-O programming - the code itself is fine. You're breaking it up into smaller objects, each well defined, each with self-contained state. Nice and easy. The problem is the behaviour of the system isn't defined in that code. Instead it's defined in the pattern of messages that are sent over time across different bits of code. And because it's temporal, that pattern isn't written into the actual code itself - but to understand what the system is doing, you need to understand that pattern. The same goes for micro-services. Each individual service can manage its individual state, it can be architected and designed in the best way for that particular requirement. But again, the behaviour of the system as a whole depends on how the different services interact (and just as importantly, fail - which is where transactions and other methods of coordinating across your code become important). So (and I write this as a dyed in the wool O-O programmer), in both cases, the surface problem is solved but all we do is move it into a harder to observe place.
staticelf 6y ago
I think microservices can work for large organizations with many teams. However, for smaller orgs and smaller teams it can for sure be a real killer. It's hard to keep track on different applications, with different dependencies, different deployments etc etc. Of course one could standardize things but then some of the benefits of using microservices to begin with disappears. I am a big believer in the big monolith but that is because I am almost always working in either a very small team or alone.
ryanthedev 6y ago
People should be more worried about writing non blocking IO code. Blocking a thread in the cloud is paying the provider to do nothing.
- zaarn 6y ago
  Using multiple threads also solves that problem. If one thread blocks on IO, others can have the CPU.
jiraticketmach 6y ago
Microservices were/are an attempt of standardizing software development practices at really large organizations. For example: your typical big bank has hundreds or thousands of teams (both internal and through professional services companies) developing all kinds of applications under different technologies/hardware. In this case due to domain/organizational scale it is not feasible to have a mono repo (or 2, or 3) with a giant codebase. And as a result the typical situation is an expensive hell full of code duplication, ad-hoc integration methods, nightmarish deployments and such. Microservices are helpful in situations like this (even if they are not perfect). But for a company with a couple of development teams and a domain that can be understood almost entirely by a couple of business analysts, it´s overkill.
glouwbug 6y ago
My last job had 43 microservices, one per SQL table. Best part, it was across 8 repos, and shared a core library. Even better, it was python, so imagine adding an additional functions argument to the core library, and updating the caller in 7 other repos, pushing 7 PRs, and it still breaking 3 weeks later in production because you missed a function argument in one of the repos

282 points by rmason 6y ago

The bit about Conway's Law >Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure. is mind blowing. How did I not notice this the whole time? Awesome article. <3

eightysixfour 6y ago
A lot of the job of effective large organizational software/management consultants is understanding where there are mismatches between org structure and software architecture and helping to drive changes to get those in alignment. A big part of this is finding better places to put the interfaces (whether they are human or API), because they're usually holdovers from the past and no longer related to the business structure or architecture.
steerpike 6y ago
It's colloquially described as "shipping the org chart". Which I always liked as a way to describe it to a wider audience.
ako 6y ago
You can use this to change the architecture of your software: change the structure of your teams, and your software architecture will follow.
layoric 6y ago
Here is a post by Martin Fowler James Lewis in 2014 pointing out the same thing [0]. A lot of the thoughts expressed there I think are still relevant but I guess we had to wait for this hype cycle to play out? [0] https://martinfowler.com/articles/microservices.html#OrganizedAroundBusinessCapabilities https://martinfowler.com/articles/microservices.html#Organiz...
baq 6y ago
note the law doesn't specify if this is by necessity/on purpose - but designing a system which doesn't do that is usually setting yourself up for failure.
vishnugupta 6y ago
It takes one to "experience" this phenomenon from the inside of an org to appreciate it. Analogous to how one would have to work on largish codebase/architectures to appreciate design patterns. I have a corollary to Conway's Law. The number of network hops an end customer's request goes through before it is served is directly proportional to the number of teams in that org. A native English speaker could help me simplify that sentence :-)

suyash 6y ago

Glad to see a reality check post around the hype of Microservices.

eightysixfour 6y ago

This article absolutely nails it. Finally, recognition that microservices are a technical tool for solving a people and organizational problem. We need to understand that a lot of "new" technology paradigms (especially those coming down from FAANG or other large organizations) are often designed to solve the organizational problems of operating at scale, not to provide some kind of technical panacea we should all aspire to built towards.

burade 6y ago
People are godamn hopeless. I just saw a guy trying to argue that writing functional-style Java is the "new" way to write Java, and that not using functional interfaces means you're writing "old" style code.
- akmarinov 6y ago
  I don’t know about Java, but we’re seeing a shift to functional in the mobile space, where non functional code is definitely “old” code. In Swift with Combine and in Kotlin with their thing.
  - burade 6y ago
    "Old" in what sense though? In the style/fashion sense?
    - akmarinov 6y ago
      In the “writing non-functional code in 2 years will be the same as writing Objective-C today” sense.
    - rualca 6y ago
      > "Old" in what sense though? In the style/fashion sense? If you take the time to look into it, you'll discover that the functional style is only a change in the "fashion sense" to those who are totally oblivious to their advantages. I get how the pedantic take on monads turns people away from the functional side of programming, but you'd be hard pressed to find anything to criticize how returning either/result monads from promises is not a huge improvement over the boilerplate-rich/pure OO approach to Java.

Pamar 6y ago

From what I understand (never actually worked with MS, so no hands-on experience) a "proper" MS architecture will also include separate data persistence layers for each service. I.e. if your service manages persisted data, it will access (and own) a dedicated storage instance which no other service has access to. Assuming it is really true, folding a microservice back into a monolith should also mean that the data are ported back into the main DB? The article does not seem to spend much time discussing this, though. So maybe it is one of those ideas that everyone conveniently decides to ignore when they actually start dealing with a real world problem?

majormajor 6y ago
Some services don't own any of their own data, so for them the question wouldn't arise. You could split them out without migrating data, and you could fold them in without it as well. Otherwise, yes, moving a service out of a monolith without moving its data leaves you in a pretty tricky and problematic place, though the reverse is much less true: a single service having multiple data layers for different things isn't really a big operational or organizational problem.
- nthj 6y ago
  > a single service having multiple data layers for different things isn't really a big operational or organizational problem. Your statement is true, and I’d like to consider the product/feature side as well. Enterprise SAAS customers expect their Single Source of Truth to be, well, a single source. They want reports across all of the data they have put into a product. A feature that should be a couple of JOINs can quickly become pretty dicey when data is sliced across multiple data stores.
  - Pamar 6y ago
    This is exactly what always made me very skeptical about microservices (as I said, no actual experience so I might just change my mind if I start using them) - for any non-trivial problem reasoning about your data in a integrated and coherent way seems pretty daunting.

fourseventy 6y ago

Seems like microservices has moved past the "peak of inflated expectations" and into the "Trough of Disillusionment" phase of the hype lifecycle. Thank god. I'm tired of semi-technical product managers throwing around the term.

DamnYuppie 6y ago
I agree this is a good thing. Now that knowledge just needs to be accelerated up the corporate ladder to CTO's/CEO's.
rualca 6y ago
> Seems like microservices has moved past the "peak of inflated expectations" and into the "Trough of Disillusionment" phase of the hype lifecycle. The only change I've been witnessing with regards to microservices is where their critics place their personal goalpost. Microservices is a buzzword that is used as synonym for distributed systems and the evolution of service-oriented architectures after removing the constraints of rigid interface-related XML-based technologies like UDDI and WISDL in favour of ad-hoc interfaces. Some responsibilities are moved to dedicated services when the operations side justifies it, and services are developed based on key design principles to reflect lessons that have been learned throughout the past couple of decades. But even if the hype associated with a buzzword comes and goes, the concepts and its uses are still the same.
pjmlp 6y ago
Unfortunately containers are still into the "peak of inflated expectations".

bob1029 6y ago

Monolithic software maintained in a single computer/repository is so much better than software spread across many computers and repositories. There are some first-order arguments against a monolithic codebase, but the higher-order points will win out every time. I.e. the reason you want a separate codebase for each "concern" is probably because no one figured out how to work together as a team toward a common objective. This lack of teamwork will ultimately kill your project if you do not resolve it, so whether or not you do microservices in the expected way (i.e. HTTP RPC) will make no difference either way. If you still want "micro" services because it actually makes technological sense, you can build these at a lower level of abstraction under some /Services folder. Many languages have namespaces, which allow for boundless organizational schemes. Only for a lack of imagination would you fail to organize even the most complex of applications in this manner.

paperpunk 6y ago
Out of curiosity, what approaches do you use to manage zero-downtime deployments during system operation with a monolith?
- gizzlon 6y ago
  Not parent, but how about a load ballancer and VM's? =) Or, if you have 1 machine I believe you can do things in Linux / BSD to start a new binary without killing ongoing connections to the old process. Edit: Oh, or you know, just don't? Maybe no-one will notice 5 secs of downtime :P depends ..
  - antoinealb 6y ago
    Even if you have only 1 machine, you could deploy a reverse proxy in front of it (think nginx, haproxy and co). Then you can start your application on two different endpoints, and instruct your load balancer to go from one to the other. Nginx supports live reloading of configuration, I think that should work.
  - spockz 6y ago
    You can use SO_REUSEADDR and SO_REUSEPORT to bind multiple instances to the same port. This means you will be using the Linux network stack as load balancer though.

posharma 6y ago

Oh the amount of time our industry wastes behind technology for the sake of technology! First, let's write tons of articles extolling the virtues of microservices. Then let's counter that with tons extolling the virtues of using microservices the right way. Hype followed by wisdom and then go back to hype again! When are we going to mature! Boring is good and predictable.

nindalf 6y ago
Evolution through natural selection depends on mutation. If we stopped all mutation, we would stop evolution too. We would be stuck in local optima, never trying to find a better global optimum. Not every new practice will work out. The bad ones will be weeded out by natural selection. But if we try to stop evolution by discouraging mutation, we will stagnate.
hackerfromthefu 6y ago
This century has been programming's Eternal September. So many programmers are too new to understand the huge amount of context for our industry, and it will probably not improve while the number of programmers continues in hypergrowth mode. Another way to think of it is that half the industry is still in the stage of not even knowing what they don't know.
- onethought 6y ago
  Huge amount of context of our industry!? All 60 years is it! How do we manage! (Meanwhile over in the 1000s of years old industries...)

ajsharp 6y ago

The author correctly identifies the problem as part technical, part people. My shot-in-the-dark estimation is probably 70% of teams doing microservices do it to solve the people problem before (if ever) they need it to solve the technical problem. Technical solutions are _usually_ bad solutions for people problems, and architectural patterns are probably even worse solutions at solving the problem of human collaboration. It doesn't help that microservices are mostly a better-sounding name for "SOA, but smaller", that has grown in prominence mostly to sell you hosting for your very many microservices. Microservices takes one of the hardest and most important parts of of SOA (service boundaries) and replaces it with...smaller. Glad to see someone at a larger company publishing about this.

axegon_ 6y ago

"Future thinking is future trashing". Though I've had mixed feelings about this quote, in the case of the author this is probably valid. Yes, microservices are not a bad idea. They are in fact a great idea and they are needed SOMETIMES. But there is a time and place for everything. They are unnecessary and completely counterproductive and cumbersome unless you have a very large and complex system which needs to be distributed. If that is the case, of course, god speed. But if you can work without them you should not go for a microservice architecture: let's face it, they are hard to design, develop and in many cases a nightmare to debug when something goes wrong. Annoyingly the term "microservice" is yet another PR campaign gone horribly wrong. As a consequence it has become a buzzword like blockchain, AI, agile, etc. Just because FAANG is doing it, does not mean that your online shop for selling socks needs microservices in any shape or form. A short while ago a friend dragged me in as a side consultant for one of his clients. He owned a site which is basically craigslist for musical instruments and he was in the process of hiring a company to rebuild and subsequently modernize his site. Realistically the site has around 5k users/day tops and no more than a GB of traffc, and a mysql database which over the course of 15 years is less than 50GB. Basically a tiny site. The company he was about to hire had designed an architecture, which involved a large EKS cluster, 15 microservices, mysql, postgeres, elastic, memcache and redis, complicated grpc communication and their claims were that this architecture would make it the best in the business. OK, let's give them the benefit of a doubt, I advised him to ask them how much more performance would he gain out of that and if there are any other advantage over the typical vps with a webserver. Their response was "Infinitely more performant, you'll be able to scale to 100 million users per day and the system wouldn't feel a thing. That is what Google, Facebook and all other large companies are doing". Yeah... And does he expect to have 100 million users pour in at any point in time? Take a wild guess... Mind you, they asked for the same amount of money for a simple "buy now" website.

Animats 6y ago

A real issue with microservices is that you can no longer use the database system for transactional integrity. Microservices are fine for things you never have to back out. But if something can go wrong that requires a clean cancel of an entire transaction, now you need all the logic for that in the microservice calls. Which is very hard to get right for the cases where either side fails. (Today's interaction with phone support: Convincing Smart and Final that when their third-party delivery company had their systems go down, they didn't actually deliver the order, even though they generated a receipt for it.)

gizzlon 6y ago
Yeah, and while things like the Saga pattern[1] is interesting, it hasn't impressed me with its elegance or cleverness :[ (also, there seems to many real life cases where it would not work) 1: https://www.youtube.com/watch?v=xDuwrtwYHu8 https://www.youtube.com/watch?v=xDuwrtwYHu8
- yeneek 6y ago
  I personally don't like saga pattern. For example when some operation fails, it should be reverted ob every service it went through. But what to do, when that revert fails? It's just matter of time when the system ends up in inconsistent state.
  - nicoburns 6y ago
    Is there a better option? And what is it? (genuinely curious, I've never had to deal with scale beyond where a single RDBMS could handle atomicity for me, and I'm curious about the best practice for handling this in bigger systems)
    - throwaway189262 6y ago
      Two phase commit is the traditional answer. Only extreme "enterprise" environments support it like JTA transaction managers in Java land. But 2PC is slow. The modern answer is to move scalability to your DB layer and have X instances of your monolith. Some NewSQL databases like CockroachDB and TiDB are reasonable choices as long as you avoid complex SQL

dpix 6y ago

This is a great article - One thing I always try and vouch for is that you don't need to go "all-in" on microservices. There is nothing wrong with having a main monolithic application with a few parts separated out into microservices that are necessary for performance or team management. The author hits the nail on the head at the end: If I could go back and redo our early microservice attempts, I would 100% start by focusing on all the "CPU bound" functionality first: image processing and resizing, thumbnail generation, PDF exporting, PDF importing, file versioning with rdiff, ZIP archive generation. I would have broken teams out along those boundaries, and have them create "pure" services that dealt with nothing but Inputs and Outputs (ie, no "integration databases", no "shared file systems") such that every other service could consume them while maintaining loose-coupling.

bakedbeanz 6y ago
> One thing I always try and vouch for is that you do need to go "all-in" on microservices Did you mean "don't need"? I agree with the rest of your post, but this first sentence confuses me
- dpix 6y ago
  You are right! Sorry typo fixed
gizzlon 6y ago
yeah, but is that microservices? This is what we did the first place I worked, 15 years ago.
- rusticpenn 6y ago
  It was called modular design. I think the main difference with micro services is the accessibility on the network. I would like to be corrected if I am wrong.
  - ArloL 6y ago
    Microservices are indepently deployable modules. Thus the same principles of modular design in a monolith apply. So, yes, you're correct with the network part.

loopz 6y ago

This is about a legacy system, and scaling back to a monolith, because of that need. Not holding my breath in anticipation this won't be misapplied and cargo culted.

throwaway9870 6y ago

We used services to great effect at the company I previously worked at. There was not a single engineer that ever voiced a concern over how things worked. A couple important notes: * We used a mono repo and services only had to talk to other services from the same commit (or branch). We didn't version service APIs, I think that would have been a huge waste of time for us. * Our tests did unit tests and integration tests. * We could use different languages as the problems required. We started one data processing service in Python and migrated it to Go when it became more CPU bound than we were comfortable with. We have an old but extremely reliable C++ service that had been running for almost a decade. * Each service ran in its own container. When a container/vm would run out of memory, we all didn't have to scramble to figure out who broke something. The person who owned the code for that service dealt with it. If a service started kicking out errors nobody but the owner had to be distracted. * We had a good team of architects that understood how to build things. We didn't split services across atomic boundaries. We knew how to build interfaces that handled errors robustly. If you don't know how to do these things, you service APIs can quickly become a mess. * The world is services. Just think about AWS and how it works. There is no such thing as a monolith, eventually you interface to services. Be it a weather API, a CI server, deployment system, a central logger, etc. Just because you didn't write all those services doesn't mean you don't have a service architecture. * We had really disjoint services that would have made zero sense to put together. For example, the web server that handled the front-end vs a back-end data system that refreshed a product database and built PDF assets based on the data in overnight batch runs. We actually had many of the later for different customers. We had services for central logging, for monitoring, for metric storage, etc. No way would I ever want to push all that into a single service and debug a GC issue or segfault. * If we had to hotfix a service, we could deploy just that service with zero concern the other services would have an issue. For us, services were a key part of building a very reliable system that we could update rapidly with minimal risk. We did this for almost a decade. We had a very good group of engineers and never did I hear one of them say services were holding us back. After this experience, I would say everyone one of them is an advocate for the appropriate use services.

gizzlon 6y ago
> * We used a mono repo and services only had to talk to other services from the same commit (or branch) Huh, that was a new one, can you expand? How did that work in practice? Did all the services update to the latest commit automatically?

3bodyProblem 6y ago

I don't really see what this article nails. So, if I understand the argument correctly. Microservices were used to increase development speed. While old pieces were left behind (legacy) new systems have come up to modernize (these are still microservices)? His team, now responsible for many, legacy unloved old microservices, are being merged back into a monolith. The real question is, is remerging all the code back into 1 application the right solution for their problems they had with stability. I think mental models are important, and having a huge blob of unrelated roles, makes sense to the current development team. But won't, just like the old situation, to the new developers. Perhaps it's just the clickbait article, but a better title would have been. "Homogenizing our wild-west legacy microservices". For me personally microservices was a god send, working on getting stuff done, instead of dealing with ancient code that doesn't reflect the current business anymore. I still buy in the thought of, if you can't develop a great monolith, you sure won't be great at building microservices. modular-monolith is the cool thing currently. Create a Monolith, without the shortcuts that create problems in the long run. Public interfaces are your most valuable pieces in the system. Worship them, code-review them, fight about them. Currently I could care less about the implementation itself. Does it solve our problems, is it fast enough, is it tested great, ship it. What language you used, architecture. database, I don't care. Just make sure it's a joy to use from the outside. If more developers would spend longer on thinking about the problems and less with throwing large amount of code that makes them feel smart. Making a microservice doesn't fix that problem. Think that what is missing is the stability that microservice are able to give in its most optimal form. Each service being the main source, the second it leaves the system it is stale reference data. Is stability important? use the old reference data, how fresh does your data really needs to be.

waynesonfire 6y ago

I really like the energy you bring to working on maintenance / legacy code. I also have some bananas that need straightening. Maybe next year you can show us one more time how to break up a monolith???

louwrentius 6y ago

This article should be No. 1 on HN. For a week.

wly_cdgr 6y ago

Because it's the closest you can come to something that feels like meaningful agency in that dreary an environment

ChicagoDave 6y ago

This article isn't about microservices. It's about a monolith that microservices were built on top of, then that code pulled back into the monolith. If you built serviced with its own data, built translation layers to handle deltas, you could move away from the monolith over time and have discreet services for your domain and sub-domains.

avl999 6y ago

It's insane to me that for a few years so many organizations were somehow convinced that MONSTROSITIES like trying to do distributed transactions using things like 2 phase commit just to make the almighty microservices god happy was an acceptable way of doing things. The first law of Microservice architecture should have been- do the microservice boundaries you have defined result in the need for cross service transactions? If so, slam on those breaks hard.

NicoJuicy 6y ago

Microservices taught me how to correctly apply DDD in a monolith. That's probably the most useful thing I picked up on the last year's.

mintyc 6y ago

There is an alternate view of what microservices should be. Check out the sample chapter for Monolith to microservice by Sam Newman. As described in this article, microservices seem like just a way to horizontally scale development and deployed implementations. Nothing like independently deployable network functions corresponding to business needs.

kovac 6y ago

"Not only were the various teams all competing for the same deployment resources, it meant that every time an "Incident" was declared, several teams' code had to get rolled-back; and, no team could deploy while an incident was being managed." So, how exactly did switching back to monolith solved the above problems? Seems to me like your actual problem is in QA and managing releases than application architecture whether it's towards or away from microservices. What I'm hearing is that you decided to fallback on relying on people to not push broken code out to production. Which is fine and all but what are you going to do if your team grows again? Rearchitect the system into microservices? A little heavy handed don't you think? In your particular case, I think, be it monolith or microservice architecture you need to work more on improving QA and streamlining releases. These, once automated, don't depend much on the number of developers. I've experienced this with a team of 3 as well as a team of about 15 (not 30 though). I don't think that the application architecture has anything to do with your problems one way or the other. Having said that, if the late night calls stopped after this exercise, then that's fine because no one should have to go through that!

jdlshore 6y ago
> So, how exactly did switching back to monolith solved the above problems? It seems that you didn’t read the whole article, but instead rushed to express your superiority. Switching to a monolith helped because they are a single team maintaining multiple services. Combining the services reduces their overhead. The deployment problems haven’t recurred because their team is the only one deploying the monolith (or at least, that’s what’s implied.)
- kovac 6y ago
  I did read the article. Saying your now small team to be "careful" is not a real solution which I mentioned in my comment. What do you mean by expressing my superiority? Expressing my opinion backed by my own professional experience which happened to disagree with your own?
  - bobthebuilders 6y ago

banq 6y ago

microservices are a technical tool for solving business problem , business problem => people and organizational problem

slad 6y ago

I have worked at several companies with SOA. In fact, I was lead at one of the companies where we were breaking monolith to smaller services. We were having lots of issues with scalability with monolith. First we tried breaking to scale it horizontally by creating shards and routing users to different shards. That helped but with the growth we were seeing, we were back to scaling issue in 14 months. We broke it further into 4 shards and started working on SOA. After year and half we had dozens of smaller services and scaling was really smaller problem as it boiled down to scaling specific service. Over all, few points to add for without regards to SOA that I didn’t see here in threads - - smaller blast radius: every change is small and specific to service so easy to rollback and understand the impact - load tests: capacity management was relatively easy; small services with small dependencies - easier to update dependencies: java version updates was not huge project with every feature development on hold - autonomy: team had more autonomy as it didn’t require committee approvals - prolific design patterns: services could use different architectural patterns This obviously came with lot of other issues - latencies, cross service atomocity, logs correlation. But at the end I believe pros outweigh the cons and I would continue to use SOA pattern wherever I could. Industry has been trending towards microservices/lambdas which in my opinion take it too far. Finding that balance between Monolith and micro service is what works in my opinion.

bertjk 6y ago
In your opinion what is the difference between an SOA and microservice architecture? When does a microservice become just an SOA service or vice versa?
- mr_nobody 6y ago
  I think the biggest difference is as defined by Martin Fowler: " When building communication structures between different processes, we've seen many products and approaches that stress putting significant smarts into the communication mechanism itself. A good example of this is the Enterprise Service Bus (ESB), where ESB products often include sophisticated facilities for message routing, choreography, transformation, and applying business rules. The microservice community favours an alternative approach: smart endpoints and dumb pipes. Applications built from microservices aim to be as decoupled and as cohesive as possible - they own their own domain logic and act more as filters in the classical Unix sense - receiving a request, applying logic as appropriate and producing a response. These are choreographed using simple RESTish protocols rather than complex protocols such as WS-Choreography or BPEL or orchestration by a central tool." https://martinfowler.com/articles/microservices.html#SmartEndpointsAndDumbPipes https://martinfowler.com/articles/microservices.html#SmartEn...

TeeWEE 6y ago

It's not about microservices or not, it's about domain boundaries, application architecture, coupling. Whether it runs in a service or in a monolith isn't the deciding factor of wether its good or bad. Architecture and design is. A 100 services app with 5000 lines of code each could be really badly designed and entangled, and a monolith with 500.0000 lines of code could be very well designed... It's about code quality, right abstractions, decoupling, cohesion. Correct domain modelling Etc. Microservices are just a tool, but you have to have a good design first.

globular-toast 6y ago
Yeah. One thing I've noticed is that people are really bad at working towards high-level, nebulous goals like "code quality, right abstractions, decoupling, cohesion". Keeping these goals in mind requires constant thought, analysis and decision making as each case is different and there's no universal solution. So people jump at simple, single-faceted goals like "normalise the database", "spin out microservices", "don't use HTML tables", "rewrite using Rust" etc. These involve no further thought, no continuous evaluation of nebulous targets like quality, no keeping up with moving goals. It explains quite a lot of problems with politics too. Choose from red or blue. No further thinking required. If even that's too difficult, just pick the one your parents/friends picked. Easy!
konschubert 6y ago
Yea. Once you have good domain boundaries you can use microservices to make the split more visible for less thoughtful devs and use the associated tooling (contract testing, ... ) to reduce operational coupling around deployments and tests. But aside from that, anything you can do with microservices you can also do in monoliths.
- pydry 6y ago
  Some people think that if you start creating micro services you will magically just "get" those boundaries.
  - 6y ago

lmilcin 6y ago

I have been working in couple organizations where I have thoroughly explained why we should undo the "microservices" approach and go back to monolythic application. Basically, the problem is that in most cases going to microservices is being pushed by management to complete exclusion of understanding whether the organization is ready to implement it. Microservices require that you have mature approach to many topics. You need to really have nailed down automated deployments and you really need to have nailed down automatically creating development infrastructure. In one company I had a tool where I could log in, give a new project name, click couple parameters like namespace and a completely new microservice would be generated with BitBucket, Jira, automated deployments, confluence space, etc. If you need to spend time to configure anything individually for a project, to do deployments, etc., you are not yet ready to do microservices. So in all those cases where we have scaled back on microservices the developers switched from developing the application to being full time engaged with managing the application and its numerous configurations. In one of the applications we had 140 services rolled back to one. Before, preparing the deployment would take 2 weeks as the developers would be sending and compiling emails with information what needed to be done where. Then the deployment would take one day as an engineer would be executing all those runbook instructions. Each of the 140 services had its own version which made ensuring correct versions a separate problem. After the change where we have rolled it all into a single service under a single version, the entire thing took 2h. Still manually, but way cheaper and more reliably.

emj 6y ago
It would be nice to have some of those basic ideas that led to your thorough explaination of the pit falls of microservice development. What you write in this post is too general, sure if you have to release your microservices as one mega monolith, then it feels like an easy choice. When your application is not suited for microservices there are still things that can be easier: updating dependencies because of security is usually easier if the whole code base needs it. So it all depends on the application, what kind of development work does it need in a ten year frame, etc. I really do not recommend big monoliths, but the most important this is that it should be easy to start development, that is usually easier with microservices. I feel your pain the automation really only starts to matter when you are above 40 services in my view.

02020202 6y ago

an application that takes messages from a queue and sends them as emails is a microservice. anything more complicated thant that is just SoA and SoA is evil. stay mono my friends.

bullen 6y ago

I think most microservice implementations get it wrong. You're not supposed to run one service per machine or even process (that leads to orchestration hell and fragmentation cost at the loss of redundancy): The way you get all benefits of a microservice architecture is if you run all services on all machines: by VM sandboxing them all on many distributed machines f.ex. like this does: http://host.rupy.se http://host.rupy.se This requires your entire solution to be async.-to-async. end-to-end though. But performance, scalability and strength is unmatched: it becomes anti-fragile!

rahoulb 6y ago

My take on this (which I've not really thought through). With O-O programming - the code itself is fine. You're breaking it up into smaller objects, each well defined, each with self-contained state. Nice and easy. The problem is the behaviour of the system isn't defined in that code. Instead it's defined in the pattern of messages that are sent over time across different bits of code. And because it's temporal, that pattern isn't written into the actual code itself - but to understand what the system is doing, you need to understand that pattern. The same goes for micro-services. Each individual service can manage its individual state, it can be architected and designed in the best way for that particular requirement. But again, the behaviour of the system as a whole depends on how the different services interact (and just as importantly, fail - which is where transactions and other methods of coordinating across your code become important). So (and I write this as a dyed in the wool O-O programmer), in both cases, the surface problem is solved but all we do is move it into a harder to observe place.

staticelf 6y ago

I think microservices can work for large organizations with many teams. However, for smaller orgs and smaller teams it can for sure be a real killer. It's hard to keep track on different applications, with different dependencies, different deployments etc etc. Of course one could standardize things but then some of the benefits of using microservices to begin with disappears. I am a big believer in the big monolith but that is because I am almost always working in either a very small team or alone.

ryanthedev 6y ago

People should be more worried about writing non blocking IO code. Blocking a thread in the cloud is paying the provider to do nothing.

zaarn 6y ago
Using multiple threads also solves that problem. If one thread blocks on IO, others can have the CPU.

jiraticketmach 6y ago

Microservices were/are an attempt of standardizing software development practices at really large organizations. For example: your typical big bank has hundreds or thousands of teams (both internal and through professional services companies) developing all kinds of applications under different technologies/hardware. In this case due to domain/organizational scale it is not feasible to have a mono repo (or 2, or 3) with a giant codebase. And as a result the typical situation is an expensive hell full of code duplication, ad-hoc integration methods, nightmarish deployments and such. Microservices are helpful in situations like this (even if they are not perfect). But for a company with a couple of development teams and a domain that can be understood almost entirely by a couple of business analysts, it´s overkill.

glouwbug 6y ago

My last job had 43 microservices, one per SQL table. Best part, it was across 8 repos, and shared a core library. Even better, it was python, so imagine adding an additional functions argument to the core library, and updating the caller in 7 other repos, pushing 7 PRs, and it still breaking 3 weeks later in production because you missed a function argument in one of the repos