8 ms·

> A huge part of my career has been splitting monoliths into cohesive smaller codebases Did that ... create business value for your company? > Eg, currently w

> A huge part of my career has been splitting monoliths into cohesive smaller codebases Did that ... create business value for your company? > Eg, currently we have user profiles (user id > the users names, email etc) in one microservice, relationships between users (user id > other user ids) in another. This sounds like a parody of microservices.

bedobi 6y ago
Yes, it did create business value. * our enterprise db was bursting at the seams containing Literally Everything. Now, every part of the split up monolith has it's own self contained data store tailored to what is appropriate for that particular thing. (some use MariaDB, others Redis etc etc) * developing, building, testing and deploying took ages. Eg if I only needed to capture some new detail about a business partner user (eg their mfa preference app vs sms) I would still have to do deal with the unwieldy monolith. Now, I can do it in the dedicated business partner user service, which is much easier and faster. * the whole monolith, including business partner facing operations, could go down because of issues to do with completely unrelated, non critical things like eg internal staff vacation hours. I could go on. As for the different services I described, if most of the callers did need both pieces of data it would have made sense to combine them into a single service. But, overwhelmingly, callers are interested in one piece of data or the other, and the load profile, tolerance for staleness in caching etc etc for each of the two services are vastly, vastly different. This is why we chose to split the two into different services. The few callers that do need to obtain both pieces of data just make concurrent calls to both and them zip them into a single result.
- candiddevmike 6y ago
  > * our enterprise db was bursting at the seams containing Literally Everything. Now, every part of the split up monolith has it's own self contained data store tailored to what is appropriate for that particular thing. (some use MariaDB, others Redis etc etc) Why do you consider an enterprise DB "bursting at the seams" to be a bad thing? Isn't that what enterprise DBs are built for? Seems like you traded having everything in one large database to having everything scattered in different databases. You probably sacrificed some referential integrity in the process. > * developing, building, testing and deploying took ages. Eg if I only needed to capture some new detail about a business partner user (eg their mfa preference app vs sms) I would still have to do deal with the unwieldy monolith. Now, I can do it in the dedicated business partner user service, which is much easier and faster. You traded a clean codebase with a solid toolchain for probably a template repository that you hope your users use or everyone is reinventing some kind of linting/testing/deployment toolchain for every microservice > * the whole monolith, including business partner facing operations, could go down because of issues to do with completely unrelated, non critical things like eg internal staff vacation hours. This could apply to any software. Sure, a monolith can have a large blast radius, but I can guarantee one of your microservices is critical path and would cause the same outage if it goes offline. > The few callers that do need to obtain both pieces of data just make concurrent calls to both and them zip them into a single result. Almost like a database join?
  - bedobi 6y ago
    Who says the monolith we had was a clean codebase with a solid toolchain? It was a giant piece of software that had existed since the 90s, with hundreds of developers working on it at any given moment, thousands of people coming through it over the years, full of bizarre things that would make anyone scream and tear their hair out. (each of which may have been the right thing to do at the time, but combined over the years wasn't optimal to say the least) It's obvious nothing will convince you, but I maintain that eg internal vacation hours should not live in the same codebase as partner facing business critical things.
    - robertlagrant 6y ago
      > Who says the monolith we had was a clean codebase with a solid toolchain? You didn't. They're just trying to force the issue for some reason by straw manning your decision.
    - Nextgrid 6y ago
      You can very well have a bad monolith and still think micro services is a bad idea, and his concerns especially around referential integrity are perfectly valid and valuable. Maybe all you needed is to fix the existing monolith incrementally?
      - bedobi 6y ago
        Of course microservices should be sliced with referential integrity in mind. But a monolith by definition has everything, including utterly unrelated things that have absolutely nothing to do with each other, bundled up into a single, giant, well, monolith. What referential integrity could there possibly be between eg business partner user profiles and internal staff vacation hours?
        quickthrower2 6y ago
        There is nothing inherently bad about storing unrelated things in the same db, in general. There might be security issues but good DBs offer security grants that are granular enough to deal with this. Data sovereignty might be an issue forcing you to split, that depends on the domain and application requirements. But in general two things being unrelated isn’t a reason to split the DB and splitting the DB isn’t a reason to split into services. I’m not for or against microservices. I’m definitely for good reasoning though!
- yakshaving_jgt 6y ago
  Could you be specific about the size of your company and the size of the data you're processing? And could you also be more specific about the business value it created? Are you an investor in the business? Is it your own money on the line?
  - bedobi 6y ago
    Publicly listed, ~13 billion USD revenue, 22k employees, millions of b2b partners and users. When the monolith goes down, it's hundreds of thousands, sometimes millions of dollars of revenue lost both during and as a result of an outage. You really, really don't want a problem with eg internal staff vacation hours to do that.
    - yakshaving_jgt 6y ago
      Ok, interesting. That kind of scale is not exactly the norm though, so I don't agree with your implication that monoliths are always a bad idea.
      - dodobirdlord 6y ago
        I really don't know why anyone who wasn't so resource strapped as to have no alternative would want their $$$ services sharing a build-test-deploy-rollback lifecycle or database with the internal staff vacation hours tracker.
        mrgordon 6y ago
        It seems like the real solution here is to outsource vacation hour tracking to one of the HR apps for the purpose (see Workday) instead of having the domain expert engineers split the company’s application apart into micro services so they can have a custom vacation tracker.
        bedobi 6y ago
        It was just one of the worst examples. A more obvious case would be the different lines of business. They have very little to do with each other and shouldn't share code or data. Another is business partners vs end users (regular people) - there's a lot if reasons, including legal ones, why they shouldn't share code or data.
- gfodor 6y ago
  No offense, but someone coming in and splitting out a database into multiple bespoke data stores because the primary database is "bursting at the seams" is basically my nightmare. If your description of your reasoning is accurate, it's a completely broken mental model.
  - bedobi 6y ago
    Well, that's not what happened. The whole org from devs to the board were highly aware of the problems and knew and agreed what had to be done. I'm just one of the many people who architected and implemented the change.
kenrose 6y ago
> > A huge part of my career has been splitting monoliths into cohesive smaller codebases > Did that ... create business value for your company? Not OP, but the big value of splitting into microservices is isolation. In production, this isolation offers a limited blast radius in the case of an errant service. Also, independent scaling. Business value: improved reliability. In code, isolation lets development teams have a smaller and more focused domain / set of concerns to reason about (vs. the entirety of the monolith). Business value: Increased dev velocity.
- bedobi 6y ago
  Exactly this!
  - e-brake 6y ago
    Agreed, isolation is a good reason to opt for microservices. One app goes offline, others still accessible, and you can do isolated maintenance. That, and features that aren't directly related, are super lightweight (quick to build, run, low on memory and CPU) as little projects.
- whakim 6y ago
  In my experience, it's easy enough to have services which have a large blast radius themselves and can become points of failure for your entire ecosystem. I don't find this to be a huge point of difference from a monolith, although of course it depends a lot on what you're working on. To pick on the OP a bit here (sorry!), they said that their entire legacy system "could go down because of issues to do with completely unrelated, non critical things like eg internal staff vacation hours." To me that sounds like poorly written software regardless of architecture. I can't imagine a scenario in any of the recent codebases I've worked on (microservices and monoliths both) where errors in what sounds like an internal CRUD tool would cause an entire production application to crash. I find it even harder to imagine if the application has a halfway decent test suite. To add to that, when you have hundreds of services running around and something goes wrong, it ends up being a lot harder to track down exactly what's happening. So when you do get that critical error, oftentimes the downtime is worsened. As for dev velocity, I find the claims of the microservice gospel a little bit exaggerated. Your layers of nested services all talk to each other, and any of them could be a point of failure. This isn't really all that different from calling another function in your monolithic app - you've just distributed that function call across a network boundary. You still need to know the callee's API, and you'll still spend a decent amount of time trying to understand the ways that the callee might fail. But you've also created a huge amount of additional developer work whenever you need to do something that spans the boundaries of existing services. I think microservices certainly have their advantages but a lot of the simplistic claims made by their biggest proponents only hold up prima facie.
inshadows 6y ago
> Did that ... create business value for your company? Image what a piece of crap Linux kernel would be if it was developed with this mindset. :-) EDIT: The context is that there's constant code splitting code moving being done inside Linux.

bedobi 6y ago

Yes, it did create business value. * our enterprise db was bursting at the seams containing Literally Everything. Now, every part of the split up monolith has it's own self contained data store tailored to what is appropriate for that particular thing. (some use MariaDB, others Redis etc etc) * developing, building, testing and deploying took ages. Eg if I only needed to capture some new detail about a business partner user (eg their mfa preference app vs sms) I would still have to do deal with the unwieldy monolith. Now, I can do it in the dedicated business partner user service, which is much easier and faster. * the whole monolith, including business partner facing operations, could go down because of issues to do with completely unrelated, non critical things like eg internal staff vacation hours. I could go on. As for the different services I described, if most of the callers did need both pieces of data it would have made sense to combine them into a single service. But, overwhelmingly, callers are interested in one piece of data or the other, and the load profile, tolerance for staleness in caching etc etc for each of the two services are vastly, vastly different. This is why we chose to split the two into different services. The few callers that do need to obtain both pieces of data just make concurrent calls to both and them zip them into a single result.

candiddevmike 6y ago
> * our enterprise db was bursting at the seams containing Literally Everything. Now, every part of the split up monolith has it's own self contained data store tailored to what is appropriate for that particular thing. (some use MariaDB, others Redis etc etc) Why do you consider an enterprise DB "bursting at the seams" to be a bad thing? Isn't that what enterprise DBs are built for? Seems like you traded having everything in one large database to having everything scattered in different databases. You probably sacrificed some referential integrity in the process. > * developing, building, testing and deploying took ages. Eg if I only needed to capture some new detail about a business partner user (eg their mfa preference app vs sms) I would still have to do deal with the unwieldy monolith. Now, I can do it in the dedicated business partner user service, which is much easier and faster. You traded a clean codebase with a solid toolchain for probably a template repository that you hope your users use or everyone is reinventing some kind of linting/testing/deployment toolchain for every microservice > * the whole monolith, including business partner facing operations, could go down because of issues to do with completely unrelated, non critical things like eg internal staff vacation hours. This could apply to any software. Sure, a monolith can have a large blast radius, but I can guarantee one of your microservices is critical path and would cause the same outage if it goes offline. > The few callers that do need to obtain both pieces of data just make concurrent calls to both and them zip them into a single result. Almost like a database join?
- bedobi 6y ago
  Who says the monolith we had was a clean codebase with a solid toolchain? It was a giant piece of software that had existed since the 90s, with hundreds of developers working on it at any given moment, thousands of people coming through it over the years, full of bizarre things that would make anyone scream and tear their hair out. (each of which may have been the right thing to do at the time, but combined over the years wasn't optimal to say the least) It's obvious nothing will convince you, but I maintain that eg internal vacation hours should not live in the same codebase as partner facing business critical things.
  - robertlagrant 6y ago
    > Who says the monolith we had was a clean codebase with a solid toolchain? You didn't. They're just trying to force the issue for some reason by straw manning your decision.
  - Nextgrid 6y ago
    You can very well have a bad monolith and still think micro services is a bad idea, and his concerns especially around referential integrity are perfectly valid and valuable. Maybe all you needed is to fix the existing monolith incrementally?
    - bedobi 6y ago
      Of course microservices should be sliced with referential integrity in mind. But a monolith by definition has everything, including utterly unrelated things that have absolutely nothing to do with each other, bundled up into a single, giant, well, monolith. What referential integrity could there possibly be between eg business partner user profiles and internal staff vacation hours?
      - quickthrower2 6y ago
        There is nothing inherently bad about storing unrelated things in the same db, in general. There might be security issues but good DBs offer security grants that are granular enough to deal with this. Data sovereignty might be an issue forcing you to split, that depends on the domain and application requirements. But in general two things being unrelated isn’t a reason to split the DB and splitting the DB isn’t a reason to split into services. I’m not for or against microservices. I’m definitely for good reasoning though!
yakshaving_jgt 6y ago
Could you be specific about the size of your company and the size of the data you're processing? And could you also be more specific about the business value it created? Are you an investor in the business? Is it your own money on the line?
- bedobi 6y ago
  Publicly listed, ~13 billion USD revenue, 22k employees, millions of b2b partners and users. When the monolith goes down, it's hundreds of thousands, sometimes millions of dollars of revenue lost both during and as a result of an outage. You really, really don't want a problem with eg internal staff vacation hours to do that.
  - yakshaving_jgt 6y ago
    Ok, interesting. That kind of scale is not exactly the norm though, so I don't agree with your implication that monoliths are always a bad idea.
    - dodobirdlord 6y ago
      I really don't know why anyone who wasn't so resource strapped as to have no alternative would want their $$$ services sharing a build-test-deploy-rollback lifecycle or database with the internal staff vacation hours tracker.
      - mrgordon 6y ago
        It seems like the real solution here is to outsource vacation hour tracking to one of the HR apps for the purpose (see Workday) instead of having the domain expert engineers split the company’s application apart into micro services so they can have a custom vacation tracker.
        bedobi 6y ago
        It was just one of the worst examples. A more obvious case would be the different lines of business. They have very little to do with each other and shouldn't share code or data. Another is business partners vs end users (regular people) - there's a lot if reasons, including legal ones, why they shouldn't share code or data.
gfodor 6y ago
No offense, but someone coming in and splitting out a database into multiple bespoke data stores because the primary database is "bursting at the seams" is basically my nightmare. If your description of your reasoning is accurate, it's a completely broken mental model.
- bedobi 6y ago
  Well, that's not what happened. The whole org from devs to the board were highly aware of the problems and knew and agreed what had to be done. I'm just one of the many people who architected and implemented the change.

kenrose 6y ago

> > A huge part of my career has been splitting monoliths into cohesive smaller codebases > Did that ... create business value for your company? Not OP, but the big value of splitting into microservices is isolation. In production, this isolation offers a limited blast radius in the case of an errant service. Also, independent scaling. Business value: improved reliability. In code, isolation lets development teams have a smaller and more focused domain / set of concerns to reason about (vs. the entirety of the monolith). Business value: Increased dev velocity.

bedobi 6y ago
Exactly this!
- e-brake 6y ago
  Agreed, isolation is a good reason to opt for microservices. One app goes offline, others still accessible, and you can do isolated maintenance. That, and features that aren't directly related, are super lightweight (quick to build, run, low on memory and CPU) as little projects.
whakim 6y ago
In my experience, it's easy enough to have services which have a large blast radius themselves and can become points of failure for your entire ecosystem. I don't find this to be a huge point of difference from a monolith, although of course it depends a lot on what you're working on. To pick on the OP a bit here (sorry!), they said that their entire legacy system "could go down because of issues to do with completely unrelated, non critical things like eg internal staff vacation hours." To me that sounds like poorly written software regardless of architecture. I can't imagine a scenario in any of the recent codebases I've worked on (microservices and monoliths both) where errors in what sounds like an internal CRUD tool would cause an entire production application to crash. I find it even harder to imagine if the application has a halfway decent test suite. To add to that, when you have hundreds of services running around and something goes wrong, it ends up being a lot harder to track down exactly what's happening. So when you do get that critical error, oftentimes the downtime is worsened. As for dev velocity, I find the claims of the microservice gospel a little bit exaggerated. Your layers of nested services all talk to each other, and any of them could be a point of failure. This isn't really all that different from calling another function in your monolithic app - you've just distributed that function call across a network boundary. You still need to know the callee's API, and you'll still spend a decent amount of time trying to understand the ways that the callee might fail. But you've also created a huge amount of additional developer work whenever you need to do something that spans the boundaries of existing services. I think microservices certainly have their advantages but a lot of the simplistic claims made by their biggest proponents only hold up prima facie.

inshadows 6y ago

> Did that ... create business value for your company? Image what a piece of crap Linux kernel would be if it was developed with this mindset. :-) EDIT: The context is that there's constant code splitting code moving being done inside Linux.