6 ms·
I have done the full theme park ride on monolith->microservices->monolith. Both have ups and downs. The most important thing I learned is that "microservices"
by bob1029 5y ago
I have done the full theme park ride on monolith->microservices->monolith. Both have ups and downs.
The most important thing I learned is that "microservices" in absolutely no way necessitates "bullshit spread across multiple cloud vendors and other scenarios involving more than 1 computer". What part of microservices says things must be separated by way of an arbitrary wire protocol?
We now have a "monolith" (process) that is comprised of many "microservices" (class files), each of which is responsible for its own database, business logic, types, etc. These services are able to communicate amongst themselves using the simple path of direct method invocation.
If you are looking to scale up your capacity or even make things more resilient, microservices vs monolith is really not the conversation you need to be having. If you are trying to better organize your codebase, you might be in the right place but you also need to be super careful with how you proceed.
We wasted a solid 3 years trying to make microservices solve all of our problems. Looking back, we spent more time worrying about trying to put Humpty Dumpty back together again with JSON APIs and other technological duct tape than what our customers were complaining about.
- crdrost 5y agoYeah, the biggest problem that people get into with microservices is that they allow the communication structure to dictate the app structure. A monolith gives you some refactorability because you can run what would have been "integration" tests locally, as you massage your module boundaries to match the problem that you're solving. So a monolith can become a clean monolith, and then a clean monolith can maybe become microservices if you need the scale attributes. The basic problem is that before you know where the module boundary is, you cannot have a clear module boundary. So three popular approaches emerge: (a) DDD "milliservices". Basically define the different sorts of users, come up with a clustering of the different sorts of people that you think will be using your app. People are considered to be in the same cluster if they use the same jargon to refer to things, or different clusters if they both use some word but they both mean subtly different things by it. (b) Gut-feeling microservices that become feature microservices. I should probably have an "auth service," I don't know what it does but I'm going to be doing auth so that's probably a service. I need to import a git library to contact GitHub, probably there should be a "git service" that handles all communication to GitHub or other git repos. Each of these things exposes some swagger/openAPI docs, maybe we should have a docs service? -- that sort of thing. The danger is that the things that are easy to break off are usually not the core competency of the product, and so there emerges some sort of "core services," one or two big honkers that basically are monoliths. People are aware that they aren't supposed to keep adding to the core services and so new services emerge named after feature requests, hello "sharing service," hello "wallet service." Except the core usually is tightly coupled to these new feature services and they all kind of connect to each other. The idea of splitting the core services into other services to fit the newfound module boundaries becomes complicated by all these ties to nearby features, you are not actually loosely coupled because you do RPCs and you probably code in the expectation of success into these RPCs like method calls. (c) Every noun becomes a "nanoservice". This is a service that watches just one or two tables (or NoSQL document types or what have you) and exposes a CRUD API for that noun, plus a couple auxiliary verbs to do actual business needs with those nouns. So if you were implementing Git there would be a file service, a tree service, a branch service, a commit service, probably a working tree service... But you just have to "know" that logs and rebases and cherrypicks live in CommitService while diffing for some reason lives in TreeService and adding a file to your working tree requires first creating it in FileService and then handing that link off to the WorkingTreeService which will make needed calls to TreeService, creating your own tree is exposed via TreeService's CRUD but the developers tell you that down that road There Be Dragons and you should not have been messing with that. Any of these three can be successful but only insofar as you can create new module boundaries and move module boundaries and test to make sure that your users will not notice any performance regressions. Of them I would only recommend option (a), because it gives a really nice place for these tests to live and an intuition that each test should document a user journey for the system.
- jamesfinlayson 5y agoA company I worked at went all in on option c - there was a random number service (that just wrapped a random number generation library), and sending an email was split across multiple services - one to pull the email request from the database, one to generate the email from the template, one to actually send the email and one to save the templated email to the database.
- dm3 5y agoHow did it work out? Very curious, as such granularity of deployable units seems absurd unless it's deployed into a single runtime...
- jamesfinlayson 5y agoIn reality it worked - each service only talked to the next so in that sense it was okay. But each service was its own binary running in its own container (someone went to a conference and heard that S3 was made up of 500 microservices, to they decided we needed as many microservices as possible). The bit where it didn't work so well was making changes and testing those changes - the original developers hacked it together then didn't want to touch it after that, so there was no dev environment (just staging environments to hijack) and no functional/e2e tests.
- yakshaving_jgt 5y agoI’m imagining that whomever is running that company prefers to heat his home by throwing cash directly into his fireplace instead of burning wood.
- tacone 5y ago"Gut-feeling microservices" is a fantastic definition, thanks for that.
- twic 5y ago> Gut-feeling microservices I'm in this comment and i don't like it. My current team has this pattern. I think we've ended up with two core services, with a fairly thin channel between them, but a constellation of tightly-coupled peripheral services around each core. To be honest, it works pretty well. It wasn't intentional, and it grew in exactly the haphazard way you describe. But it's not a disaster, or at least doesn't feel like one. Maybe we should rebrand this as "natural microservices"?
- withinboredom 5y agoCurrently at my day job, we have a monolithic code base with multiple entry points. If you update a library, you have to update the client or keep the changes backwards compatible. I really prefer this over multiple micro services.
- ruffrey 5y agoI’ve done monoliths and microservices. Having a monolithic codebase with many entry points is my preferred.
- blakehaswell 5y agoWhat exactly do you mean by multiple entry points? Do you have multiple processes which run independently but are co-located in the same repository or are you talking about something else?
- withinboredom 5y agoYou can have more than one Main function, just pick which one to use when you compile. In our case it’s PHP, so the API entry point uses a different index than jobs, for example. They can be deployed differently and scaled differently, but operationally it’s just deploying the same code/configuration everywhere and the only difference is routing. When you are writing code, you very, very rarely have to worry about which context you are writing for.
- bvm 5y agoI've done this before, it felt like a bit of a hack tbh, but I'm glad to hear someone else is doing it!
- sockpuppet69 5y ago
- twic 5y agoI have done this several times, with various small variations, and find it works well. I don't have a good name for it. I think it's a variety of cookie-cutter scaling: https://paulhammant.com/2011/11/29/cookie-cutter-scaling/ https://paulhammant.com/2011/11/29/cookie-cutter-scaling/ But there's nothing in that about using different entrypoints or routing. FWIW, variations on the theme: 1. A single binary with a single entrypoint, which can play multiple roles simultaneously (UI, API, scheduled jobs), but where different kinds of request are routed to different pools of instances 2. A single binary with multiple embedded configurations, selecting via command line arguments etc, each for a single different role (UI, admin console, data ingestion) 3. A single (Java) binary with multiple entrypoints (main methods), each playing a different role (live calculations, batch calculations, data recording) 4. A single (C++) codebase building multiple binaries (via CMake add_executable), each playing a different role (calculating prices for potatoes, calculating prices for oranges) 5. A single repository with multiple completely separate applications, with some shared submodules, each built separately, playing a different role (receiving transactions, validating transactions, reporting transactions) That last one is probably not an example of what you are talking about, but it's closer to the one before than to the first in the list. There is a sort of "ring species" [1] shape to this variation. [1] https://en.wikipedia.org/wiki/Ring_species https://en.wikipedia.org/wiki/Ring_species
- jayd16 5y agoFarming out libraries or subprojects to separate teams is an ok way to go as long as you don't mind a single language/build system. Its not as flexible but you maybe get some perf out of it. You don't get the convenience of a single datastore with simple transactions but it sounds like you prefer the flexibility of services owning their own data. As it turns out there's no silver bullet. Do what works for you.
- frozenport 5y ago>> single language/build system Imagine designing your entire architecture around your build system.
- rgbrgb 5y agoIf you're a startup, I think it's actually a pretty good design constraint. You're going to end up with a single system to optimize (inevitably in your 110% time because there's no platform team) and a leg-up in shipping with fast and well-tested infra (or at least better tested because it's what you're using everywhere else). Dropping-in and tweaking stuff across many parts of the system is way easier if it's consistent, easily testable wherever you are, and you know that tests for every dependency are being run on each PR. On the opposite extreme you're talking multiple PRs, languages, test suites and deploys for every new feature. I've watched startups do this and die before finding something people want to buy.
- kitd 5y agoIt's not as stupid as you make it sound. It is after all how normal manufacturing works. What you build is constrained by your assembly line. If you want to build something new, start off with redesigning your assembly line. It's time well-invested that pays back in heaps later on.
- bob1029 5y ago> If you want to build something new, start off with redesigning your assembly line. It's time well-invested that pays back in heaps later on. We applied this ideology to our software project. Our build/deploy toolchain is 100% in-house code. Turns out, there's actually not a lot of it required because you can build a monster .net project into a single exe distribution with 1 simple command line operation. This applies to most other ecosystems as well. Everything else is basic file operations and moving shit between computers. No reason to vendor either of these problems out. The benefits of having bespoke CI tooling are difficult to overstate. We've got automated processes where a project manager can multiselect 20 different customers, select a commit hash, and schedule an email containing a link to a zip of the resulting build that will be delivered to each customer's configured IT contact upon completion - all with a single button press. This process also entails injection of customer-specific configuration that is pulled at runtime from another system into each customer's distribution. The email templates are configured on a per customer basis and email delivery is delayed based upon project parameters. I can't imagine wiring something like this up in Jenkins or GH Actions.
- brundolf 5y agoYou could even have a mono-repo that holds services which run as separate processes, if you need that. That way you still get the benefits of sharing code, types, version numbers, build processes, etc, which seem like the main headaches with the usual approach.
- bob1029 5y agoWe passed right by this mode of operation while migrating from microservices=>monolith. Everything is a separate process but running on the same box. Ran that way for quite some time. We pulled services into the main process one-by-one. It went about as smoothly as one could have hoped for.
- jamesfinlayson 5y agoI've inherited a project like this - lots of common code in a shared library that gets used by multiple services. Doesn't work too badly but things have been split out more than they need to be.
- antishatter 5y agoPretty clearly the pros and cons come down to the application and necessary stability. Need ultra stable? More and smaller microservices usually better. Need fairly stable? Bigger services is fine. Need it to work generally? Build it however you can.
- manuelabeledo 5y ago> Need ultra stable? More and smaller microservices usually better. I don't think I agree with this statement. "More and smaller" also means potentially more routing and communication paths, thus increased latency and complexity.
- sorokod 5y agoMore smaller microservices bring along a combinatorial explosion of failure scenarios.
- rapind 5y agoPretty sure microservices will be remembered as a horribly convoluted stop gap once we have better language modularization. They encourage modular patterns which is usually good, but all that plumbing will eventually become unnecessary. I can’t help but remember building 6 versions of each class in the old ejb days whenever I hear microservices hype.
- steve_taylor 5y agoOSGi was supposed to solve this problem. You could start by running all bundles together in the same JVM, then split them across a cluster later if required. But it never quite took off beyond an implementation detail of various application servers.
- kavalg 5y agoOSGi was a horrible dev experience. Lots of quirks and legacy. Nobody wants to maintain that code anymore.
- sackerhews 5y ago> horrible dev experience That's an understatement. It was soul destroying awfulness.
- throwaway1777 5y agoExample of better language modularization? Who is working on this?
- NicoJuicy 5y agoThat's exactly what i'm doing for my ( almost fulltime) side-project. While we do Microservices at work ( distrubited team and Conway's law does make sense: https://ardalis.com/conways-law-ddd-and-microservices/ https://ardalis.com/conways-law-ddd-and-microservices/ ) It doesn't make sense for me personally. So I applied all DDD logic and implemented it in my app. In dotnet there is a useful feature where functionality from referenced projects is included in your main WebApp. Which makes everything pretty clean. I use application logic over 2 "gateways". Where a gateway is a dummy solution that contains Swagger, global things for that project ( eg. auth verification) and uses the controllers/logic from the referenced projects. - SPA - Angular that contains a Ocelot gateway on the same domain ( = no cors issues) - ApiGateway - Contains all the backend api logic and custom integrations with partners. - ShopGateway - Is included in the Frontend. - Frontend - the shop itselve. Currently in the process of making this db-less ( everything over API / messages or using the gateway. The gateway is currently a referenced project ( see up), so everything is pretty fast) In the meantime, i can make "reasonable" quick adjustments and the logic is really flexible. Overview of the solution: https://ibb.co/k5wn07x https://ibb.co/k5wn07x Note: Db migration from "1 project" to this is not complete yet :( . But it's one of the bigger shifts and i'm already 90% finished before i can start the actual shift to MartenDb from EF / Dapper ) If you want a quick summary on the logic behind, DDD would be the answer.
- chmod600 5y ago"What part of microservices says things must be separated by way of an arbitrary wire protocol?" That's not a requirement, cf. Erlang/OTP. But to qualify as a microservice, you need it decoupled enough that you can independently deploy and/or restart individual services. Pretty sure that's a minimum to call yourself microservice-based, and hard to do by just using classes in C++ or something. Arguably you should also have quite a bit of control over dependencies and their versions. If your service isn't ready for python3, it can keep using python2, or whatever. Taking this to the extreme would generally mean a black box that serializes everything over a network, so that any service can be implemented on any stack. I'm not sure that's fundamental to the idea of microservices, though.
- iamcreasy 5y ago> simple path of direct method invocation Does it mean calling a method of a class?