13 ms·
I've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into mult
by thermodynthrway 8y ago
I've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into multiple services.
I've worked with microservices a lot. It's a never-ending nightmare. You push data consistency concerns out of the database and between service boundaries.
Fanning out one big service in parallel with a matching scalable DB is by far the most sane way to build things.
- madrox 8y agoMicroservices aren't some magic bullet for scaling. If anything, they conform to Conway's Law [1]. I'd agree, though, if a single engineer is singularly responsible for 2+ microservices that are only supporting a single product...you're doing it wrong. [1] https://en.wikipedia.org/wiki/Conway%27s_law https://en.wikipedia.org/wiki/Conway%27s_law
- Marazan 8y agoI think that's possibly wrong as a key advantage of micro service over monolith is scaling. We have a 'single' product built as 2 micro services which is basically an external facing orchestration service that calls the compute heavy backend and it allows the external facing service to serve all the incoming traffic out of (on average) 4 containers whilst the backend computes with 32 containers and scales independently - very small changes to the incoming traffic can have large effects on the volume of traffic going to the compute service.
- davidkellis 8y ago> You push data consistency concerns out of the database and between service boundaries. Sing that from the rooftops. That is exactly my observation as well. All the vanilla "track some resource"-style webapps I've worked on were never designed to cope with a consistency boundary that spans across service boundaries. Turning a monolith into distributed services is hard for that reason - you have to redesign your data access to ensure that consistency boundaries don't span across multiple services. If you don't do that, then you have to learn to cope with eventual consistency; in my experience, most people just don't think that way. I know I have trouble with it. Surely I'm not the only one.
- noir_lord 8y agoThey also throw Single Source of Truth out the window way too often.
- stickfigure 8y agoIt's worse than that; it's my observation that most microservice architectures just ignore consistency altogether ("we don't need no stinking transactions!") and blindly follow the happy path. I've never quite understood why people think that taking software modules and separating them by a slow, unreliable network connection with tedious hand-wired REST processing should somehow make an architecture better. I think it's one of those things that gives the illusion of productivity - "I did all this work, and now I have left-pad-as-a-service running! Look at the little green status light on the cool dashboard we spent the last couple months building!" Programmers get excited about little happily running services, these are "real" to them. Customers couldn't care less, except that it now takes far longer to implement features that cross multiple services - which, if you've decomposed your services zealously enough, is pretty much all of them.
- ThJ 8y agoWhen I was designing a microservices architecture for a former employee, the justification for it was security. We needed separation of the components in the system because we were going to be handling money. It used ZeroMQ for communication, because I wanted something more lightweight and falut-resistant for message passing (in such a sensitive application, we didn't want to trust the network). Although it was of course fun to design, microservices weren't my first choice. It just made sense to us to use it in that particular scenario.
- hosh 8y agoSometimes, I've seen a lack of regard for data consistency within a monolith. That is not completely on the developer, either. Pre 4.0 Mongodb, for example, does not do transactions. On the other hand, I've seen some pretty flagrant disregard for it just because there are not atomicity guarantees. Microservices makes reasoning on that harder.
- kaoussi 8y agoWhat kind of issues you faced while working with microserviceS?
- kamaal 8y ago>>You push data consistency concerns out of the database and between service boundaries. No architecture principle will help you if you design things the wrong way. The point of Micro services architecture is to design large working systems, from smaller working systems.
- dualogy 8y ago"Need to" and "sane" are among my favourite subjective terms! (Further below, I'll go into in which contexts I'd agree with your assessment and why. But for now the other side of the coin.) In the real world, current-day, why do many enterprises and IT departments and SME shops go for µservice designs, even though they're not multimillion-user-scale? Not for Google/Netflix/Facebook scale, not (primarily/openly) for hipness, but they do like among other reasons: - that µs auto-forces certain level of discipline in areas that would be harder-to-enforce/easier-to-preempt by devs in other approaches --- modularity is auto-enforced, separation of concerns, separation of interfaces and implementations, or what some call (applicably-or-not) "unix philosophy" - they can evolve the building blocks of systems less disruptively (keep interfaces, change underlyings), swap out parts, rewrites, plug in new features to the system etc - allows for bring-your-own-language/tech-stack (thx to containers + wire-interop) which for one brings insights over time as to which techs win for which areas, but also attracts & helps retain talent, and again allows evolving the system with ongoing developments rather than letting the monolith degrade into legacy because things out there change faster than it could be rewritten I'd prefer your approach for intimately small teams though. Should be much more productive. If you sit 3-5 equally talented, same-tech/stack and superbly-proficient-in-it devs in a garage/basement/lab for a few months, they'll probably achieve much more & more productively if they forgoe all the modern µservices / dev-ops byzantine-rabbithole-labyrinths and churn out their packages / modules together in an intimate tight fast-paced co-located self-reinforcing collab flow. No contest! Just doesn't exist often in the wild, where either remote distributed web-dev teams or dispersed enterprise IT departments needing to "integrate", rule the roost. (Update/edit: I'm mostly describing current beliefs and hopes "out there", not that they'll magically hold true even for the most inept of teams at-the-end-of-the-day! We all know: people easily can, and many will, 'screw up somewhat' or even fail in any architecture, any language, any methodology..)
- codemac 8y agoIs there any evidence of this being true? Do they actually force a discipline? Do people actually find swapping languages easier with RPC/messaging than other ffi tooling? And do they really attract talent?! You make some amazing claims that I have seen no evidence of, and would love to see it.
- hinkley 8y agoPeople look at microservices as a solution to the Big Ball of Mud, and then confuse "a solution" with "the solution". You really do have to modularize. In some languages, you can even use separate compilation units for separate modules to enforce the separation. You can do all of that but get simultaneous deployment, which cuts out whole classes of integration nightmares.
- nickthemagicman 8y agoIt seems like a Big Ball of mud is much more difficult to deal with than a bunch of small balls of mud just in general?
- cloverich 8y agoIt depends but in general I'd disagree. If the small balls of mud have all kinds of implicit dependencies, but you have to find them by searching across codebases (and languages) -- does that sound easier than finding them all in the same codebase and language? Overall, its comparing bad design to bad design. I think the main argument I'd make here is, micro-services doesn't actually solve the big ball of mud problem, it solves a completely different problem.
- stefano 8y agoYou can step-through your big ball of mud in a debugger, but you can't do the same with your small balls of mud. Not easily at least. That alone makes a huge difference.
- hinkley 8y agoThey're both bad. It feels like a reverse Sophie's Choice to have to pick one. The real friction in the system is always in the boundaries between systems. With microservices it's all boundaries. Instead of a Ball of Mud you have Trees and No Forest. Refactoring is a bloody nightmare. Perf Analysis is a game of finger pointing that you can't defuse.
- thermodynthrway 8y agoA thousand small balls of mud that communicate with each other randomly is nearly impossible to grok because the IDE can't save you
- nodesocket 8y agoI generally agree that for the vast majority of web applications KiiS holds true and use monolith. In terms of the common code divergence why not just use private NPM and enforce latest? Have a hard-fast rule that all services must always use the latest version of common.
- NickBusey 8y agoRight, but the thing that makes Linux actually useful isn't really the kernel is it? I would say what makes it useful is all the various small, targeted programs (some might call them microservices) it lets you interact with to solve real world problems. If Linux tried to be an entire computing system all in one code base, (sed, vim, grep, top, etc., etc.) what do you think that would look like code base/maintainability wise? Sounds like a nightmare to me.
- lmm 8y ago> If Linux tried to be an entire computing system all in one code base, (sed, vim, grep, top, etc., etc.) what do you think that would look like code base/maintainability wise? You mean like BSD?
- boramalper 8y agoSame with BSD; the userspace programs are not part of the kernel, and developed separately by the same entity. Busybox is a slightly better example (even though it’s also a userspace program).
- jcelerier 8y ago> Same with BSD; the userspace programs are not part of the kernel, and developed separately by the same entity. they're part of the same repo and built at the same time than the kernel. Run a big "make universe" here : https://github.com/freebsd/freebsd https://github.com/freebsd/freebsd and see for yourself. That they are different binaries does not matter a lot, it's just a question of user interface. See for instance busybox where all the userspace is in a single binary.
- zerd 8y agoI would argue that the fact that they're in the same repo does not make it a monolith at all. All of google and facebook is in a single repo and built at the same time, but it doesn't make them monoliths.
- briandear 8y agoTeams have a lot to do with microservice use. If you have a small team a monolith can work well. If you have a large team, microservices have some significant advantages. And in some cases, you might have a microservice in Swift when you need high performance, but other less intensive services might be in Ruby or some other language, etc. Right took for the job should be the goal as opposed to chasing fashion. Microservices are definitely overused, but they do have many legitimate use cases. Your CRUD web application probably doesn’t need microservices, but complex build systems might. Imagine the complexity of Amazon.com or Netflix were a monolith. But something like Basecamp is probably better as a monolith. There is also the issue of scale. An ML processor might need more (and different) hardware than a user with system. Right tool (and architecture) for the job.
- User23 8y ago> Imagine the complexity of Amazon.com or Netflix were a monolith. That was called Obidos and it's shortcomings were why Jeff pushed through the services mandate at Amazon.
- Q6T46nT668w6i3m 8y agoLinux isn’t a giant monolith! Linux is composed of subsystems with independent maintainers.
- jrs95 8y agoYes, but "monolithic" web applications can be built in the same way. It might not be a 100% accurate usage of the term, but microservice/SOA advocates love to call modular applications monliths anyways, to the point that it's something that most people seem to do.
- slededit 8y agoThe key to a large code base is a hierarchical design. People are looking for a technical solution to a design problem. No matter what technology you use if you don't maintain the hierarchical layers your code will turn to mush. Micro services can provide a really strict boundary but if you draw the line in the sand at the wrong place it isn't going to matter.
- johannes1234321 8y agoThe kernel can't be categorized fully as monolith: Many kernel subsystems are developed individually, by different teams and merged into the main branch by subsystem.
- rhizome 8y agoYou push data consistency concerns out of the database and between service boundaries. Gotta put that CS Masters to work somehow. Can't just sit here doing plumbing all day every day. Maybe there could be a software development corollary to the Politician's Syllogism[1], or even just a webdev one. 1. https://en.wikipedia.org/wiki/Politician%27s_syllogism https://en.wikipedia.org/wiki/Politician%27s_syllogism
- richardlblair 8y ago> 100 web applications in the world that need to be split into multiple services. Services are not micro services. Most large scale applications can and should be split into multiple services. However, when approaching a new problem you should work within the monolith resisting the service until you absolutely can't any longer. Ideally this will make your services true services, that could capture an entire business unit. When it's all said and done you should be able to sell off the service as a business. The other use case, which should be obvious, is compliance. If you are thinking about implementing anything that would require PCI or SOX you should do that in a service to shield the rest of the dev org from the complexities. So, any webapp that takes payment and interacts directly a payment processor. That said, you're correct in that you should not be rolling out a new service to avoid sharding.
- akvadrako 8y agoThis is really missing the point. Microservices are not about code organisation - they are about runtime separation. And even kernels have kernel threads, which are basically local microservices. Anything which needs to scale beyond a single system is more deserving of microservices than a kernel.
- sanderjd 8y ago> I've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into multiple services. This is apples and oranges. The production profile of operating system kernels and web applications are so dissimilar that the analogy is not useful. It may be true that most web applications don't need to be split into multiple services, but the Linux kernel provides no evidence either way.
- jjeaff 8y ago>Fanning out one big service in parallel with a matching scalable DB is by far the most sane way to build things. Agreed but I also prefer to keep any changing storage data as a separate concern (s3 or similar). So the trinity of services would be DB, storage, application.
- deleted 8y ago[deleted]
- gaius 8y agoI've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into multiple services. Similarly it’s made with make. If anyone has a project more complex than the Linux kernel or GCC I’ll gladly listen to why they need some exotic build system... never met anyone yet...
- dralley 8y agoFirefox probably qualifies. Split between C/C++/Rust/JavaScript/XUL/Python/god knows what else
- erfgh 8y agoBecause make is built with the assumption that you can launch a process (cc) with minimal startup overhead. That doesn't work for javac for example.
- rcxdude 8y agoThere's a decent amount of custom supporting machinery around make to create the linux build system.
- radisb 8y ago"I've worked with microservices a lot. It's a never-ending nightmare. You push data consistency concerns out of the database and between service boundaries." This exactly. Me too. Data consistency concerns in sufficiently large real world projects can be practically dealt with only 2 ways IMO: transactions or spec changes.
- amirouche 8y agowhat about idempotent updates?