10 ms·
So many gems in here but this one about microservices is my favorite: grug wonder why big brain take hardest problem, factoring system correctly, and introduce
by anthomtb 1y ago
So many gems in here but this one about microservices is my favorite:
grug wonder why big brain take hardest problem, factoring system correctly, and introduce network call too
- jiggawatts 1y agoI keep trying to explain this to tiny dev teams (1-2 people) that will cheerfully take a trivial web app with maybe five forms and split it up into “microservices” that share a database, an API Management layer, a queue for batch jobs to process “huge” volumes (megabytes) of data, an email notification system, an observablity platform (bespoke!) and then… and then… turn the trivial web forms into a SPA app because “that’s easier”. Now I understand that “architecture” and “patterns” is a jobs program for useless developers. It’s this, or they’d be on the streets holding a sign saying “will write JavaScript for a sandwich”.
- someothherguyy 1y ago> Now I understand that “architecture” and “patterns” is a jobs program for useless developers. Yet, developers are always using patterns and are thinking about architecture. Here you are doing so too, a pattern, "form submission" and an architecture, "request-response".
- balamatom 1y agoA word and its meaning are usually antonyms.
- djeastm 1y ago>I keep trying to explain this to tiny dev teams I'm curious what role you have where you're doing this repeatedly
- jiggawatts 1y agoThe customer is a government department formed by the merger of a bunch of only vaguely related agencies. They have “inherited” dozens of developers from these mergers, maybe over a hundred if you count the random foreign outsourcers. As you can imagine there’s no consistency or organisational structure because it wasn’t built up as a cohesive team from the beginning. The agencies are similarly uncoordinated and will pick up their metaphorical credit card and just throw it at random small dev teams, internally, external, or a mix. Those people will happily take the credit! The money just… disappears. It’s like a magic trick, or one of those street urchins that rips you off when you’re on holiday in some backwards part of the world like Paris. I get brought in as “the cloud consultant” for a week or two at the end to deploy the latest ball of mud with live wires sticking out of it to production. This invariably becomes an argument because the ball of mud the street urchins have sold to the customer is not fit for… anything… certainly not for handling PII or money, but they spent the budget and the status reports were all green ticks for years. Fundamentally, the issue is that they're "going into the cloud" with platform as a service, IaC, and everything, but at some level they don't fully grok what that means and the type of oversight required to make that work at a reasonable cost. "But the nice sales person from Microsoft assured me the cloud is cheaper!"
- angry_octet 1y agoOmg this is something I have experienced too many times, and constantly warring with the other side of the coin: people who never want to make any change unless it is blessed by a consultant from Microsoft/VMWare/SAP and then it becomes the only possible course of action, and they get the CIO to sign off on some idiocy that will never work and say "CIO has decreed Project Falcon MUST SUCCEED" when CIO can't even tie his shoelaces. Giant enterprise integration will happen! In fact we're going through one of these SAP HANA migrations at present and it's very broken, because the prime contractor has delivered a big ball of mud with lots of internal microservices.
- deleted 1y ago[deleted]
- vermilingua 1y agoIs this DCS in NSW? If so that would explain so much about my own work interactions with them.
- mattmanser 1y agoIt's all they've seen. They don't get why they're doing it, because they're junior devs masquerading as architects. There's so many 'senior' or 'architect' level devs in our industry who are utterly useless. One app I got brought in late on the architect had done some complicated mediator pattern for saving data with a micro service architecture. They'd also semi-implemented DDD. It was a ten page form. Literally that was what it was supposed to replace. An existing paper, 10 page, form. One of those "domains" was a list of the 1,000 schools in the country. That needed to be updated once a year. A government spent millions on this thing. I could have done it on my todd in 3 months. It just needed to use simple forms, with some simple client side logic for hiding sections, and save the data with an ORM. The funniest bit was when I said that it couldn't handle the load because the architecture had obvious bottlenecks. The load was known and fairly trivial (100k form submissions in one month). The architect claimed that it wasn't possible as the architecture was all checked and approved by one of the big 5. So I brought the test server down during the call by making 10 requests at once.
- jiggawatts 1y ago> So I brought the test server down during the call by making 10 requests at once. Back in the very early 2000s I got sent to "tune IIS performance" at a 100-developer ISV working on a huge government project. They showed me that pressing the form submit button on just two PCs at once had "bad performance". No, not it didn't. One was fast[1], the other took 60 seconds almost exactly. "That's a timeout on a lock or something similar", I told them. They then showed me their 16-socket database server that must have cost them millions and with a straight face asked me if I thought that they needed to upgrade it to get more capacity. Upgrade to what!? That was the biggest machine I have ever seen! I've never in the quarter century since then seen anything that size with my own two eyes. I don't believe bigger Wintel boxes have ever been made. I then asked their database developers how they're doing transactions and whether they're using stored procedures or not. One "senior" database developer asked me what a stored procedure is. The other "senior" database developer asked me what a transaction is. "Oh boy..." [1] Well no, not really, it took about a second, which was long enough for a human button press to to "overlap" the two transactions in time. That was a whole other horror story of ODBC connection pooling left off and one-second sleeps in loops to "fix" concurrency issues.
- frollogaston 1y agoThe only useful definition of a "service" I've ever heard is that it's a database. Doesn't matter what the jobs and network calls are. One job with two DBs is two services, one DB shared by two jobs is one service. We once had 10 teams sharing one DB, and for all intents and purposes, that was one huge service (a disaster too).
- glurgl 1y agoYes! My viewpoint also. I've been very alone in that view over the last few decades. Nice to know there's someone else out there.
- frollogaston 1y agoAt least it's widely considered bad practice for two services to share a database. But that's different from orienting your entire view of services around databases. The case of one job with two DBs matters too, mostly because there's no strong consistency between them.
- jihadjihad 1y agoGreat point. I've only worked at a couple places that architected the system in this manner, where the data layer defines the service boundary. It really helps keep the management of separate services sane IMO, vs. different "services" that all share the same database.
- jiggawatts 1y agoA more precise view is that there are boundaries inside of which certain operations are atomic. To make this more precise: the difference between a “dedicated schema” and a “database” is that the latter is the boundary for transactions and disaster recovery rollbacks. If you mix in two services into a single database - no matter how good the logical and security isolation is — they will roll back their transactions together if the DBA presses the restore button. Similarly they have the option (but not the obligation!) to participate in truly atomic transactions instead of distributed transactions. If this is externally observable then this tight coupling means they can no longer be treated as separate apps. Many architects will just draw directed arrows on diagrams, not realising that any time two icons point at the same icon it often joins them into a single system where none of the parts are functional without all of the others.
- default-kramer 1y agoI'm convinced that some people don't know any other way to break down a system into smaller parts. To these people, if it's not exposed as a API call it's just some opaque blob of code that cannot be understood or reused.
- dkarl 1y agoThat's what I've observed empirically over my last half-dozen jobs. Many developers treat decomposition and contract design between services seriously, and work until they get it right. I've seen very few developers who put the same effort into decomposing the modules of a monolith and designing the interfaces between them, and never enough in the same team to stop a monolith from turning into a highly coupled amorphous blob. My grug brain conclusion: Grug see good microservice in many valley. Grug see grug tribe carry good microservice home and roast on spit. Grug taste good microservice, many time. Shaman tell of good monolith in vision. Grug also dream of good monolith. Maybe grug taste good monolith after die. Grug go hunt good microservice now.
- stavros 1y agoWe've solved this problem by making the modules in the monolith only able to call each other from well-defined APIs, otherwise CI fails.
- PaulHoule 1y agoIn the Java world both Spring and Guice are meant to do this, and if you have an ISomething you've got the possibility of making an ILocalSomething and a IDistributedSomething and swap one for the other.
- pbh101 1y agoThis is generally a bad idea imo. You fundamentally will have a hard time if your api is opaquely network-dependent or not. I suppose, you’ll be ok if you assume there is a network call, but that means your client will need to pay that cost every time, even if using the ILocal.
- api 1y agoI have a conspiracy theory that it’s a pattern pushed by cloud to get people to build applications that: - Cannot be run without an orchestrator like K8S, which is a bear to install and maintain, which helps sell managed cloud. - Uses more network bandwidth, which they bill for, and CPU, which they bill for. - Makes it hard to share and maintain complex or large state within the application, encouraging the use of more managed database and event queue services as a substitute, which they bill for. (Example: a monolith can use a queue or a channel, while for microservices you’re going to want Kafka or some other beast.) - Can’t be run locally easily, meaning you need dev environments in cloud, which means more cloud costs. You might even need multiple dev and test environments. That’s even more cloud cost. - Tends to become dependent on the peculiarities of a given cloud host, such as how they do networking, increasing cloud lock in. Anyone else remember how cloud was pitched as saving money on IT? That was hilarious. Knew it was BS way back in the 2000s and that it would eventually end up making everything cost more.
- pphysch 1y agoThose are all good points, but missing the most important one, the "Gospel of Scalability". Every other startup wants to be the next Google and therefore thinks they need to design service boundaries that can scale infinitely...
- nyarlathotep_ 1y agoIt's 100% this; you're right on the money (pun intended). Don't forget various pipelines, IaC, pipelines for deploying IaC, test/dev/staging/whatever environments, organization permissions strategies etc etc... When I worked at a large, uh, cloud company as a consultant, solutions were often tailored towards "best practices"--this meant, in reality, large complex serverless/containerized things with all sorts of integrations for monitoring, logging, NoSQL, queues etc, often for dinky little things that an RPI running RoR or NodeJS could serve without breaking a sweat. With rare exceptions, we'd never be able to say, deploy a simple go server on a VM with server-side rendered templates behind a load balancer with some auto-scaling and a managed database. Far too pedestrian. Sure, it's "best practices" for "high-availability" but was almost always overkill and a nightmare to troubleshoot.
- fellatio 1y agoUnfortunately it is useful to do this for many other reasons!
- chamomeal 1y agoUnfortunately indeed. I lament the necessity of microservices at my current job. It’s just such a silver bullet for so many scaling problems. The scaling problems we face could probably be solved by other solutions, but the company is primed and ready to chuck functionality into new microservices. That’s what all our infrastructure is set up to do, and it’s what inevitably happens every time
- jppope 1y agoThe frequency that you use the term "re-factor" over the term "factor" is often very telling about how you develop your systems. I worked a job one time where the guys didn't even know what factoring was.
- zelphirkalt 1y agoProbably many people don't pick up on the word "to factor" something these days. They do not make the connection between the thing that mathematicians do and what that could relate to in terms of writing code. At the same time everyone picks up the buzzword "to refactor". It all depends on what ecosystems you expose yourself to. I think I first heard the term "to factor" something in math obviously, but in software when I looked at some Forth. Most people will never do that, because it is so far off the mainstream, that they have never even heard of it.
- deleted 1y ago[deleted]
- closeparen 1y agoThe network boundary gives you a factoring tool that most language module systems don't: the ability for a collection of packages to cooperate internally but expose only a small API to the rest of the codebase. The fact that it's network further disciplines the modules to exchange only data (not callbacks or behaviors) which simplifies programming, and to evolve their interfaces in backwards compatible ways, which makes it possible to "hot reload" different modules at different times without blowing up. You could probably get most of this without the literal network hop, but I haven't seen a serious attempt.
- jakewins 1y agoAny language that offers a mechanism for libraries has formal or informal support for defining modules with public APIs? Or maybe I’m missing what you mean - can you explain with an example an API boundary you can’t define by interfaces in Go, Java, C# etc? Or by Protocols in Python?
- closeparen 1y agoThe service I'm working on right now has about 25 packages. From the language's perspective, each package is a "module" with a "public" API. But from the microservices architecture's perspective, the whole thing is one module with only a few methods.
- rcxdude 1y agoBut why would users of the module care about the dependency packages? You could still have a module with only a few methods and that's the interface.
- dijksterhuis 1y agoi’ve seen devs do stuff like this (heavily simplified example) from submodule import pandas why? no idea. but they’ve done it. and it’s horrifying as it’s usually not done once. microservices putting a network call in on the factoring is a feature in this case, not a bug. it’s a physical blocker stopping devs doing stuff like that. it’s the one thing i don’t agree with grug on. HOWEVER — it’s only a useful club if you use it well. and most of the time it’s used because of expectations of shiny rocks, putting statements about microservices in the company website, big brain dev making more big brain resume.
- npodbielski 1y agoI think mostly this is to brake down the system between teams. This is easier to manage this way. Nothing to do with technical decision - more the way of development. What is the alternative? Mono-repo? IMHO it is even worse.
- nothrabannosir 1y agomicroservices and mono repo are not mutually exclusive. Monolith, is. Important distinction imo, Micro services in mono repo definitely works and ime is >>> multi repo. Of course the best is mono repo and monolith :3
- npodbielski 1y agoI did not stated that they can be used together. I just meant that they are mainly, for me, work as a tool to allow multiple teams work on a single big project. So the tool is more for work organisation purpose not for architectural one. If you prefer working on mono repo it s fine though I think micro services are more popular one now.
- nothrabannosir 1y agoAm I reading it wrong ? I understand both your original comment and this one to paint mono repo as an alternative to micro services. E.g.: “ What is the alternative [to micro services]? Mono-repo?” and “ If you prefer working on mono repo it s fine though I think micro services are more popular one now.” What I understand here is that “mono repo is an alternative to micro services”. Is that what you’re saying? If so: mono repo is not an alternative to micro services. You can have both. You can have micro services in a mono repo. One single repository with many micro services. That is possible. What is an alternative to micro services is “monolith”. Curious how I’m misunderstanding this because I looked at it for a while and I really don’t see it.