6 ms·
Of course there’s no mention of performance loss or gain after migration. I remember when microservices architecture was the latest hot trend that came off the
by xyst 2y ago
Of course there’s no mention of performance loss or gain after migration.
I remember when microservices architecture was the latest hot trend that came off the presses. Small and big firms were racing to redesign/reimplement apps. But most forgot they weren’t Google/Netflix/Facebook.
I remember end user experience ended up being _worse_ after the implementation. There was a saturation point where a single micro service called by all of the other micro services would cause complete system meltdown. There was also the case of an “accidental” dependency loop (S1 -> S2 -> S3 -> S1). Company didn’t have an easy way to trace logs across different services (way before distributed tracing was a thing). Turns out only a specific condition would trigger the dependency loop (maybe, 1 in 100 requests?).
Good times. Also, job safety.
- api 2y agoThis is a very fad driven industry. One of the things you earn after being in it for a long time is intuition for spotting fads and gratuitous complexity traps.
- makeitdouble 2y agoI think that aspect is indirectly covered, as one of the main motivation was to get on a popular platform that helps hiring. I agree on how it's technically a waste of time to pursue fads, but it's also a huge PITA to have a platform that good engineers actively try to avoid, as their careers would stagnate (even as they themselves know that it's half a fad)
- sangnoir 2y agoI avoid working at organisations with NIH syndrome - if they are below a certain size (i.e. they lack a standing dev-eng team to support their homegrown K8s "equivalent"). Extra red flags if the said homegrown-system was developed by that guy[1] who's ostensibly a genius and has very strong opinions about his system. Give me k8s' YAML-hell any day instead, at least that bloat has transferable skills, amd I can actually Google common resolutions. 1. Has been at org for so long, management condones them flaunting the rules, like pushing straight to prod. Hates the "inefficiency" of open source platforms and purpose-built something "suitable for the company" by themselves, no documentation you have to ask them to fix issues because they don't accept code or suggestions from others. The DSL they developed is inconsistent and has no parser/linter.
- fragmede 2y agoyeah. If you think Kubernetes is too complicated, the flip side of that is someone built the simpler thing, but then unfortunately it grew and grew, and now you've got this mess of a system. you could have just used a hosted k8s or k3s system from the start instead of reinventing the wheel. absoutely start as simple as you can, but plan to move to a hosted kube something asap instead of writing your own base images, unless that's a differentiator for your company.
- keybored 2y ago1. You have to constantly learn to keep up! 2. Fad-driven I wonder why I don’t often see (1) critiqued on the basis of (2).
- api 2y agoThere’s definitely a connection. Some of the change is improvement, like memory safe languages, but a ton of it is fads and needless complexity invented to provide a reason for some business or consultancy or group in a FAANG to exist. The rest of the industry cargo cults the big companies.
- teleforce 2y ago> There was also the case of an “accidental” dependency loop (S1 -> S2 -> S3 -> S1). The classic dependency loop example that you thought will never encounter again for the rest of your life after OS class
- pram 2y agoThe best part is when it’s all architected to depend on something that becomes essentially a single point of failure, like Kafka.
- bamboozled 2y agoNot saying you're wrong, but what is your grand plan ? I've never seen anything perfect.
- deleted 2y ago[deleted]
- intelVISA 2y agoShhh, you're ruining the party!
- arctek 2y agoIsn't this somewhat better, at least when it fails it's in a single place? As someone using Kafka, I'd like to know what the (good) alternatives are if you have suggestions.
- happymellon 2y agoIt really depends on what your application is. Where I'm at, most of Kafka usage adds nothing of note and could be replaced with a rest service. It sounds good that Kafka makes everything execute in order, but honestly just making requests block does the same thing. At least then I could autoscale, which Kafka prevents.
- NortySpock 2y agoNATS JetStream seemed to support horizontal scaling (either hierarchical via leaf nodes or a flat RAFT quorum) and back pressure when I played with it. I found it easy to get up and running, even as a RAFT cluster, but I have not tried to use JetStream mode heavily yet.
- lmm 2y ago
- jamesfinlayson 2y ago> There was a saturation point where a single micro service called by all of the other micro services would cause complete system meltdown. Yep - saw that at a company recently - something in AWS was running a little slower than usual which cascaded to cause massive failures. Dozens of people were trying to get to the bottom of it, it mysteriously fixed itself and no one could offer any good explanation.
- spyspy 2y agoAny company that doesn’t have some form of distributed tracing in this day and age is acting with pure negligence IMO. Literally flying blind.
- bboygravity 2y agoDistributed tracing? Is that the same as what Elixir/Erlang call supervision trees?
- williamdclt 2y agoLikely about Opentelemetry
- ahoka 2y agoOh, come on! Just make sure your architecture is sound. If you need to run an expensive data analysis cluster connected to massive streams of collected call information to see that you have loops on your architecture, then you have a bigger issue.
- hylaride 2y agoI don't know if you're being sarcastic, which if you are: heh. But to the point, if you're going to build a distributed system, you need tools to track problems across the distributed system that also works across teams. A poorly performing service could be caused by up/downstream components and doing that without some kind of tracing is hard even if your stack is linear. The same is true for a giant monolithic app, but the sophisticated tools are just different.
- jiggawatts 2y agoEven if the microservices platform is running at 1% capacity, it's guaranteed to have worse performance than almost any monolith architecture. It's very rare to meet a developer who has even the vaguest notion of what an RPC call costs in terms of microseconds. Fewer still that know about issues such as head-of-line blocking, the effects of load balancer modes such as hash versus round-robin, or the CPU overheads of protocol ser/des. If you have an architecture that involves about 5 hops combined with sidecars, envoys, reverse proxies, and multiple zones you're almost certainly spending 99% to 99.9% of the wall clock time just waiting. The useful compute time can rapidly start to approach zero. This is how you end up with apps like Jira taking a solid minute to show an empty form.
- throwaway48540 2y agoAre you talking about cloud Jira? I use it daily and it's very quick, even search results appear immediately...
- nikau 2y agoHave you ever used an old ticketing system like remedy? Ugly as sin, but screens appear instantly. I think Web apps have been around so long now people have forgotten how unresponsive things are vs old 2 tier stuff.
- BeefWellington 2y agoThere's a definite vibe these days of "this is how it's always been" when it really hasn't.
- throwaway48540 2y agoThe SPA Jira cloud is faster than anything server rendered for me, my connection is shit. In Jira I can at least move to static forms quickly and I'm not downloading the entire page on every move.
- chuckadams 2y agoI’ve used Remedy in three different shops, and the experience varied dramatically. The entry screen might have popped instantly, but RPC timeouts on submission were common. Tab order for controls was the order they were added, not position on the screen. Remedy could be pleasant, but it was very dependent on a competent admin to set up and maintain it.
- beeandapenguin 2y agoExtremely slow times - from development to production, backend to frontend. Depending on how bad things are, you might catch the microservice guys complaining over microseconds from a team downstream, in front of a FE dev who’s spent his week optimizing the monotonically-increasing JS bundles with code splitting heuristics. Of course, it was because the client app recently went over the 100MB JS budget. Which they decided to make because the last time that happened, customers abroad reported seeing “white screens”. International conversion dropped sharply not long after that. It’s pretty silly. So ya, good times indeed. Time to learn k8s.
- dangus 2y agoThis article specifically mentions that they are not running microservices and has pretty clearly defined motivations for making the migration.
- ec109685 2y agoWhy would moving to Kubernered from ECS introduce performance issues? They already has their architecture, largely, and just moved it over to K8s. They even mention they aren’t a micro service company.
- trhway 2y ago>there’s no mention of performance loss or gain after migration to illustrate performance cost i usually ask people what ping they have say from one component/pod/service to another and to compare that value to what ping they'd get between 2 linux boxes sitting on that gorgeous 10Gb/40Gb/100Gb or even 1000Gb network that they are running their modern microservices architecture over.
- ellieh 2y agoI imagine because the article mentions: > More broadly, we’re not a microservices company, and we don’t plan to become one
- dgb23 2y agoIt seems the "micro" implies that services are separated by high level business terms, like "payment" or "inventory" with each having their own databases instead of computational terms like "storage", "load balancing" or "preprocessing" etc. Is this generally correct? How well is this term defined? If yes, then I'm not surprised this type of design has become a target for frustration. State is smeared across system, which implies a lot of messaging and arbitrary connections between services. That type of design is useful if you are an application platform (or similar) where you have no say in what the individual entities are, and actually have no idea what they will be. But if you have the birds-eye view and implement all of it, then why would you do that?
- rco8786 2y agoOr cost increase or decrease!
- crossroadsguy 2y agoI see this in Android. Every few years (sometimes multiple times in a year) a new arch. becomes the fad and every TDH dev starts hankering for one way or the other on the lines "why are you not doing X..". Problem is, at immature firms (specially startups) the director and senior manager level leaders happily agree for the re-write i.e re-archs because they usually leave every season for the next place and they get to talk about that new thing which probably didn't last beyond their stay, or might have conked off before that. And the "test cases" porn! Goodness! People want "coverage" and that's it. It's a box to tick, irrespective of whether those test cases and they way those are written are actually meaningful in anyway. There are more like "let's have dependency injection everywhere" charade.
- alexpotato 2y agoAs often, Grug has a great line about microservices: grug wonder why big brain take hardest problem, factoring system correctly, and introduce network call too seem very confusing to grug https://grugbrain.dev/#grug-on-microservices https://grugbrain.dev/#grug-on-microservices
- tsss 2y ago> single micro service called by all of the other micro services So they didn't do microservices correctly. Big surprise.
- zbentley 2y agoI mean, that pattern is pretty common in the micro service world. Services for things like authz, locking, logging/tracing, etc. are often centralized SPOFs. There are certainly ways to mitigate the SPOFiness of each of those cases, but that doesn’t make having them an antipattern.
- jknoepfler 2y agoOne of the points of a microservice architecture (on k8s or otherwise) is that you can easily horizontally scale the component that was under pressure without having to scale out a monolithic application... that just sounds like people being clueless, not a failure of microservice architecture...
- okr 2y agoAnd if it was a single application, how would that solved it? You still would have that loop, no? Personally, i think, it does not have to be hundreds of microservices, basically each function a service. But i see it more as the web in the internet. Things are sometimes not reachable or overloaded. I think that is normal life.
- deleted 2y ago[deleted]