7 ms·
“50 thousand API endpoints exposed by microservices” Feels that their problem may just be they went a bit too far on the microservices band wagon.
by iamflimflam1 3y ago
“50 thousand API endpoints exposed by microservices”
Feels that their problem may just be they went a bit too far on the microservices band wagon.
- cbb330 3y agoSpeaking as a current engineer at LinkedIn, Big Tech companies will often conflate micro service with large distributed system. These services are by no means at all micro.
- bingemaker 3y agoCurious to know how many macroservices or monoliths LinkedIn has currently
- whatshisface 3y agoOur interlinked system of distributed monoliths includes one in AWS region af-south-1, another on the Moon, and a third orbiting Jupiter. All of these services are yours, except for the fourth on Europa. Attempt no POSTing there.
- jraph 3y agoOk for Jupiter and Europa, you need some redundancy and to be close to your users, but AWS would is a bit unexpected with LinkedIn belonging to MS. Then again, GitHub still uses AWS IIRC.
- midnitewarrior 3y agoConvert all of my JSON to protobuf to get 60% performance improvement? I'm sorry Dave, I'm afraid I can't do that.
- olliej 3y agoI feel that wholly depends on how you’re classifying those. What makes a service micro vs macro?
- meinheld111 3y agoIt’s micro if it pulls <1gig of dependencies </s>
- wg0 3y agoI have seen 5 table micro services. A solo backend developer had created "3 micro services" and as I joined the team, their argument was that we're doubled in strength so there were a series of architectural meetings to split everything nicely into 5 different services.
- Cthulhu_ 3y agoHello world in NodeJS?
- mitchitized 3y ago* clones hello_world repo cd hello_world npm install * watches text scroll for five hours
- usrusr 3y agoThere's certainly a spectrum, but you can have huge clusters of even the most monolithic monolith. The real semantic fun begins when you have monolithic codebase/deployments that could do everything, but each instance gets assigned a specialized role it serves to its peers _ (I think I've seen someonevdescribing that approach somewhere). I'd consider that micro if there are more than a handful of "roles", perhaps with some qualifier, but it would hopefully be easy to agree than no description is entirely wrong.
- fnordpiglet 3y agoThere are 19,000 LinkedIn employees world wide according to my cursory Kagi. That means more than two endpoints for every employee. I feel like regardless of micro or macro that’s an enormous sprawl of entropy.
- xnx 3y agoAre we verbing kagi now?
- jraph 3y agoI'd whynot this.
- 9dev 3y agoIn which case I'll decisively nope that.
- jraph 3y agoMore seriously though, I wish we used generic terms instead of using brands. "Look up" (or lookup as a noun) or "search" can't be too hard to use. We don't need to turn our thoughts into ads at every corner. (so, yes, I'll take your nope for my whynotting) Now, if you are launching a search engine, you need to name it wisely for when people verb it. As a brand you probably want this to happen. Duck Duck Go was not very wise.
- fnordpiglet 3y agoGiven that’s already happened with Google that’s a bit late. I’d rather support Kagi than reinforce Googles brand.
- wood_spirit 3y agoThe term “endpoint” is pretty fuzzy. Often it’s meant how many different url handlers there are (eg a service exposing “customers” might have dozens of endpoints for searching them, listing them, creating, updating and deleting them etc. So number of endpoints can quickly grow to a big number.
- HatchedLake721 3y agoHow’s Ember.js doing these days at LinkedIn?
- photonerd 3y agoPretty good tbh!
- jonathankoren 3y agoAs a former LinkedIn engineer, let me ask, what happened? LI was using protobufs and BJSON like ten years ago. That was part of the whole conversion from Java serialization to rest APIs. This article implies that tossed everything, went back to text based JSON, and then said, “Well somabitch! Binary formats are faster than uncompressed text after all!”
- bezout 3y agoDo you really need that many endpoints even at LinkedIn scale? I’d expect a lots of them is due to engineers reinventing the wheel due to undocumented endpoints
- Aeolun 3y agoNever want to deprecate an endpoint, so you end up with /service/v1, /service/v2, /service/v3… /service/v37
- paulmd 3y agoOne of the problems with API versioning is that it’s really a contract for the whole shebang and not just a specific endpoint. You almost certainly want them to move in sync with each other. So if you have an API with 10 endpoints, and one of them changes 10 times… you now have 100 endpoints.
- heavenlyblue 3y agoIt doesn't make sense, are you implying that every change changes the behaviour of every single endpoint?
- constantly 3y agoI’m trying to parse this too and am reaching the same conclusion. Basically they seem to be implying with the “100 endpoints” number a scenario where I have 10 endpoints and change endpoint 0v1 and now call it endpoint 0v2, I must duplicate endpoints in the range 1v1 to 9v1 to go alongside my endpoint 0v2 so I serve them all together. This doesn’t make sense to me as there’s no reason to upgrade or duplicate the other nine endpoints just because I updated the first as far as I can tell. It really reaches absurdity when the first endpoint is on its tenth iteration (the others haven’t changed) and now you’re serving ten duplicate endpoints per version, or 100 total endpoints where 90 of them are duplicate of themselves.
- 3y ago
- Gigachad 3y agoMaybe they just call every single data object they expose it’s own api endpoint even if the vast majority come from the same program.
- Voultapher 3y agoI remember a talk about the Zalando infrastructure, and they had some 500 Kubernetes clusters, that's not even individual services. Sorry but no one can convince me that is an appropriate or needed amount to sell cloths online.
- zuppy 3y agowhy does it matter what you sell? the backend of a big online retailer is extremely complex. i agree that 500 sounds large, but (1) i'm not sure that's real and (2) we don't know if this doesn't cover other related businesses. the part about selling clothes baffles me.
- 9dev 3y agoA K8s cluster with 500 nodes sounds more sensible to me.
- jongjong 3y agoAgreed. The headline leads people towards a deceptive conclusion. I've worked with both JSON and Protocol Buffers and there is no way that merely changing from JSON to PB would reduce latency by 60%; not on an average request. Clearly other refactorings were undertaken to achieve that speedup. I've done some raw performance comparison tests between JSON and PB and the speedup with PB was insignificant for most of the payloads I tried. Does any developer here on HN really believe that JSON parsing (plus schema validation) is what adds the most latency to a request? It just doesn't add up that just switching to PB would deliver that speedup. It reminds me of another headline I read a few years ago about a company thanking TypeScript for reducing bugs by 25% (I can't remember what the exact number was) after a rewrite... Common. You rewrote the whole thing from scratch. What actually cut the bugs by 25% was the rewrite itself. People don't usually write it more buggy the second time... It's a pattern with these big tech consultants. They keep pushing specific tech solutions as silver bullets then they deliver flawed analyses of the results to propagate the myth that they solved the problem in that particular way... When in fact, the problem was solved by pure coincidence due to completely different factors.
- avidphantasm 3y agoDevil’s advocate: I would generally expect that a rewrite would increase bugs, at least at first.
- HankB99 3y agoI would expect that the target for a rewrite would be better known than the during the first write and that the morphing to hit a moving target would provide more opportunities for bugs. OTOH if there was a lot of pressure to get the rewrite done, that would be conducive to producing buggy code. I think management would be a bigger factor than any technical issues.
- Cthulhu_ 3y agoOther sources say protobuf is 6x as fast than JSON though. I mean with REST / JSON you get the overhead of converting data to JSON objects, JSON objects to text representation, text to gzip or whichever, then the transfer over TCP / HTTP (establish connection, handshake, etc), the actual transfer, and then gzip -> text -> json -> data again. Protobuf is a lot more direct and compact, you get a lot of those things for free, and that's even before considering the schema and code generation which is still far from ideal in the REST/JSON world. I don't believe that for the purposes of inter-service communication, REST/JSON can compete with protobuf. Caveat: I only have experience with REST/JSON and a little GraphQL, I haven't had the opportunity to work with protobuf yet. I'm more of a front end developer unfortunately, and I try to talk people out of doing microservices as much as possible.
- highwaylights 3y agoThe final few sentences quietly mention that they’re going to migrate from Rest.li to gRPC, which seems like the bigger story here for anyone that’s adopted the open-sourced Rest.li for their own projects.
- Alifatisk 3y ago50 thousand endpoints, oh my gosh.
- joshhart 3y agoFormer director at LinkedIn. There might be that many internal URL endpoints (I am skeptical though). There are far fewer services than that though, each service supports many APIs.