5 ms·
Meta reveals serverless platform processing trillions of function calls a day
- eddtries 3y agoWorks out to around 115 function calls a second per server
- booi 3y agoThat doesn’t sound right… maybe per user?
- eddtries 3y ago1 Trillion function calls over 100,000 servers. Technically they say trillions over hundreds of thousands, but I went for the lower case. 1,000,000,000,000 / 100,000 = 10,000,000 10,000,000 / 24 / 3600 = 115.7
- xxs 3y agoWe don't know what the servers are; dual socket EPIC, i.e. 128cores, makes the number look beyond trivial.
- kunley 3y agoSo... 115.7 rps doesn't sound groundbreaking, right?
- eddtries 3y agoWas just trying to break the number down to something more easy to understand, I don’t know enough if this is impressive or not! Depends on the complexity of the request, and I guess the complexity of routing that many requests over such a large network. I’ve never worked at that scale.
- nick0garvey 3y agoThis isn't unreasonable. ML workloads benefit from more computational time per request. Lower QPS = better results.
- kyeb 3y agoThe calls per server is probably not the difficult part - this is the type of scale where you start hitting much, much harder problems, e.g.: - Load balancing across regions [0] without significant latency overhead - Service-to-service mesh/discovery that scales with less than O(# of servers) - Reliable processing (retries without causing retry storms, optimistic hedging) - Coordinating error handling and observability All without the engineers actually writing the functions needing to know anything about the system (which requires airtight reliability, abstractions, observability). I don't mean to comment on whether this is impressive or not, just pointing out that per-server throughput would never be the difficult part of reaching this scale. [0] And apparently for this system, load balancing across time, which is at least a mildly interesting way of thinking about it
- fnordsensei 3y ago“Trillions of functions” is a metric that’s hard to know whether to be impressed by or not. I don’t think it’s impossible that my laptop runs “trillions of functions” every day. But the callers on this platform are likely remote, and therefore it handles I/O as well, etc. Like I said, hard to understand whether it’s impressive or not. I’ll assume it’s impressive.
- hiddencost 3y agoTBH anyone using "per day" I assume is dishonest. QPS is the standard measure, and per day is just a sly way to multiply by 86400. If you want an average measure then report the average and peak QPS ...
- zmgsabst 3y agoI divided through, assuming sustained rate: 1T/day = 11.575M/s I personally find 11.5M/s a lot more impressive sounding. Though another comment suggests 100k servers — for about 100/s per server. 10ms per request isn’t particularly good or bad; volume is still impressive.
- deleted 3y ago[deleted]
- almost_usual 3y ago> One example of load they demonstrate has 20 million function calls submitted to XFaaS within 15 minutes. > Meta’s XFaaS is their serverless platform that “processes trillions of function calls per day on more than 100,000 servers spread across tens of datacenter regions.” This seems very low per server?
- fhd2 3y agoJust for one trillion (not "trillions"), that'd be 10 million function calls per server per day, ~7k per minute. Sounds about right to me. They'll surely want to leave some room for traffic spikes, server failures and unforeseen issues. Server uplink could also be a factor - at least that was the major bottleneck the last time I ran infrastructure serving lots of clients (about 100 million), but they are probably smarter about this than I was back then.
- type_Ben_struct 3y agoAs an aside, I’m always surprised at just how bad Meta’s software quality is, especially for a FAANG company with lots of expensive engineers. Instagram and Facebook are some of the buggiest platforms I use. I often run into bugs in several key user journeys that go unresolved and it baffles me how they don’t pick up on them. One of the problems is it’s basically impossible to get in touch with Meta to report them.
- DoingIsLearning 3y agoNo positive incentives, nobody gets promoted doing bug fixes, everybody tries to focus on hot features. No negative incentives, it's not safety-critical so nobody will recall or sue.
- kristiandupont 3y agoI can't speak to what is going on but one thing I've heard that makes things difficult for at least the frontend teams is that they are battling ad blockers. In order to work around them, they create incredibly obfuscated HTML that no one would normally ever create.
- benterix 3y agoSo how is this working for them? Because thanks for uBlock Origin I haven't seen an ad on FB for years.
- disgruntledphd2 3y agoAdblock plus is what they focused on, back in the day at least.
- c7DJTLrn 3y agoAnd this is where some of the greatest, most educated minds in computing are going. What a waste.
- KingOfCoders 3y agoIf you think they are buggy, become a seller on Amazon. The buggiest software I have ever had. And they are influencers with the "two pizza rule". The second worst is Spotify. It still does not remember where you've stopped listening on radio plays after a decade. From the influencer people who brought you "tribes".
- foolfoolz 3y agosomehow every comment in here is downplaying this achievement as “low” or unimpressive you truly underestimate the scale of an operation like this the vast majority of software companies will never count a trillion of anything. even big companies that scale will only have a small subset of teams work on something this large
- almost_usual 3y agoThis is HN, there are definitely users on this site who have experienced or have worked at places with these workloads.
- darkwater 3y agoAnd this is also the HN where people boosts they can rebuild on their own $SUCCESSFUL_SOFTWARE over a 3-days weekend. There are tons of very brilliant and very smart people here, but there are also many that are too fond of themselves or have big issues understanding problems' ramifications in real life/real business.
- kyleyeats 3y agoHow do you know the difference between stupidity and ambition unless you try?
- rokkitmensch 3y agoThe best inoculation against hubris is trying to fly to the sun. Humans, to generalize, bias towards talking over walking.
- darkwater 3y agoAmbition is saying "I think I can bootstrap a successful competitor to $POPULAR_SOFTWARE if I work hard enough, I have the talent and perseverance". Adding "in 3-days" is stupidity.
- hiddencost 3y ago
- charcircuit 3y agoWhat's the difference between a PHP function and regular PHP?
- friendzis 3y agoNot having to spin up and down whole interpreter/VM is, apparently, such a performance boost that some companies used to make whole business out of it
- fxtentacle 3y agoabout 500x price markup going from a dumb bare metal server to AWS Lambda I believe for PHP there is little difference. Those function hosting solutions are typically used with Java where starting the JVM for every single request would be too much overhead.
- hexo 3y agoThe only impressive about this thing is technology abuse to do such a slow job on such a large scale. I recall ruby on rails doing at most 150 requests per second on a laptop 15 years ago, which was laughably low even for those days. I could do hundreds times more using c++, or at least tens times more using unoptimized python. Their achievement should not be praised, on contrary they need to be told they're wasting resources, heating up planet and emit carbon dioxide for no good reason. This is not how optimization looks like. And no, it is not serverless, since obviously something serves that stuff, stop lying already.
- CommanderData 3y agoWhat sort of isolation did you achieve with your Ruby experiments?
- klodolph 3y agoTrillion per day is 10M per second. That’s hard. “Serverless” just means you’re not running a long-running process that handles these specific requests.
- hexo 3y agoOn multiple 100k servers? Not as much as you think. Serverless means you have no server. Stop repeating amazons lies. It is nonsense.
- klodolph 3y ago“Server” is a long-running process that serves requests. “Serverless” means that your requests are handled by short-lived processes of some kind. This is just the name for it. If you want to poke holes in the definition I gave, it won’t do any good, because definitions are kinda fuzzy and incomplete to begin with. If you try to argue about what “serverless” means, you’re just going to end up drifting away from the consensus about what words mean. Consensus is the only thing that matters here. Consensus generally won’t be universal—I know people that refuse to accept “literally” as an intensifier—but that is a problem we have to deal with.
- samsquire 3y agoThank you for this. I am a beginner hobbyist in server tech. I am running into the perennial latency Vs throughout problem. A batch of queue can hide latency problems. If you process 1 million per second (1 batch or 1 million separate requests) with 100 nanosecond latency or 10 microseconds per batch. In XFaas do they run an interpreter in a thread ? Or a warm process?
- choeger 3y agoI assume these are mostly procedures, not functions, (that is, they have side effects) as applying a function 1.3M times per minute immediately raises the question of caching which doesn't seem to be important for them. But if you execute that many procedure calls, how do you guard against them influencing each other due to their side effects? Memory leaks come to mind, or other weird bugs. Also, how do you manage the credentials to effectively issue side effects in a (hopefully) zero-trust environment?
- sspiff 3y agoOff topic: I ended up not finishing the article, because first it presented me with a full screen overlay asking me to sign up to their news letter, forcing me to click on "I want to read it first" to see the article. Then after scrolling down two paragraphs, it presented me with another overlay blocking the article, again asking me to sign up for the newsletter. Can we please stop rewarding these kinds of practices with traffic by posting and upvoting them?
- Nextgrid 3y agoA solution to these would be to build a browser extension that automatically submits any such forms with bogus email addresses. It can't easily be blocked by a captcha as that would hurt conversion rates of legitimate users.
- sspiff 3y agoI wonder how much actual conversion happens in these kind of forms, anyway. Who are the people that sign up to newsletters of a random website before even reading anything? And who signs up after reading just an intro? I get the idea of the mailing lists - generate traffic that does not rely on external platforms, with their ever changing algorithms that need to be appeased and courted, and who might even change the rules or ban you from promoting your content. But sure the conversion rates for these full screen modal overlays must be lower than the attrition you get from scaring away passer by readers?
- AndrewDucker 3y agoMy first question there is...could they do the same work with fewer calls?