7 ms·

XFaaS: Hyperscale and Low Cost Serverless Functions at Meta

194 points by greghn 3y ago

deleted 3y ago
[deleted]
danpalmer 3y ago
> Furthermore, XFaaS explicitly does not handle functions and the path of a user-interaction The wording of this is a little strange, but does this really mean that XFaaS is not used to serve end-user traffic at all? The general approach here is interesting. I've always thought that there are two potential benefits for Function-based hosting ("Serverless") – low cost of components via scale-to-zero infrastructure (good for rare events, or highly variable traffic), and developer experience (coding against an infra framework rather than build-your-own infra). I'd have expected the former to be much less of an issue at scale, to the point where it's not the leading benefit, so I expected to hear more about the dev productivity side, but instead this article focuses significantly on the performance and cost side of things.
- withinboredom 3y ago
  > but does this really mean that XFaaS is not used to serve end-user traffic at all? When I worked at a company with a similar-ish setup, we were told that if you could do it in less than a minute and the user has to wait on it, do it in the request, don't send it to a job. This was because it was "cheaper" to keep a thread running vs. spinning up the resources to do a job. When I say cheaper, I mean in dev-time, user experience, and actual resources. 1) on the front-end, the dev doesn't have to "refresh" or "poll" an endpoint to get the status. 2) user experience is better. many people naively check "every 30s" or something ... well, what happens if services are degraded and it takes more than 30s to respond to your status check? Now, after a few minutes, you have dozens of pending requests for the exact same resource. 3) we didn't have a fancy SLO priority queue, just a regular one, so sometimes, you could have your job stuck behind a ton of jobs that are a higher priority than you, causing what was only 30s in your tests to now take 2 hours with the user polling your status endpoint and randomly refreshing, then eventually contacting support asking why it isn't working.
- mlerner 3y ago
  Author of the paper summary here - my understanding is that XFaaS doesn't run functions that are run in response to user input (e.g. XFaaS does not execute code that fetches and returns data because a user clicked on a button).
- KaiserPro 3y ago
  I think they are trying to hammer home that its not a realtime system. Anything you pass into will be executed at some point in the future. so if you want something synchronous, its not the tool to use.
Rodeoclash 3y ago
Any insight to what kinds of work is being run in the functions? I run a lot of jobs in background queues but I'm wondering if I'm missing out on some new paradigm.
- maigret 3y ago
  They recently mentioned functions as well in an article about the Threads infrastructure. https://engineering.fb.com/2023/12/19/core-infra/how-meta-built-the-infrastructure-for-threads/ https://engineering.fb.com/2023/12/19/core-infra/how-meta-bu...
- WJW 3y ago
  No it's basically just a big background job cluster, but for all the microservices at the same time so they can get a higher average utilization out of their servers by averaging out the peaks.
- withinboredom 3y ago
  It's a "job queue" in old-people-words, just managed by someone else. Sometimes they even offer integrations like putting a job on the queue on database triggers, or file changes. Nothing really new, except that you don't have to build it yourself ... but if you do and you sell it to your coworkers/clients/customers, you just call it FaaS.
- xylophile 3y ago
  It's fundamentally the same as a job queue, but the difference is that the people writing the job are not creating a running OS process. You literally just write a function, and it gets compiled into a process owned and executed by the job system. Why would you want that? Well, who really wants to think about the OS, or how to get their data into main()? You just want to write business logic, and FaaS lets your developers focus on that. It's a small development process optimization, but a significant one at scale if you have enough developers / unique jobs being created. And it lets platform engineers focus on the best way to shovel data into main() in your particular environment.
freedomben 3y ago
It's interesting to see the heavily growing demand graph. Is that because people want to adopt it, or is it being mandated or encouraged as "best practice" etc? I'm not implying that true organic demand wouldn't exist because it definitely might, but I have seen in practice where leadership encourages or even mandates usage of FaaS, so the numbers go up even though on a neutral field people wouldn't necessarily choose it. There's also the "I'd like to try it" group who hasn't yet had experience with it so choose it as a target for learning/curiosity reasons. I wonder because my own experience with FaaS has been mostly bad. There are some nice things for sure, and a handful of use cases where it's wonderfully superior, such as executing jobs where number of concurrent processes fluctuates wildly making rapid scalability highly desirable. The canonical use case being "make a thumbnail for this image." For web servers though, I find the opacity and observability, and the difficulty running locally to be significant hindrances. There's also lock-in which I hate with a passion. At this point I'd rather manage static EC2 instances than Lambdas, for example. (To be clear I'm not advocating static EC2 instances. My preference is Kubernetes all the things. Not perfect of course, but K8s makes horizontal scalability very easy while improving on visibility, but that's a different conversation)
- willvarfar 3y ago
  Yeah it sucks to be a FaaS customer on a cloud run by someone else... you're just overpaying for easy instead of simple etc. But if you're in Meta, and you're running on an abstraction built and maintained by Meta, then the it's running for cost instead of profit and the the incentives all align between user and infrastructure provider? I imagine the development and deployment ease for a system at Meta for Meta could be dreamy. At least, it has the potential to be... :)
- charcircuit 3y ago
  From the paper: >The rapid growth at the end of 2022 is due to the launch of a new feature that allows for the use of Kafka-like data streams [12] to trigger function calls In regards to opacity and observability I don't see why it would be any worse to run PHP code on XFaaS than running the PHP code on another host.
rafaelturk 3y ago
Inovation regarding FaaS comes from BigTechs. I wish we had more OpenSource FaaS solutions, albeit OpenFaas amazing achievements still lacks more robust ecosystem.
- smt88 3y ago
  What would the point of self-hosted FaaS be? They lose their whole value proposition when you're paying for dedicated servers or when you don't have large numbers of servers.
  - willvarfar 3y ago
    FaaS could be sharing your hardware at a function level inside the JVMs or whatever you are running, rather than at the VM or hypervisor level. You can (potentially) get much better elasticity and utilisation?
    - Capricorn2481 3y ago
      Isn't that just functions?
      - p_l 3y ago
        Isn't FaaS just wrapping around functions and their deployment?
        Capricorn2481 3y ago
        Yes, but I'm just confused because saying "Why isn't there an open source FaaS for on prem that shares hardware" makes no sense. At that point, you're just talking about regular coding with functions. Unless there's something I'm missing.
        p_l 3y ago
        There's a bunch of open source software that can do it for you, but it's very much "assemble this pile of blocks"
realprimoh 3y ago
Related discussions: https://news.ycombinator.com/item?id=37981813 https://news.ycombinator.com/item?id=37981813 https://news.ycombinator.com/item?id=39029439 https://news.ycombinator.com/item?id=39029439 Both link to https://read.engineerscodex.com/p/meta-xfaas-serverless-functions-explained https://read.engineerscodex.com/p/meta-xfaas-serverless-func... The actual paper is https://dl.acm.org/doi/abs/10.1145/3600006.3613155 https://dl.acm.org/doi/abs/10.1145/3600006.3613155
zokier 3y ago
> To reduce latency, a common approach is to keep a VM idle for 10 minutes or longer after a function invocation to allow for potential reuse [45]. In contrast, if a FaaS platform is optimized for hardware utilization and throughput, this waiting time should be reduced by a factor of 10 or more, because starting a VM consumes significantly fewer resources than having a VM idle for 10 minutes This seems somewhat surprising in such controlled environment. Why does idle function consume so many resources? If I think a normal Linux system, an idle process doesn't really consume much resources at all
- mike_hearn 3y ago
  RAM, disk. Functions are often written in high level JITd languages, they may need to load large reference datasets from disk, they may require a lot of code to be transferred over the network before they can begin running, and because nobody trusts the security of the Linux kernel you also have to pay the VM startup time. An idle process on Linux isn't much different: it consumes RAM, disk and a kernel. If you trust the software you're running enough you can amortize the kernel cost but the others remain.
  - dig1 3y ago
    > and because nobody trusts the security of the Linux kernel Seriously? I imagine all the infra worldwide is run on FaaS and short-lived VMs, right?
    - jitl 3y ago
      What they mean is that if you trusted the kernel you’d run functions in containers instead of micro-VMs, since containers need fewer resources and better start/stop latency.
      - mike_hearn 3y ago
        Yes. Old-style pre AWS clouds (ISPs with an FTP+LAMP stack) worked by using UNIX user sharing features. It went out of fashion because Linux wasn't good enough at work isolation and because it was insecure - too many local root escalation vulns. So now clouds all run on custom hypervisors. The VM/hypervisor interface is smaller and easier to secure than the userspace/kernelspace interface. Strangers don't share kernels, they share hypervisors. Unfortunately FaaS platforms actually suffer twice, because Linux userland is too chaotic to use directly. So you have to boot a clean VM with Docker in it, then install a container into that VM so the user can send you software in the now-standard format, then start up the function. Booting a Docker container from a standard running Linux system already takes ~100msec if I recall correctly. But 100msec of added latency can actually reduce usage and hurt revenue on big sites, so it's not acceptable. And that's just the inner container. Then you have the cost of booting Linux, cost of downloading the container (docker format is highly unoptimized) etc. So all these costs add up and then the only way to solve them is to amortize them. Oracle Cloud is working on a thing called GraalOS which is intended to help address this. It works by letting apps share a Linux kernel and using a userspace "supervisor" that relies on CFI, MPKs and NaCL style binary analysis to prevent code from connecting to the kernel directly. Containers and VMs are no longer necessary in that model and loading an app is just a case of copying it down to a node and mmapping it, but it does mean you have to be able to compile your app for this alternative pseudo-operating system. If you work with Java then you use native-image that makes Java apps compile to native code and start up super fast, so it works. https://graal.cloud/graalos/ https://graal.cloud/graalos/
create-card 3y ago
[dead]
chrishare 3y ago
Intuitively, the popularisation of FaaS feels inevitable as lower level technical challenges get solved and give way to abstractions like this. Who knows though what our stack will look like in 20 years.
- pjmlp 3y ago
  Looking behind what I have seen since I started paying attention in the 1980's, as it looks today, resold under new marketing terms, resold by newly founded startups that are disrupting the ecosystem.
  - dijit 3y ago
    I agree, FaaS is treading deep "CGI" territory, except there are a lot more resources available for nice to have operational overheads so people think it's largely different.
- mhd 3y ago
  As Gibson said, "The future is already here, it's just not very evenly distributed". And in IT, sometimes the future takes a long nap. I wouldn't be too surprised if e.g. descendants of Linda spaces get re-discovered because external circumstances favor that model. Although with FaaS, I don't see anything particularly new from a development perspective. It's, well, functions. Most of the time they aren't communicating in any kind of novel way, and often they're doing the decade-old stuff of reading files and slurping databases. The abstraction being more on the operational side of things this time. Not that I'm complaining or doing the "it's just CGI" dance. Compared to other "cloudy" tech, there's actually potential for simplifying things and not just simulated VAX computers with more effort…
- asdfman123 3y ago
  But what is left to "solve" here? Functions already work pretty well for what they do. But any abstraction runs the risk of revealing itself as a leaky abstraction, not worth the overhead. That's been my experience with functions.
aestetix 3y ago
"Function as a Service" This sounds familiar... http://left-pad.io/ http://left-pad.io/
DeathArrow 3y ago
As a developer who worked at small to medium sized companies, mainly developing microservices running on Kubernetes, I don't see a big advantage using FaaS as a customer. Maybe I am missing something? I am sure the provider gets more utilization for the hardware, but for the customer what's the point?
- sanderjd 3y ago
  Isn't it that you don't have to set up and maintain a kubernetes cluster?
  - btreecat 3y ago
    Right, you just need a collection of XFaaS services, which need to be hosted somewhere... So you'll want some container orchestration tool, might as well use k8s.
  - DeathArrow 3y ago
    If setting up a Kubernetes cluster isn't a terrible overhead for us, switching to FaaS would mean we would have more control or less control? I am thinking mainly of migration to another provider. Maybe we would benefit more if we would have large spikes in resource usage, but we don't.
    - jppittma 3y ago
      FaaS is an abstraction over a (several) managed k8s clusters. Like most abstractions, you're trading convenience for power/flexibility.
- WJW 3y ago
  Most of the companies I've worked for or consulted with had some sort of background worker system for , usually with a queue in postgres/redis/rabbitMQ/kafka/whatever in between. It usually gets used for non-latency sensitive work like sending emails instead of doing that from the main request handler, so that the user gets their response without having to wait for the other stuff to be completed. Bigger companies sometimes have multiple groups of background workers, one for each service and each with their own queues, servers and autoscaling policies. As I understand the XFaaS system from the article it functions a bit like a consolidated background worker system that any service can submit their work to and it will (try to) make sure it gets done in whatever SLO is specified. This is usually cheaper because now you can do low-urgency work from service B when there are urgent jobs from service A and vice versa, leading to higher average server usage. For small/medium businesses this is probably not something you'd want, since developing and operating such a system costs more than you save. But if you have a multi-million dollar bill for your background workers alone then it might be worth it.
Erwin 3y ago
I'm curious whether their "Locality Groups" concept can be implemented using stock AWS. Imagine you have 1000 type of tasks that can be called millions of times by users; each node can do any task but resource usage would be better if same task was performed on a node that already has done it. Sticky sessions is not it, as the same task can be initiated by different users.
RebekahKlein 3y ago
[dead]

194 points by greghn 3y ago

deleted 3y ago

[deleted]

> Furthermore, XFaaS explicitly does not handle functions and the path of a user-interaction The wording of this is a little strange, but does this really mean that XFaaS is not used to serve end-user traffic at all? The general approach here is interesting. I've always thought that there are two potential benefits for Function-based hosting ("Serverless") – low cost of components via scale-to-zero infrastructure (good for rare events, or highly variable traffic), and developer experience (coding against an infra framework rather than build-your-own infra). I'd have expected the former to be much less of an issue at scale, to the point where it's not the leading benefit, so I expected to hear more about the dev productivity side, but instead this article focuses significantly on the performance and cost side of things.

withinboredom 3y ago
> but does this really mean that XFaaS is not used to serve end-user traffic at all? When I worked at a company with a similar-ish setup, we were told that if you could do it in less than a minute and the user has to wait on it, do it in the request, don't send it to a job. This was because it was "cheaper" to keep a thread running vs. spinning up the resources to do a job. When I say cheaper, I mean in dev-time, user experience, and actual resources. 1) on the front-end, the dev doesn't have to "refresh" or "poll" an endpoint to get the status. 2) user experience is better. many people naively check "every 30s" or something ... well, what happens if services are degraded and it takes more than 30s to respond to your status check? Now, after a few minutes, you have dozens of pending requests for the exact same resource. 3) we didn't have a fancy SLO priority queue, just a regular one, so sometimes, you could have your job stuck behind a ton of jobs that are a higher priority than you, causing what was only 30s in your tests to now take 2 hours with the user polling your status endpoint and randomly refreshing, then eventually contacting support asking why it isn't working.
mlerner 3y ago
Author of the paper summary here - my understanding is that XFaaS doesn't run functions that are run in response to user input (e.g. XFaaS does not execute code that fetches and returns data because a user clicked on a button).
KaiserPro 3y ago
I think they are trying to hammer home that its not a realtime system. Anything you pass into will be executed at some point in the future. so if you want something synchronous, its not the tool to use.

Rodeoclash 3y ago

Any insight to what kinds of work is being run in the functions? I run a lot of jobs in background queues but I'm wondering if I'm missing out on some new paradigm.

maigret 3y ago
They recently mentioned functions as well in an article about the Threads infrastructure. https://engineering.fb.com/2023/12/19/core-infra/how-meta-built-the-infrastructure-for-threads/ https://engineering.fb.com/2023/12/19/core-infra/how-meta-bu...
WJW 3y ago
No it's basically just a big background job cluster, but for all the microservices at the same time so they can get a higher average utilization out of their servers by averaging out the peaks.
withinboredom 3y ago
It's a "job queue" in old-people-words, just managed by someone else. Sometimes they even offer integrations like putting a job on the queue on database triggers, or file changes. Nothing really new, except that you don't have to build it yourself ... but if you do and you sell it to your coworkers/clients/customers, you just call it FaaS.
xylophile 3y ago
It's fundamentally the same as a job queue, but the difference is that the people writing the job are not creating a running OS process. You literally just write a function, and it gets compiled into a process owned and executed by the job system. Why would you want that? Well, who really wants to think about the OS, or how to get their data into main()? You just want to write business logic, and FaaS lets your developers focus on that. It's a small development process optimization, but a significant one at scale if you have enough developers / unique jobs being created. And it lets platform engineers focus on the best way to shovel data into main() in your particular environment.

freedomben 3y ago

It's interesting to see the heavily growing demand graph. Is that because people want to adopt it, or is it being mandated or encouraged as "best practice" etc? I'm not implying that true organic demand wouldn't exist because it definitely might, but I have seen in practice where leadership encourages or even mandates usage of FaaS, so the numbers go up even though on a neutral field people wouldn't necessarily choose it. There's also the "I'd like to try it" group who hasn't yet had experience with it so choose it as a target for learning/curiosity reasons. I wonder because my own experience with FaaS has been mostly bad. There are some nice things for sure, and a handful of use cases where it's wonderfully superior, such as executing jobs where number of concurrent processes fluctuates wildly making rapid scalability highly desirable. The canonical use case being "make a thumbnail for this image." For web servers though, I find the opacity and observability, and the difficulty running locally to be significant hindrances. There's also lock-in which I hate with a passion. At this point I'd rather manage static EC2 instances than Lambdas, for example. (To be clear I'm not advocating static EC2 instances. My preference is Kubernetes all the things. Not perfect of course, but K8s makes horizontal scalability very easy while improving on visibility, but that's a different conversation)

willvarfar 3y ago
Yeah it sucks to be a FaaS customer on a cloud run by someone else... you're just overpaying for easy instead of simple etc. But if you're in Meta, and you're running on an abstraction built and maintained by Meta, then the it's running for cost instead of profit and the the incentives all align between user and infrastructure provider? I imagine the development and deployment ease for a system at Meta for Meta could be dreamy. At least, it has the potential to be... :)
charcircuit 3y ago
From the paper: >The rapid growth at the end of 2022 is due to the launch of a new feature that allows for the use of Kafka-like data streams [12] to trigger function calls In regards to opacity and observability I don't see why it would be any worse to run PHP code on XFaaS than running the PHP code on another host.

rafaelturk 3y ago

Inovation regarding FaaS comes from BigTechs. I wish we had more OpenSource FaaS solutions, albeit OpenFaas amazing achievements still lacks more robust ecosystem.

smt88 3y ago
What would the point of self-hosted FaaS be? They lose their whole value proposition when you're paying for dedicated servers or when you don't have large numbers of servers.
- willvarfar 3y ago
  FaaS could be sharing your hardware at a function level inside the JVMs or whatever you are running, rather than at the VM or hypervisor level. You can (potentially) get much better elasticity and utilisation?
  - Capricorn2481 3y ago
    Isn't that just functions?
    - p_l 3y ago
      Isn't FaaS just wrapping around functions and their deployment?
      - Capricorn2481 3y ago
        Yes, but I'm just confused because saying "Why isn't there an open source FaaS for on prem that shares hardware" makes no sense. At that point, you're just talking about regular coding with functions. Unless there's something I'm missing.
        p_l 3y ago
        There's a bunch of open source software that can do it for you, but it's very much "assemble this pile of blocks"

realprimoh 3y ago

Related discussions: https://news.ycombinator.com/item?id=37981813 https://news.ycombinator.com/item?id=37981813 https://news.ycombinator.com/item?id=39029439 https://news.ycombinator.com/item?id=39029439 Both link to https://read.engineerscodex.com/p/meta-xfaas-serverless-functions-explained https://read.engineerscodex.com/p/meta-xfaas-serverless-func... The actual paper is https://dl.acm.org/doi/abs/10.1145/3600006.3613155 https://dl.acm.org/doi/abs/10.1145/3600006.3613155

zokier 3y ago

> To reduce latency, a common approach is to keep a VM idle for 10 minutes or longer after a function invocation to allow for potential reuse [45]. In contrast, if a FaaS platform is optimized for hardware utilization and throughput, this waiting time should be reduced by a factor of 10 or more, because starting a VM consumes significantly fewer resources than having a VM idle for 10 minutes This seems somewhat surprising in such controlled environment. Why does idle function consume so many resources? If I think a normal Linux system, an idle process doesn't really consume much resources at all

mike_hearn 3y ago
RAM, disk. Functions are often written in high level JITd languages, they may need to load large reference datasets from disk, they may require a lot of code to be transferred over the network before they can begin running, and because nobody trusts the security of the Linux kernel you also have to pay the VM startup time. An idle process on Linux isn't much different: it consumes RAM, disk and a kernel. If you trust the software you're running enough you can amortize the kernel cost but the others remain.
- dig1 3y ago
  > and because nobody trusts the security of the Linux kernel Seriously? I imagine all the infra worldwide is run on FaaS and short-lived VMs, right?
  - jitl 3y ago
    What they mean is that if you trusted the kernel you’d run functions in containers instead of micro-VMs, since containers need fewer resources and better start/stop latency.
    - mike_hearn 3y ago
      Yes. Old-style pre AWS clouds (ISPs with an FTP+LAMP stack) worked by using UNIX user sharing features. It went out of fashion because Linux wasn't good enough at work isolation and because it was insecure - too many local root escalation vulns. So now clouds all run on custom hypervisors. The VM/hypervisor interface is smaller and easier to secure than the userspace/kernelspace interface. Strangers don't share kernels, they share hypervisors. Unfortunately FaaS platforms actually suffer twice, because Linux userland is too chaotic to use directly. So you have to boot a clean VM with Docker in it, then install a container into that VM so the user can send you software in the now-standard format, then start up the function. Booting a Docker container from a standard running Linux system already takes ~100msec if I recall correctly. But 100msec of added latency can actually reduce usage and hurt revenue on big sites, so it's not acceptable. And that's just the inner container. Then you have the cost of booting Linux, cost of downloading the container (docker format is highly unoptimized) etc. So all these costs add up and then the only way to solve them is to amortize them. Oracle Cloud is working on a thing called GraalOS which is intended to help address this. It works by letting apps share a Linux kernel and using a userspace "supervisor" that relies on CFI, MPKs and NaCL style binary analysis to prevent code from connecting to the kernel directly. Containers and VMs are no longer necessary in that model and loading an app is just a case of copying it down to a node and mmapping it, but it does mean you have to be able to compile your app for this alternative pseudo-operating system. If you work with Java then you use native-image that makes Java apps compile to native code and start up super fast, so it works. https://graal.cloud/graalos/ https://graal.cloud/graalos/

create-card 3y ago

[dead]

chrishare 3y ago

Intuitively, the popularisation of FaaS feels inevitable as lower level technical challenges get solved and give way to abstractions like this. Who knows though what our stack will look like in 20 years.

pjmlp 3y ago
Looking behind what I have seen since I started paying attention in the 1980's, as it looks today, resold under new marketing terms, resold by newly founded startups that are disrupting the ecosystem.
- dijit 3y ago
  I agree, FaaS is treading deep "CGI" territory, except there are a lot more resources available for nice to have operational overheads so people think it's largely different.
mhd 3y ago
As Gibson said, "The future is already here, it's just not very evenly distributed". And in IT, sometimes the future takes a long nap. I wouldn't be too surprised if e.g. descendants of Linda spaces get re-discovered because external circumstances favor that model. Although with FaaS, I don't see anything particularly new from a development perspective. It's, well, functions. Most of the time they aren't communicating in any kind of novel way, and often they're doing the decade-old stuff of reading files and slurping databases. The abstraction being more on the operational side of things this time. Not that I'm complaining or doing the "it's just CGI" dance. Compared to other "cloudy" tech, there's actually potential for simplifying things and not just simulated VAX computers with more effort…
asdfman123 3y ago
But what is left to "solve" here? Functions already work pretty well for what they do. But any abstraction runs the risk of revealing itself as a leaky abstraction, not worth the overhead. That's been my experience with functions.

aestetix 3y ago

"Function as a Service" This sounds familiar... http://left-pad.io/ http://left-pad.io/

DeathArrow 3y ago

As a developer who worked at small to medium sized companies, mainly developing microservices running on Kubernetes, I don't see a big advantage using FaaS as a customer. Maybe I am missing something? I am sure the provider gets more utilization for the hardware, but for the customer what's the point?

sanderjd 3y ago
Isn't it that you don't have to set up and maintain a kubernetes cluster?
- btreecat 3y ago
  Right, you just need a collection of XFaaS services, which need to be hosted somewhere... So you'll want some container orchestration tool, might as well use k8s.
- DeathArrow 3y ago
  If setting up a Kubernetes cluster isn't a terrible overhead for us, switching to FaaS would mean we would have more control or less control? I am thinking mainly of migration to another provider. Maybe we would benefit more if we would have large spikes in resource usage, but we don't.
  - jppittma 3y ago
    FaaS is an abstraction over a (several) managed k8s clusters. Like most abstractions, you're trading convenience for power/flexibility.
WJW 3y ago
Most of the companies I've worked for or consulted with had some sort of background worker system for , usually with a queue in postgres/redis/rabbitMQ/kafka/whatever in between. It usually gets used for non-latency sensitive work like sending emails instead of doing that from the main request handler, so that the user gets their response without having to wait for the other stuff to be completed. Bigger companies sometimes have multiple groups of background workers, one for each service and each with their own queues, servers and autoscaling policies. As I understand the XFaaS system from the article it functions a bit like a consolidated background worker system that any service can submit their work to and it will (try to) make sure it gets done in whatever SLO is specified. This is usually cheaper because now you can do low-urgency work from service B when there are urgent jobs from service A and vice versa, leading to higher average server usage. For small/medium businesses this is probably not something you'd want, since developing and operating such a system costs more than you save. But if you have a multi-million dollar bill for your background workers alone then it might be worth it.

Erwin 3y ago

I'm curious whether their "Locality Groups" concept can be implemented using stock AWS. Imagine you have 1000 type of tasks that can be called millions of times by users; each node can do any task but resource usage would be better if same task was performed on a node that already has done it. Sticky sessions is not it, as the same task can be initiated by different users.

RebekahKlein 3y ago

[dead]