6 ms·

Lambda is also distributed, though — while CGI scripts just live on the webserver itself. So if you're old like me and want a crisper mental model, the actual '

by derefr 2y ago

Lambda is also distributed, though — while CGI scripts just live on the webserver itself. So if you're old like me and want a crisper mental model, the actual '90s equivalent of Lambda would be: • a cluster of "diskless" web-server (let's say Apache) machines with a round-robin load-balancer in front of them; • where these web servers are all using mod_userdir, pointing /~*/ at, let's say, "/wwwhome/*/www/" — and with `Options +ExecCGI` enabled for all UserDirs; • where /wwwhome on all these web-server machines is a read-only NFS mount from the /home dir of a shared disk server; • and where each user of this service has an FTP-enabled user and home directory on the disk server. Given this setup, if a user bob FTPs a script to /home/bob/www/foo.pl on the disk server, then anyone can trigger that script to execute on some arbitrary web-server in the cluster by hitting http://lb.oldschool-lambda.example.com/~bob/foo.pl http://lb.oldschool-lambda.example.com/~bob/foo.pl. (I'm actually curious whether this exact architecture was ever implemented in the 1990s in practice, maybe by a university or ISP. It seems totally feasible.) --- One other major difference I should point out, is that Lambda functions are versioned — you don't overwrite a function, you just deploy a new content-hash-named copy of it and then update some "symbolic names" to point at it, but can always call a fixed version by its content-hash. Implementing enforced versioning wouldn't require any changes to the above architecture, though, so it doesn't matter here. (It'd "just" be a fork of ftpd that would transparently map between a versioned backend view and an unversioned frontend view. Very much like what Amazon S3 does for versioned buckets, actually.)

troupo 2y ago
> Lambda is also distributed, though — while CGI scripts just live on the webserver itself. Most of the stuff people use lambdas for they could run from the cheapest Digital Ocean droplet or cheapest Hetzner server. You need "omg distributed" perhaps after your millionth user, and even then it's highly debatable.
- derefr 2y ago
  You're thinking about the (rather few) advantages distribution confers for single-tenant scaling. But the distribution here is mainly for other reasons: • operational fault-tolerance: arbitrary servers in the cluster hosting the function can fail or be taken down for repair without any workloads going down. This allows the ops team that owns the cluster to freely keep the cluster's machines up to date with OS package updates, reboot them as needed, swap them out for newer-generation hardware, move them to new racks in new DC buildings, etc. (This is also something you get from a VM on most larger clouds — but only because the VM itself is a workload running on a hypervisor cluster, where the cluster's control-plane can live-migrate workloads to drain host nodes for maintenance.) • horizontally scaling the cluster as a whole, to keep up with more and more users sticking more and more workloads onto it, and more and more requests coming in for those workloads. As load increases, just add more diskless web servers. (This also means that in the rare case where your workload is too popular to fit on a cheap VPS, the ops team is doing your personal infrastructure scale-out for you for free, as part of scaling the capacity of the service as a whole.) • maintaining your multi-tenant Quality-of-Service in the face of other users who have hugely-expensive workloads. On that 90s cgi-bin webserver — or on a modern VPS — if someone else deploys some workload that pins all the CPU cores for five seconds each time anyone calls it, then that impacts how long it takes you to serve calls to your workload, because your request has landed on the same machine and is waiting in some kernel queue behind that other workload's CPU bottleneck. VMs solve this by reserving capacity per workload that's "always on" even when the workload is idle. Lambda solves this by collecting resource-accounting statistics for each function and resource-utilization metrics for each backend, and putting that info together at the load-balancer level to plan a hybrid of least-conn and resource-packing routing to backend nodes. --- But even ignoring all that, the point of the "distributed-ness" of the Lambda architecture on the low-usage end, is the "scales to zero" part that it shares with CGI. Unlike VMs, you aren't paying anything per month for a function nobody is calling. And for a workload that does get called, but only runs for N aggregate CPU-seconds each minute, you're only paying N/60 of what you'd pay for a VM (which runs every CPU-second of the every minute.) If you have one script and you have $5/mo burning a hole in your pocket, then sure, I guess you could put it on a DO droplet? (But you could just as well deploy a function that will cost you less than $5/mo.) But if you have ten scripts, where some of them might be high-utilization (you're not sure yet, depends on how many people hit the web pages that call them) while others are definitely extremely low-utilization — then your options are either "becoming a hobbyist DevOps engineer, utilization planning a packing arrangement of VMs or hardware to optimize cost, installing metrics to ensure things aren't falling over, getting emails when it does, re-planning things out, and migrating them as necessary"... or just deploying 10 functions and getting what will probably be a bill each month for <$1.
  - troupo 2y ago
    You write a long wall of text that does boring to dispute what I wrote. The cheapest DO droplet is 4 dollars a month. It is more than capable of running "10 scripts", and will last your startup with three users indefinitely long. If you're concerned about that cost, lambdas will not save you
    - derefr 2y ago
      Maybe we write very different kinds of scripts. For my startup, such a "script" might be: 1. An hourly itemized invoicing batch job for incremental usage-based billing, that pulls billable users from an ERP DB, grabs their usage with a complex Snowflake query (= where our structured access logs get piped), then does a CQRS reduction over the credit-and-spend state from each user's existing invoice line-items for the month, to turn the discovered usage into new line-items to insert back into the same ERP DB. 2. A Twilio customer-service IVR webhook backend, that uses local speech-processing models to recognize keywords so you don't have to punch numbers. 3. An image/SVG/video thumbnailer that gets called when a new source asset (from a web-scraping agent) is dropped into one bucket; and which writes the thumbnail for said asset out into another bucket. (For the SVG use-case especially, this requires spinning up an entire headless Chrome context per image, mostly in order to get the fonts looking right. We actually handle that part of the pipeline with serverless containers, not serverless functions, because it needs a custom runtime environment.) #1 is an example of a low-utilization "script" where we just don't want to pay for the infra required to run it when it's not running, since when it is running it "wants" better-than-cheapest resourcing to finish in a reasonable time (which, if you were deploying it to a VM, would mean paying for a more expensive VM, to sit mostly idle.) We do have a k8s cluster — and this was originally a CronJob resource on that, which made sense at first — but this is an example of a "grows over time" workload, and we're trying to avoid using k8s for those sort of workloads, because k8s expects workloads to have fixed resource quotas per pod, and can't cope well with growing workloads without a lot of node-pool finessing. #2 and #3 are high-utilization (one in CPU and RAM, one in requiring a GPU with VRAM) "scripts", where only one or two concurrent executions of these would fit on a $4/mo DO droplet; and where ten or twenty of these on a more vertically-scaled VM or machine would start to strain the network throughput of even a 2.5Gbps link. Many cheap-ish machines, each with their resourcing (including their own network bandwidth), all instantaneously reserved and released at one machine per request is a perfect match for the demand profile of these use-cases.
- MOARDONGZPLZ 2y ago
  I’m guilty of this sort of “kids these days” thinking as well. But really the AWS lambda free tier is very generous and I don’t get any sort of bill unless there’s a very large spike in usage. With DO or Hetzner I pay for the server to be up and get a monthly bill no matter what. I would even somewhat reverse your assertion: use Lambda until you have a need for a real server infra and then upgrade to things like Hetzner.

by derefr 2y ago

troupo 2y ago

> Lambda is also distributed, though — while CGI scripts just live on the webserver itself. Most of the stuff people use lambdas for they could run from the cheapest Digital Ocean droplet or cheapest Hetzner server. You need "omg distributed" perhaps after your millionth user, and even then it's highly debatable.

derefr 2y ago
You're thinking about the (rather few) advantages distribution confers for single-tenant scaling. But the distribution here is mainly for other reasons: • operational fault-tolerance: arbitrary servers in the cluster hosting the function can fail or be taken down for repair without any workloads going down. This allows the ops team that owns the cluster to freely keep the cluster's machines up to date with OS package updates, reboot them as needed, swap them out for newer-generation hardware, move them to new racks in new DC buildings, etc. (This is also something you get from a VM on most larger clouds — but only because the VM itself is a workload running on a hypervisor cluster, where the cluster's control-plane can live-migrate workloads to drain host nodes for maintenance.) • horizontally scaling the cluster as a whole, to keep up with more and more users sticking more and more workloads onto it, and more and more requests coming in for those workloads. As load increases, just add more diskless web servers. (This also means that in the rare case where your workload is too popular to fit on a cheap VPS, the ops team is doing your personal infrastructure scale-out for you for free, as part of scaling the capacity of the service as a whole.) • maintaining your multi-tenant Quality-of-Service in the face of other users who have hugely-expensive workloads. On that 90s cgi-bin webserver — or on a modern VPS — if someone else deploys some workload that pins all the CPU cores for five seconds each time anyone calls it, then that impacts how long it takes you to serve calls to your workload, because your request has landed on the same machine and is waiting in some kernel queue behind that other workload's CPU bottleneck. VMs solve this by reserving capacity per workload that's "always on" even when the workload is idle. Lambda solves this by collecting resource-accounting statistics for each function and resource-utilization metrics for each backend, and putting that info together at the load-balancer level to plan a hybrid of least-conn and resource-packing routing to backend nodes. --- But even ignoring all that, the point of the "distributed-ness" of the Lambda architecture on the low-usage end, is the "scales to zero" part that it shares with CGI. Unlike VMs, you aren't paying anything per month for a function nobody is calling. And for a workload that does get called, but only runs for N aggregate CPU-seconds each minute, you're only paying N/60 of what you'd pay for a VM (which runs every CPU-second of the every minute.) If you have one script and you have $5/mo burning a hole in your pocket, then sure, I guess you could put it on a DO droplet? (But you could just as well deploy a function that will cost you less than $5/mo.) But if you have ten scripts, where some of them might be high-utilization (you're not sure yet, depends on how many people hit the web pages that call them) while others are definitely extremely low-utilization — then your options are either "becoming a hobbyist DevOps engineer, utilization planning a packing arrangement of VMs or hardware to optimize cost, installing metrics to ensure things aren't falling over, getting emails when it does, re-planning things out, and migrating them as necessary"... or just deploying 10 functions and getting what will probably be a bill each month for <$1.
- troupo 2y ago
  You write a long wall of text that does boring to dispute what I wrote. The cheapest DO droplet is 4 dollars a month. It is more than capable of running "10 scripts", and will last your startup with three users indefinitely long. If you're concerned about that cost, lambdas will not save you
  - derefr 2y ago
    Maybe we write very different kinds of scripts. For my startup, such a "script" might be: 1. An hourly itemized invoicing batch job for incremental usage-based billing, that pulls billable users from an ERP DB, grabs their usage with a complex Snowflake query (= where our structured access logs get piped), then does a CQRS reduction over the credit-and-spend state from each user's existing invoice line-items for the month, to turn the discovered usage into new line-items to insert back into the same ERP DB. 2. A Twilio customer-service IVR webhook backend, that uses local speech-processing models to recognize keywords so you don't have to punch numbers. 3. An image/SVG/video thumbnailer that gets called when a new source asset (from a web-scraping agent) is dropped into one bucket; and which writes the thumbnail for said asset out into another bucket. (For the SVG use-case especially, this requires spinning up an entire headless Chrome context per image, mostly in order to get the fonts looking right. We actually handle that part of the pipeline with serverless containers, not serverless functions, because it needs a custom runtime environment.) #1 is an example of a low-utilization "script" where we just don't want to pay for the infra required to run it when it's not running, since when it is running it "wants" better-than-cheapest resourcing to finish in a reasonable time (which, if you were deploying it to a VM, would mean paying for a more expensive VM, to sit mostly idle.) We do have a k8s cluster — and this was originally a CronJob resource on that, which made sense at first — but this is an example of a "grows over time" workload, and we're trying to avoid using k8s for those sort of workloads, because k8s expects workloads to have fixed resource quotas per pod, and can't cope well with growing workloads without a lot of node-pool finessing. #2 and #3 are high-utilization (one in CPU and RAM, one in requiring a GPU with VRAM) "scripts", where only one or two concurrent executions of these would fit on a $4/mo DO droplet; and where ten or twenty of these on a more vertically-scaled VM or machine would start to strain the network throughput of even a 2.5Gbps link. Many cheap-ish machines, each with their resourcing (including their own network bandwidth), all instantaneously reserved and released at one machine per request is a perfect match for the demand profile of these use-cases.
MOARDONGZPLZ 2y ago
I’m guilty of this sort of “kids these days” thinking as well. But really the AWS lambda free tier is very generous and I don’t get any sort of bill unless there’s a very large spike in usage. With DO or Hetzner I pay for the server to be up and get a monthly bill no matter what. I would even somewhat reverse your assertion: use Lambda until you have a need for a real server infra and then upgrade to things like Hetzner.