5 ms·

Running a site the size of the BBC on Lambda is nothing short of an exuberant waste of a government-subsidized budget, it's absolutely crazy. Lambda VM time ha

Running a site the size of the BBC on Lambda is nothing short of an exuberant waste of a government-subsidized budget, it's absolutely crazy. Lambda VM time has a massive markup compared to regular compute. It only makes sense where usage does not exceed some threshold the BBC absolutely certainly do. There are plenty of alternative options, even on AWS, that don't suffer from such huge markup without requiring any additional ops input. The thing that runs in Lambda is practically a container image already. Does it really cost tangible budget to have CI build a full image rather than a ZIP file that already contains a few million lines of third party JS/Python deps? IMHO this is the epitome of serverless gone wrong.

jevgeni 6y ago
Completely agree. What also irks me is the corpspeak sleigh of hand, where the problem of maintaining your infrastructure is “solved” by using AWS Lambda. It’s not solved, you’re just paying some AWS contractors to do it for you and you assume it’ll be alright.
- remus 6y ago
  > It’s not solved, you’re just paying some AWS contractors to do it for you and you assume it’ll be alright. In as much as maintaining infrastructure can ever be 'solved', doesn't paying someone else who does a good job of it to provide you with the infra count as solving the problem? Otherwise you'd be down the mines picking ore out of the ground so you can build your chips right, rather than relying on intel/AMD to do it for you and assuming it'll be alright.
  - jevgeni 6y ago
    The problem arises when the infra providers’ setup stops being aligned with yours. Which, given a small organization and a long enough period of time, is guaranteed to happen. The question is then: is it just cheaper in the long run to deal with the hassle of your own infra (we are still talking about the cloud, btw, the thing that supposedly already solved it) or would it be ok to follow the practices and changes in the providers offering?
strainyy 6y ago
There wasn't any mention of cost, so you're making a big assumption here. I imagine the caching layer would significantly reduce the Lambda calls.
oppopower 6y ago
AWS Lambda $0.20 per 1M requests $0.0000166667 for every GB-second SLA 99.95% Lets assume - 2000 calls/sec - each call is 1 sec duration - 0.128 GB/sec/call - db, storage iops will be the same if deployed as K8s - 5x9s SLA (imples a three region deployment) requests per year = 2000 * 3600 * 24 * 365 = 63072000000 request costs = 0.2 * 63072000000 / 1000000 = 12,614 USD GB/sec per year = 0.128 * 2000 * 3600 * 24 * 365 = 8073216000 GB/sec costs = 8073216000 * 0.0000166667 = 134,553.87 Total cost = 12614 + 134553.87 = 147,167.87 USD The equivalent K8s would be - three clusters - 2000 cores (more likely 10% more = 2200) - 256GB memory Three clusters will require 3000 cores to cater for region loss - 3000 cores on 32 core machines => 94 machines - round up to 99 machines to give vm level redundancy => 33 machines per cluster Azure D32a_v4 (32 core, 128 GiB, 800 GiB storage) = $1.84/hour PAYG; $0.5704/hour Spot DS32a_v4 at spot pricing = 99 * 0.5704 * 24 * 365 = 494673.696 USD Plus FTE support (e.g. n FTEs @ 100k USD) With 2 FTEs total is 694,673 USD Summary: AWS Lambda is 4.7x cheaper than a Kubernetes solution
- smarx007 6y ago
  What you are missing are the Amazon API Gateway costs. I really liked my own calculations on Lambda costs similar to yours until I figured I'd need to use API Gateway too. Edit: another thing is the amount of RAM used by functions. The CPU speed you get is proportional to RAM so if your code fits in the RAM but has poor performance, doubling the RAM is what you have to do. Another hidden cost.
  - oppopower 6y ago
    Thanks, peer review is what I'm after The GB/sec calc is there My assumption was that even with k8s there'll be an API gateway, GTM etc Comparing AWS Lamda = 147,167.87 USD to K8s with autoscaling: DS32a_v4 at spot pricing = 66 * 0.5704 * 24 * 365 = 329782.46 USD With 2 FTEs @ 100k total is 529782.46 USD Caveat application can tolerate autoscaling delays Summary: AWS Lambda is 3.5x cheaper than a Kubernetes solution
    - BackBlast 6y ago
      The devil is in the details of how this is implemented. You're ignoring architecture and simply saying that equivalent raw capacity is required. If a request or process comes in, and it hits 10 services to fulfill. The entry service will be open for the entirety of the request, while billing a full 128MB even while waiting for other services and regardless of how much memory is actually used. This is also true of every nested service, which multiplies the required resources. Through the chain there will often only be one active lambda on the request though many are billing. A monolithic design will only use the actual resources dedicated towards serving the request. To put it another way, you change your memory and compute granularity from per-container scope to per-request scope. The finer or microized your services the higher minimums you have on serving a request. In my experience with such implementations this ratio is somewhere in the range of 2x-10x.

jevgeni 6y ago

Completely agree. What also irks me is the corpspeak sleigh of hand, where the problem of maintaining your infrastructure is “solved” by using AWS Lambda. It’s not solved, you’re just paying some AWS contractors to do it for you and you assume it’ll be alright.

remus 6y ago
> It’s not solved, you’re just paying some AWS contractors to do it for you and you assume it’ll be alright. In as much as maintaining infrastructure can ever be 'solved', doesn't paying someone else who does a good job of it to provide you with the infra count as solving the problem? Otherwise you'd be down the mines picking ore out of the ground so you can build your chips right, rather than relying on intel/AMD to do it for you and assuming it'll be alright.
- jevgeni 6y ago
  The problem arises when the infra providers’ setup stops being aligned with yours. Which, given a small organization and a long enough period of time, is guaranteed to happen. The question is then: is it just cheaper in the long run to deal with the hassle of your own infra (we are still talking about the cloud, btw, the thing that supposedly already solved it) or would it be ok to follow the practices and changes in the providers offering?

strainyy 6y ago

There wasn't any mention of cost, so you're making a big assumption here. I imagine the caching layer would significantly reduce the Lambda calls.

oppopower 6y ago

AWS Lambda $0.20 per 1M requests $0.0000166667 for every GB-second SLA 99.95% Lets assume - 2000 calls/sec - each call is 1 sec duration - 0.128 GB/sec/call - db, storage iops will be the same if deployed as K8s - 5x9s SLA (imples a three region deployment) requests per year = 2000 * 3600 * 24 * 365 = 63072000000 request costs = 0.2 * 63072000000 / 1000000 = 12,614 USD GB/sec per year = 0.128 * 2000 * 3600 * 24 * 365 = 8073216000 GB/sec costs = 8073216000 * 0.0000166667 = 134,553.87 Total cost = 12614 + 134553.87 = 147,167.87 USD The equivalent K8s would be - three clusters - 2000 cores (more likely 10% more = 2200) - 256GB memory Three clusters will require 3000 cores to cater for region loss - 3000 cores on 32 core machines => 94 machines - round up to 99 machines to give vm level redundancy => 33 machines per cluster Azure D32a_v4 (32 core, 128 GiB, 800 GiB storage) = $1.84/hour PAYG; $0.5704/hour Spot DS32a_v4 at spot pricing = 99 * 0.5704 * 24 * 365 = 494673.696 USD Plus FTE support (e.g. n FTEs @ 100k USD) With 2 FTEs total is 694,673 USD Summary: AWS Lambda is 4.7x cheaper than a Kubernetes solution

smarx007 6y ago
What you are missing are the Amazon API Gateway costs. I really liked my own calculations on Lambda costs similar to yours until I figured I'd need to use API Gateway too. Edit: another thing is the amount of RAM used by functions. The CPU speed you get is proportional to RAM so if your code fits in the RAM but has poor performance, doubling the RAM is what you have to do. Another hidden cost.
- oppopower 6y ago
  Thanks, peer review is what I'm after The GB/sec calc is there My assumption was that even with k8s there'll be an API gateway, GTM etc Comparing AWS Lamda = 147,167.87 USD to K8s with autoscaling: DS32a_v4 at spot pricing = 66 * 0.5704 * 24 * 365 = 329782.46 USD With 2 FTEs @ 100k total is 529782.46 USD Caveat application can tolerate autoscaling delays Summary: AWS Lambda is 3.5x cheaper than a Kubernetes solution
  - BackBlast 6y ago
    The devil is in the details of how this is implemented. You're ignoring architecture and simply saying that equivalent raw capacity is required. If a request or process comes in, and it hits 10 services to fulfill. The entry service will be open for the entirety of the request, while billing a full 128MB even while waiting for other services and regardless of how much memory is actually used. This is also true of every nested service, which multiplies the required resources. Through the chain there will often only be one active lambda on the request though many are billing. A monolithic design will only use the actual resources dedicated towards serving the request. To put it another way, you change your memory and compute granularity from per-container scope to per-request scope. The finer or microized your services the higher minimums you have on serving a request. In my experience with such implementations this ratio is somewhere in the range of 2x-10x.