4 ms·
Serverless at Scale: Lessons from 200M Lambda Invocations
- Aeolun 3y agoIn other words, they had an average of 6.3 lambda invocations per second. Why make it sound so sensational? I did much more than that on a single xeon machine.
- geraldwhen 3y agoYes, but that wouldn’t require so many hoops to jump through to make work, and it would probably cost a lot less. You have to use lambda so you can overcome artificial engineering constraints so that you can write blog posts.
- BiteCode_dev 3y agoA cheap VPS could output much more, for 5 dollars a month.
- QuadrupleA 3y agoMost of our apps, on a single t3a.nano (about $3/mo) can handle about 250 req / sec in stress tests. In sluggish python no less. People don't seem to understand modern compute speeds.
- paulddraper 3y agoDoes your app do more or less work?
- citrin_ru 3y agoXeon has a long history, but even around 2005 50 rps from a single box with perl web app was not considered highload.
- tomnipotent 3y agoThey claim to need 25 medium or 6 xlarge EC2 instances to handle ~17M monthly invocations, which seems insane. I don't know everything they're doing under the hood, but I'd expect to be able to handle billions of requests with that much hardware considering the product offering.
- Nihilartikel 3y agoI'm similarly mystified. I have first hand experience with that volume of traffic.. Per day. Per host. In 2009. On python. How is everyone making everything so slow?
- kgeist 3y agoI thought lambdas are practical when they're called rarely, with a surge of traffic occasionally? So that you don't pay for the servers when they're unused, and occasionally you can withstand black friday traffic. If a lambda is called 6 times per second, I suspect the underlying VMs/containers that power lambdas are rarely shut down (I don't know how AWS works but that's how it works in another cloud provider I'm familiar with - they wait for a little for new requests before shutting down the container). So might've as well just used an always-on server. I also wonder why their calculations show that 6rps (that's what their "17 mln monthly lambda invocations" really means) would require 25 servers. We have a single mediocre VM which serves around 6 rps on average as well without issues... Although, of course, it all depends on what kind of load each request has. We don't do number crunching and most of the time is spent in the database.
- civilitty 3y agoIf they’re called rarely, they have poor latency due to cold start times. IMO lambdas are most practical when someone else in the org is responsible for spinning up VMs and your boss is in a pissing match with their boss, preventing you from getting any new infrastructure to get work done. The technical merits rarely have anything to do with it.
- steveBK123 3y ago"it costs 2x as much but doesn't require me to navigate my messed up IT org" is an underappreciated motivation for a lot of architectures you see people pump. Your IT org political problems may not match, and therefore their architecture may not either. Also there's the question of what problem you are trying to scale at what scale, with what resiliency/uptime requirements, etc...
- jamesfinlayson 3y agoAgreed - I worked at a company that couldn't get some expensive security tool approved so they just bought it off AWS Marketplace. The AWS bill was high enough that the increase wasn't questioned.
- tuetuopay 3y agoThey talk about lambdas that exceed the max runtime of 15 minutes. For a single invocation. They go to great lengths talking about cron jobs, background jobs triggered by events, etc. I highly doubt the bulk of the lambdas they talk about are simple API calls. Otherwise yes, 6-7rps is peanuts, if we’re talking about API calls. And since the very first point of the article is them highlighting what goes to lambda and what does not (public api calls go to a dedicated box), I think it’s safe to say those 6 invocations/sec are definitely not API calls. TL;DR your Xeon box is their always-on api box.
- reese_john 3y agoFor our region that limit is set to 1000. That might sound like a lot when you start, but you quickly realise it’s easy to reach once you have enough Lambdas and you scale up. We found ourselves hitting that limit a lot once our traffic and therefore our demands from our system started scaling. You can file a support ticket to have that limit raised. https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html https://docs.aws.amazon.com/servicequotas/latest/userguide/r...
- Jamie9912 3y agoThat's really annoying though.. Why should I have to go out of my way to increase capacity given that i'm paying for it anyway?
- ebbp 3y agoGenerally these limits exist so customers don’t accidentally spend more than they intend to — e.g. implementing a sort of infinite loop where Lambdas call each other constantly. Sounds implausible but I’ve seen that more than once!
- maccard 3y ago> Sounds implausible but I’ve seen that more than once! The textbook example of this going wrong is a lambda that is invoked on uploading to S3 that writes the result to S3. There's even an AWS article on it - [0] [0] https://aws.amazon.com/blogs/compute/avoiding-recursive-invocation-with-amazon-s3-and-aws-lambda/ https://aws.amazon.com/blogs/compute/avoiding-recursive-invo...
- easton 3y agoWe actually got an email from AWS recently at work that said “hey! Your lambda writes to a queue that invokes the same lambda, that seems wack”. We need it that way, but it’s enough of a problem that they built a way to detect it automatically.
- 3y ago
- dig1 3y ago> Serverless architecture promises flexibility, infinite scalability, fast setups, cost efficiency, and abstracting infrastructure, allowing us to focus on the code. The only thing I know that serverless architecture promises are big bills and a steady income for a cloud provider. I'd be happy to see a serverless setup that won't be blown away with a (way cheaper) small/medium-sized VM.
- insanitybit 3y agoYou have to manage a VM. For example, ensuring that the VM has an up to date OS. If you don't care about that, ok, but that's something that Lambda offers. Ephemerality is a plus as well. Just from a security standpoint, having an ephemeral system means persistence is not possible.
- QuadrupleA 3y agosudo yum update, once a week
- insanitybit 3y agoIf it's one VM for a personal website that'll be fine. Good luck explaining that to a SOC2 auditor, or managing that across a fleet.
- steveBK123 3y agoNooo I can't run that, I need to pay for Bezos 3rd yacht!
- flavius29663 3y agothat is not nearly enough. You need to make sure the system still works after the update, so you need to carefully control all the versions that go in, test them in lower environments. Also, for any serious application, you will need to do this multiple times, for hundreds or thousands of machines even for small companies. I am on your side, actually, I think managing machines is better than serverless, but it's not that easy.
- 3y ago
- jeswin 3y agoGiven that AWS still has no billing caps (despite it being one of the most requested features), you're exposing yourself to uncapped downside. In addition to lambdas being a poor architectural choice in most cases, that is.
- insanitybit 3y agoAFAIK you can pretty easily cap the number of concurrent lambda executions. Of all of AWS's services, Lambda is probably the easiest one to configure limits on.
- dmattia 3y agoFor lambdas in particular, you can set reserved concurrency, which is the most of a particular lambda that can run concurrently at any point in time: https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html https://docs.aws.amazon.com/lambda/latest/dg/configuration-c....
- vcryan 3y agoThe conclusions are what is already widely understood about Lambdas. They could have just researched the topic up front and chose a better architecture.
- ecshafer 3y agoThe biggest lesson i learned when i was in an org thay started using serverless heavily (because its the future!) is that its an unmaintanable mess. You end with code spaghetti, but now its split across 100 repos that might be hard to find, and designed purely on architecture diagrams. From what i can see its basically recreating mainframe batch processing but in the cloud. X happens which triggers Y Job which triggers Z job and so on.
- politelemon 3y agoThat sounds like micro services in general rather than lambda the Aws service selectively. I've seen the same unsustainable mess with k8s crazy teams. The lesson I've learned instead is to start boring and traditional, then use serverless tech when you hit a problem the current setup cannot solve
- ecshafer 3y agoI agree that its like microservices, but the problem is turned up to 11 with serverless. 1 microservice now becomes 10 lambdas. The issue is fundamentally one of discovery, and as teams churn out more and more functions, that arent all in the same repo, youre bound to lose traxk of whats happening.
- abrookewood 3y agoAgreed. From what I have seen, lambda functions are typically even smaller than micro-services & you end up with even more of them.
- losteric 3y agoServerless has it's faults, but spaghetti code spread across 100 repos is definitely a "user error"...
- ecshafer 3y agoHow do you solve the discoverability issue when theres 1000 serverless functions written by 10 different teams then? Serverless worsens the issue of having knowledge of the entire system i think, and i dont think this is even solvable.
- politelemon 3y agoI think this does a decent job of talking about the tech without hyping up or shitting on it, just speaking matter-of-factly. They seem to have found a decent fit for the tech, and are able to recognize where it works and doesn't work, and are still able to 'diversify' to other things. Good post thanks for sharing.
- robbles 3y ago> 10GB is hardly enough. Once you import Pandas you’re on the limit. You can forget Pandas and scipy at the same Lambda. This sounds way off to me. 10 GB to install a Python library?
- sgarland 3y agoI also thought this seemed extremely odd. Here is an unoptimized example, built on an M1 Mac: $ cat <<EOF > Dockerfile FROM python:3.12-slim-bookworm RUN apt-get update && apt-get install -y python3-pip RUN pip install pandas LABEL "name"="python-pandas" ENTRYPOINT ["python3"] EOF $ docker image ls -f 'label'='name'='python-pandas' REPOSITORY TAG IMAGE ID CREATED SIZE sgarland/python-pandas latest 89e31f6eb83d 9 minutes ago 764MB A more optimized version: $ cat <<EOF > Dockerfile FROM python:3.12-slim-bookworm RUN apt-get update && \ apt-get install -y --no-install-recommends python3-pip && \ pip install pandas && \ apt-get purge -y --autoremove python3-pip && \ rm -rf /var/lib/apt/lists/* LABEL "name"="python-pandas" ENTRYPOINT ["python3"] $ docker image ls -f 'label'='name'='python-pandas' REPOSITORY TAG IMAGE ID CREATED SIZE sgarland/python-pandas smaller 102308842b88 4 seconds ago 342MB sgarland/python-pandas latest 89e31f6eb83d 27 minutes ago 764MB Even adding in scipy didn't crack 500 MB: $ docker image ls -f 'label'='name'='python-pandas' REPOSITORY TAG IMAGE ID CREATED SIZE sgarland/python-pandas scipy 808535284f03 3 minutes ago 497MB sgarland/python-pandas smaller 102308842b88 9 minutes ago 342MB sgarland/python-pandas latest 89e31f6eb83d 36 minutes ago 764MB I'm not sure how they managed 10 GB. Here's the non-slim version, with no optimizations (this is much larger because `python3-pip` has the system default Python interpreter as as dependency, so this installs Python3.11 into the image): $ cat <<EOF > Dockerfile FROM python:3.12-bookworm RUN apt-get update RUN apt-get install -y python3-pip RUN pip install pandas scipy LABEL "name"="python-pandas" ENTRYPOINT ["python3"] EOF $ docker image ls -f 'label'='name'='python-pandas' REPOSITORY TAG IMAGE ID CREATED SIZE sgarland/python-pandas bigger f8f98e9a241c 8 seconds ago 1.44GB sgarland/python-pandas scipy 808535284f03 3 minutes ago 497MB sgarland/python-pandas smaller 102308842b88 9 minutes ago 342MB sgarland/python-pandas latest 89e31f6eb83d 36 minutes ago 764MB
- acd 3y agoThe name serverless is missleading. Of course there is functions written in programming languages that runs on a real physicall servers in data centers. This should be called Fuction as a service. Acronym Faas.
- willio58 3y agoThis _is_ called FaaS generally. Serverless is kinda confusing though. Like whenever I say I work with serverless functions I always have to explain there are unmanaged servers involved
- hobobaggins 3y ago> serverless.. as long as we have enough money to throw on it This one speaks truth.
- deleted 3y ago[deleted]