4 ms·

Using Serverless for unpredictable traffic to for TV fundraiser event

76 points by skifunkster 6y ago

oblio 6y ago
I swear that half of HN articles can be best described as "Get rich quick" schemes. Ok, maybe with a few extra steps :-) Such a clickbait title.
- bognition 6y ago
  And half the comments on are from people who didn’t read the article. This isn’t about getting rich quick but setting up a system to respond to high load that comes in bursts. The authors setup a system to handle bursts of traffic driven by a popular tv show. Normally the traffic is zero so in this case lambda makes a ton of sense.
  - oblio 6y ago
    As a matter of principle I rarely read articles with clickbait titles. As the name of the category implies, the signal to noise ratio is awful, 99% of them are a waste of time. I'd rather "read" it by reading comments on articles here on HN and only then deciding to read the main article if it proves worthy of my time :-)
    - bognition 6y ago
      That’s totally fair I often do that as well. :-) However you did assert that this article was about a get rich quick scheme which is taking things a bit further than commenting on a click vanity title. Viewers likely saw your comments and used it to form their opinion of the content.
      - oblio 6y ago
        Well, looking at the other comments, it seems I was right. Many comments are about how shallow the article is from a technical point of view.
        oblio 6y ago
        Ok, I take that back. It was a submarine article as you can see at the end but the call to action is for charity. So clickbait for a worthy cause.
hu3 6y ago
The TLDR is: By using Cloudflare Workers and Stripe for payment they could deal with a very spiky traffic during the show without provisioning a ton of machines. Sadly the article doesn't go into technical details which is what I'm interested in. Therefore it can be resumed down to that one phrase.
- stefan_ 6y ago
  Until they couldn't, because their Stripe account was being rate-limited, for presumably entirely non-technical reasons
hardwaresofton 6y ago
Sure would have loved to see a breakdown of the breadth of functions and the nature of each function (did you simply just serve as a stripe proxy? did you squirrel away some data into a separate database? etc). Hard to know just how much serverless really helped here as opposed to other solutions. Also, the cost of the serverless execution would be great to know (don't forget those API Gateway fees or other such "hidden" costs). The tech empower benchmarks[0] w/ a single DB request and NodeJS show some pretty good results (10k-100k) and since the max was 2.5k. I think Node isn't a particularly performance conscious choice, though it does async well and that should be pretty common knowledge these days (that doing async well is a good idea for high load). Also normally you'd use Cloudflare workers to serve some data very close to customers -- was that the use case here? or was it more just that the workers were a good fit for deployment ease? There's also the question of essentially subsidies -- I hear that cloudflare worker ingress (egress?) traffic is free right now? [0]: https://www.techempower.com/benchmarks/#section=data-r19&hw=ph&test=db https://www.techempower.com/benchmarks/#section=data-r19&hw=...
- skifunkster 6y ago
  Yeah, we literally used it as a stripe proxy and to serve frontend. All data was served in stripe and then queried using Stripe Sigma, their flavour of Presto. Can tell you the total cost from RTE does Comic Relief, which was the same size, was about 16 euros to host the whole thing. This would be about the same, however is on RTE's cloudflare account, so not easy to get breakdown. Workers were just a good fit not to have to host anything, due to the unpredictable number of donations that we would be getting through. Deployment and development is also really easy.
  - hardwaresofton 6y ago
    Thanks for sharing this -- that's an impressive cost amount for that service, in the range of 20 euros for the event with that amount of queries is pretty impressive, with little to no maintenance. The cost makes it sound like a full on win for serverless. Anything interesting about the deployment method, choice of language, containerization/etc that made it particularly easy? There are tools in this domain (vercel springs to mind) that have adopted the static-site-over-serverless-something) model, but it sounds like you all had an easy time with just loading some workers up, did you do anything fancy to facilitate the frontend serving? I assume with the stripe proxy bit was just using Stripe's various (great) APIs for customer creation, one time charges (I guess?). Curious about the technicals of the setup, you've baited us (HN) in at this point, give out those gory details!
mwcampbell 6y ago
Since the article buried the (technical) lead a bit, I'll just mention that they used Cloudflare Workers. I imagine Cloudflare Workers is better than AWS Lambda (or Lambda at Edge) at handling sharp traffic spikes, since the use of V8 isolates yields much lower cold start times than AWS's micro-VMs. Does anyone have real experience with both models?
- ttsda 6y ago
  It's like night and day in my (small) experience. Cold start is VERY fast.
- sudhirj 6y ago
  So Lambda & (L@Edge) is a general purpose computing system. They run a normal process in a programming language of your choice, then feed it event data, take the response out and continue the pipeline. They might keep the process running for reuse sometimes. If your process is being reused, it’s “warm” and works as fast your code. If it’s “cold”, your process start time, Lambda finding an execution slot, downloading your code there etc all add up. Good news is that if you’re doing multi core stuff or specialized code it’ll work fine (now including special Intel AV-something instructions). Cloudflare workers took a different approach - they run V8 node-ish processes on all their edges, and you submit your code as JS or WASM that fits their interface (which is based on the service worker standard). They then execute it as a V8 isolate, which is blazing fast because the V8 process is already up and running, it’s just running JS/WASM code, which it knows how to do pretty well. Even the cold start is pretty small, because the JS/WASM tends to be pretty small and Cloudflare knows how to cache stuff. They’re both very different systems, but both have their uses.
  - skifunkster 6y ago
    Have run the same size campaign on lambda and cloudflare now, so can probably speak from experience. Cloudflare definitely has a place, but is definitely nowhere near as feature rich as the AWS ecosystem. For this use case it was perfect. Lack of warm up and responsiveness on cloudflare is a strong draw. Here is my previous article on our AWS donation system. https://read.iopipe.com/the-journey-to-90-serverless-at-comic-relief-a13b9245fa3c https://read.iopipe.com/the-journey-to-90-serverless-at-comi...
pjc50 6y ago
Terrible title for a fun little case study; the RTE Late Late Toy Show is a bit of a cultural institution, and setting up a scalable donation infrastructure for a "big bang" is a challenge. Of course the company "We Are Serverless" is always going to recommend serverless, but in this case it's the optimal use case of huge unpredictable scaling over a short period. Bit short on detail, mostly advertorial.
- skifunkster 6y ago
  Spikey, unpredictable traffic, serverless is the way. Sorry about the title.
- Hamuko 6y ago
  >Terrible title for a fun little case study I figured it was one of those AWS invoice shock stories from "serverless" and "millions of Euros".
- imtringued 6y ago
  When I saw the title I honestly was thinking that someone managed to convert millions of viewers into a million Euro AWS bill...
ed25519FUUU 6y ago
Having some experience running serverless applications in a large, distributed environment, I can say that this could have easily gone very wrong. I've seen first hand how a serverless application, "scaling" to meet a huge demand, will stampede the weakest downstream service and DDoS it. In this case they're lucky that their payment processor (Stripe) didn't start throttling them. You may have spoken with Stripe beforehand about your expected load and get some sort of agreement / whitelist for your API token. That would be very wise.
- skifunkster 6y ago
  In this scenario we did have an account manager at Stripe, however fortunately and unfortunately the show brought in somewhere in the region of 7X the initial target. Servereless stampeding a payment processor like Stripe as opposed to something like RDS are two different kettles of fish, as am sure you can imagine, stripe have very high capacity and there is 1:1 relationship to the user unlike DB writes, where there may be many per transaction / action. I have don't it both ways in the past, queueing payments and processing later. Problem is with doing that you cannot immediately return card errors, which has a large effect on conversion. With something like Amazon it is fine to do that, as users inevitably come back to buy the products they want, for donations not so much.

76 points by skifunkster 6y ago

I swear that half of HN articles can be best described as "Get rich quick" schemes. Ok, maybe with a few extra steps :-) Such a clickbait title.

bognition 6y ago
And half the comments on are from people who didn’t read the article. This isn’t about getting rich quick but setting up a system to respond to high load that comes in bursts. The authors setup a system to handle bursts of traffic driven by a popular tv show. Normally the traffic is zero so in this case lambda makes a ton of sense.
- oblio 6y ago
  As a matter of principle I rarely read articles with clickbait titles. As the name of the category implies, the signal to noise ratio is awful, 99% of them are a waste of time. I'd rather "read" it by reading comments on articles here on HN and only then deciding to read the main article if it proves worthy of my time :-)
  - bognition 6y ago
    That’s totally fair I often do that as well. :-) However you did assert that this article was about a get rich quick scheme which is taking things a bit further than commenting on a click vanity title. Viewers likely saw your comments and used it to form their opinion of the content.
    - oblio 6y ago
      Well, looking at the other comments, it seems I was right. Many comments are about how shallow the article is from a technical point of view.
      - oblio 6y ago
        Ok, I take that back. It was a submarine article as you can see at the end but the call to action is for charity. So clickbait for a worthy cause.

hu3 6y ago

The TLDR is: By using Cloudflare Workers and Stripe for payment they could deal with a very spiky traffic during the show without provisioning a ton of machines. Sadly the article doesn't go into technical details which is what I'm interested in. Therefore it can be resumed down to that one phrase.

stefan_ 6y ago
Until they couldn't, because their Stripe account was being rate-limited, for presumably entirely non-technical reasons

hardwaresofton 6y ago

Sure would have loved to see a breakdown of the breadth of functions and the nature of each function (did you simply just serve as a stripe proxy? did you squirrel away some data into a separate database? etc). Hard to know just how much serverless really helped here as opposed to other solutions. Also, the cost of the serverless execution would be great to know (don't forget those API Gateway fees or other such "hidden" costs). The tech empower benchmarks[0] w/ a single DB request and NodeJS show some pretty good results (10k-100k) and since the max was 2.5k. I think Node isn't a particularly performance conscious choice, though it does async well and that should be pretty common knowledge these days (that doing async well is a good idea for high load). Also normally you'd use Cloudflare workers to serve some data very close to customers -- was that the use case here? or was it more just that the workers were a good fit for deployment ease? There's also the question of essentially subsidies -- I hear that cloudflare worker ingress (egress?) traffic is free right now? [0]: https://www.techempower.com/benchmarks/#section=data-r19&hw=ph&test=db https://www.techempower.com/benchmarks/#section=data-r19&hw=...

skifunkster 6y ago
Yeah, we literally used it as a stripe proxy and to serve frontend. All data was served in stripe and then queried using Stripe Sigma, their flavour of Presto. Can tell you the total cost from RTE does Comic Relief, which was the same size, was about 16 euros to host the whole thing. This would be about the same, however is on RTE's cloudflare account, so not easy to get breakdown. Workers were just a good fit not to have to host anything, due to the unpredictable number of donations that we would be getting through. Deployment and development is also really easy.
- hardwaresofton 6y ago
  Thanks for sharing this -- that's an impressive cost amount for that service, in the range of 20 euros for the event with that amount of queries is pretty impressive, with little to no maintenance. The cost makes it sound like a full on win for serverless. Anything interesting about the deployment method, choice of language, containerization/etc that made it particularly easy? There are tools in this domain (vercel springs to mind) that have adopted the static-site-over-serverless-something) model, but it sounds like you all had an easy time with just loading some workers up, did you do anything fancy to facilitate the frontend serving? I assume with the stripe proxy bit was just using Stripe's various (great) APIs for customer creation, one time charges (I guess?). Curious about the technicals of the setup, you've baited us (HN) in at this point, give out those gory details!

mwcampbell 6y ago

Since the article buried the (technical) lead a bit, I'll just mention that they used Cloudflare Workers. I imagine Cloudflare Workers is better than AWS Lambda (or Lambda at Edge) at handling sharp traffic spikes, since the use of V8 isolates yields much lower cold start times than AWS's micro-VMs. Does anyone have real experience with both models?

ttsda 6y ago
It's like night and day in my (small) experience. Cold start is VERY fast.
sudhirj 6y ago
So Lambda & (L@Edge) is a general purpose computing system. They run a normal process in a programming language of your choice, then feed it event data, take the response out and continue the pipeline. They might keep the process running for reuse sometimes. If your process is being reused, it’s “warm” and works as fast your code. If it’s “cold”, your process start time, Lambda finding an execution slot, downloading your code there etc all add up. Good news is that if you’re doing multi core stuff or specialized code it’ll work fine (now including special Intel AV-something instructions). Cloudflare workers took a different approach - they run V8 node-ish processes on all their edges, and you submit your code as JS or WASM that fits their interface (which is based on the service worker standard). They then execute it as a V8 isolate, which is blazing fast because the V8 process is already up and running, it’s just running JS/WASM code, which it knows how to do pretty well. Even the cold start is pretty small, because the JS/WASM tends to be pretty small and Cloudflare knows how to cache stuff. They’re both very different systems, but both have their uses.
- skifunkster 6y ago
  Have run the same size campaign on lambda and cloudflare now, so can probably speak from experience. Cloudflare definitely has a place, but is definitely nowhere near as feature rich as the AWS ecosystem. For this use case it was perfect. Lack of warm up and responsiveness on cloudflare is a strong draw. Here is my previous article on our AWS donation system. https://read.iopipe.com/the-journey-to-90-serverless-at-comic-relief-a13b9245fa3c https://read.iopipe.com/the-journey-to-90-serverless-at-comi...

pjc50 6y ago

Terrible title for a fun little case study; the RTE Late Late Toy Show is a bit of a cultural institution, and setting up a scalable donation infrastructure for a "big bang" is a challenge. Of course the company "We Are Serverless" is always going to recommend serverless, but in this case it's the optimal use case of huge unpredictable scaling over a short period. Bit short on detail, mostly advertorial.

skifunkster 6y ago
Spikey, unpredictable traffic, serverless is the way. Sorry about the title.
Hamuko 6y ago
>Terrible title for a fun little case study I figured it was one of those AWS invoice shock stories from "serverless" and "millions of Euros".
imtringued 6y ago
When I saw the title I honestly was thinking that someone managed to convert millions of viewers into a million Euro AWS bill...

ed25519FUUU 6y ago

Having some experience running serverless applications in a large, distributed environment, I can say that this could have easily gone very wrong. I've seen first hand how a serverless application, "scaling" to meet a huge demand, will stampede the weakest downstream service and DDoS it. In this case they're lucky that their payment processor (Stripe) didn't start throttling them. You may have spoken with Stripe beforehand about your expected load and get some sort of agreement / whitelist for your API token. That would be very wise.

skifunkster 6y ago
In this scenario we did have an account manager at Stripe, however fortunately and unfortunately the show brought in somewhere in the region of 7X the initial target. Servereless stampeding a payment processor like Stripe as opposed to something like RDS are two different kettles of fish, as am sure you can imagine, stripe have very high capacity and there is 1:1 relationship to the user unlike DB writes, where there may be many per transaction / action. I have don't it both ways in the past, queueing payments and processing later. Problem is with doing that you cannot immediately return card errors, which has a large effect on conversion. With something like Amazon it is fine to do that, as users inevitably come back to buy the products they want, for donations not so much.