3 ms·

Nice post! Would be great if you could post some total latency numbers from client to server and back, as in my experience API Gateway is extremely slow (AWS e

by nstj 10y ago

Nice post! Would be great if you could post some total latency numbers from client to server and back, as in my experience API Gateway is extremely slow (AWS employees admit as much in their forums). Another problem is that once the Lambda is 'warmed up', if you have scaling needs beyond the 1 container it's allocated then you again have a 'cold start latency' problem. Interested to see whether this had an effect on your setup also (and as a side note, I'm extremely impressed with the potential for Lambda and serverless, but it still doesn't seem ready for prime time IMO).

philliphaydon 10y ago
I'm using gateway to proxy requests to IIS, Lambdas, and S3/CloudFront. It's not slow at all for me. It's also nice having a single domain and no longer dealing with CORS.
- nstj 10y ago
  Interesting - what's your P95/P99 like? I'm seeing ~800ms with a Hello World Lambda called from API Gateway.
  - philliphaydon 10y ago
    It's a relatively new app being worked on since November, currently seeing ~400-500ms. But it's very dependent on what it's hitting. Hitting Gateway -> Cloudfront -> S3 (for gzip / caching) it's ~100-200ms. Hitting Gateway -> ELB -> EC2 -> IIS (.NET Web API) it's ~400-500ms. Hitting Gateway -> Lambda -> NodeJS it's ~700-800ms. Still need more time to in production to get more statistics.
  - zwily 10y ago
    Lambda via API gateway requires a lot of traffic to be fast. From what i've read, it's because of the SSL handshake between the Cloudfront POP (which all API Gateways use automatically) and Lambda. Only when you have a lot of traffic will enough SSL sessions be cached to get a good hit rate. (I haven't tested this myself, just what I've gleaned from the forums.)
    - nstj 10y ago
      Yeah, I'm skeptical about that though, as I still see lowest requests in the 400ms range when hitting the API Gateway with serious load (say 100k requests running at a 400 req. concurrency level). Also, even if you cache SSL sessions you still have the cold start problem when your Lambda scales beyond the container it's initially running in if you have a large number of concurrent requests. That said, I'm excited for the potential of serverless offerings - whether using containers or any other implementation mechanism. I'm building a backend for a native app right now and the initial beta version was using Lambda. The slow responses really made it tough though so a move to GKE and Kubernetes have made response times a lot lower, and it actually scales faster too.
    - deleted 10y ago
      [deleted]

by nstj 10y ago

philliphaydon 10y ago

I'm using gateway to proxy requests to IIS, Lambdas, and S3/CloudFront. It's not slow at all for me. It's also nice having a single domain and no longer dealing with CORS.

nstj 10y ago
Interesting - what's your P95/P99 like? I'm seeing ~800ms with a Hello World Lambda called from API Gateway.
- philliphaydon 10y ago
  It's a relatively new app being worked on since November, currently seeing ~400-500ms. But it's very dependent on what it's hitting. Hitting Gateway -> Cloudfront -> S3 (for gzip / caching) it's ~100-200ms. Hitting Gateway -> ELB -> EC2 -> IIS (.NET Web API) it's ~400-500ms. Hitting Gateway -> Lambda -> NodeJS it's ~700-800ms. Still need more time to in production to get more statistics.
- zwily 10y ago
  Lambda via API gateway requires a lot of traffic to be fast. From what i've read, it's because of the SSL handshake between the Cloudfront POP (which all API Gateways use automatically) and Lambda. Only when you have a lot of traffic will enough SSL sessions be cached to get a good hit rate. (I haven't tested this myself, just what I've gleaned from the forums.)
  - nstj 10y ago
    Yeah, I'm skeptical about that though, as I still see lowest requests in the 400ms range when hitting the API Gateway with serious load (say 100k requests running at a 400 req. concurrency level). Also, even if you cache SSL sessions you still have the cold start problem when your Lambda scales beyond the container it's initially running in if you have a large number of concurrent requests. That said, I'm excited for the potential of serverless offerings - whether using containers or any other implementation mechanism. I'm building a backend for a native app right now and the initial beta version was using Lambda. The slow responses really made it tough though so a move to GKE and Kubernetes have made response times a lot lower, and it actually scales faster too.
  - deleted 10y ago
    [deleted]