4 ms·

It doesn't surprise me that people want to do that, but the design intention of Lambda is to invoke it asynchronously via writes to dynamodb, s3, sqs, IoT etc.

It doesn't surprise me that people want to do that, but the design intention of Lambda is to invoke it asynchronously via writes to dynamodb, s3, sqs, IoT etc. I wouldn't make any assumption that someone building tools for Lambda is going to prioritise integration with the API Gateway and its somewhat alien synchronous behaviour. The best fitted Lambda apps are going to have an event-sourced CQRS style, I think. AWS IoT is kinda the last piece of that puzzle to fall into place.

pmontra 10y ago
But there are already tools like https://github.com/Miserlou/django-zappa https://github.com/Miserlou/django-zappa https://aws.amazon.com/blogs/compute/building-enterprise-level-web-applications-on-aws-lambda-with-deep/ https://aws.amazon.com/blogs/compute/building-enterprise-lev... plus something else that was featured on HN recently and I can't find or remember. Amazon itself has a tutorial about the integration of the API gateway with Lambda at http://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started.html http://docs.aws.amazon.com/apigateway/latest/developerguide/... I saw an evangelist demoing it at an Amazon Summit a month ago. It's so time consuming and error prone that you don't want to do it manually more than once (not that he made any mistake). So I won't say it's a corner case. Maybe it's too early to know what Lambda will grow into.
- homero 10y ago
  What's the point of API Gateway? Looks like a proxy where you recreate your api and does nothing more.
  - pmontra 10y ago
    I think I leant about it when I learnt about Lambda. In this context it's a convenient way to give a URL to a function, pass parameters to it and get back the result. You could build HN with the API gw, one Lambda function per each request and a db.
    - inopinatus 10y ago
      That's exactly what I'm suggesting is an anti-pattern. Yes you can do that. Yes it's hardly surprising that AWS make it possible, since synchronous RESTful CRUD is such a popular integration approach and often the first one we learn. But I'd advise you to design your LambdaNews service using an asynchronous event-driven model instead. You'll find it reduces complexity and makes for a snappier, more reliable client experience (e.g. no 429 throttling responses to handle). You'll also find that the supporting AWS services like Kinesis, Cognito, IoT, SNS, SQS, DynamoDB, S3 etc are much more in tune with that pattern. The fact is that async platforms scale much better than anything that has to block and poll. It should hardly be surprising that at AWS's massive scale, it becomes a preferred service design pattern. Pub/sub integration people have known this for decades, it is not news. obdisclosure: I am ex-AWS but all the above is derived from public statements and documentation.
      - pmontra 10y ago
        I'm sincerely interested. The POST I use to submit this comment can be asynchronous, no problem (just do ajax). Same for votes and flags. The GET I use to get the page could be a real HTTP GET or diffs coming over a websocket (but I need a page to start with). I'm not sure how to implement that in the AWS ecosystem because I don't have much experience on it, but is it this what you mean with "asynchronous event-driven model"?
  - beat 10y ago
    API Gateway does a number of things. It distributes endpoints via cloudfront, which is pretty awesome. It serves as a unified front end for a collection of back end microservices. It handles client cert authentication (important to us, but not to most people) and api keys for you, and a bunch of other neat stuff as well. And it offers a test harness where you can mock up data easily. In theory, you could do the same thing with nginx and load balancers and multiple ec2 instances, but it'd be a lot more work. And I think the real magic is with integration with Lambda. That's a game-changer.
    - homero 10y ago
      Offloading some calls into lambda does look cool
Touche 10y ago
> but the design intention of Lambda is to invoke it asynchronously via writes to dynamodb, s3, sqs, IoT etc. I wish they (they being AWS or Apex or someone in this space) said this explicitly. Every time I've read about Lambda I've thought it's interesting but am not sure how/where to invoke it in production.
- dminor 10y ago
  Here's their documentation on the various ways to invoke a lambda function (including the API Gateway, contrary to what was stated above): http://docs.aws.amazon.com/lambda/latest/dg/intro-core-components.html#intro-core-components-event-sources http://docs.aws.amazon.com/lambda/latest/dg/intro-core-compo...
southpolesteve 10y ago
There is nothing wrong with using lambda to synchronously respond to user facing API requests. It actually works really well and I would highly recommend it. Were doing it in production at ~15 million calls/day.
- Touche 10y ago
  Isn't it expensive though?
  - southpolesteve 10y ago
    That is the best part! It is cheaper! We had a node service that went from $2500/mo to $400/mo just by switching form Ec2 to Lambda. That is the most apples to apples comparison I can give you. Other services were split up or rewritten, but honestly my gut feeling tells me they are costing 10x-50x less. IMO Autoscaling ec2 instances to daily traffic is not a trivial problem and you'll never get the granularity you can with Lambda
    - leoalves 10y ago
      You are also saving in not having someone maintaining and making sure the server is always online. Even if it was the same price, not having to worry about the server is a big plus
  - skrowl 10y ago
    Cheaper than standing up enough EC2 instances to do the same volume. That's the idea at least.
  - zbjornson 10y ago
    Strictly looking at server uptime, yes, it's nearly 5x the cost of comparable t2 on-demand EC2 instances. That can be mitigated if you have spiky traffic where you'd need to turn on several EC2 instances for less than an hour but are stuck paying for the full hour, or if you can scale to zero for extended periods of time. You don't have the ability to serve concurrent requests from a single lambda worker, so if you're waiting for async IO then you're wasting money you wouldn't be wasting with EC2. -- For us lambda was prohibitively expensive at scale. I suspect a fair number of people reporting significant savings vs EC2 had over-provisioned instances.

pmontra 10y ago

But there are already tools like https://github.com/Miserlou/django-zappa https://github.com/Miserlou/django-zappa https://aws.amazon.com/blogs/compute/building-enterprise-level-web-applications-on-aws-lambda-with-deep/ https://aws.amazon.com/blogs/compute/building-enterprise-lev... plus something else that was featured on HN recently and I can't find or remember. Amazon itself has a tutorial about the integration of the API gateway with Lambda at http://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started.html http://docs.aws.amazon.com/apigateway/latest/developerguide/... I saw an evangelist demoing it at an Amazon Summit a month ago. It's so time consuming and error prone that you don't want to do it manually more than once (not that he made any mistake). So I won't say it's a corner case. Maybe it's too early to know what Lambda will grow into.

homero 10y ago
What's the point of API Gateway? Looks like a proxy where you recreate your api and does nothing more.
- pmontra 10y ago
  I think I leant about it when I learnt about Lambda. In this context it's a convenient way to give a URL to a function, pass parameters to it and get back the result. You could build HN with the API gw, one Lambda function per each request and a db.
  - inopinatus 10y ago
    That's exactly what I'm suggesting is an anti-pattern. Yes you can do that. Yes it's hardly surprising that AWS make it possible, since synchronous RESTful CRUD is such a popular integration approach and often the first one we learn. But I'd advise you to design your LambdaNews service using an asynchronous event-driven model instead. You'll find it reduces complexity and makes for a snappier, more reliable client experience (e.g. no 429 throttling responses to handle). You'll also find that the supporting AWS services like Kinesis, Cognito, IoT, SNS, SQS, DynamoDB, S3 etc are much more in tune with that pattern. The fact is that async platforms scale much better than anything that has to block and poll. It should hardly be surprising that at AWS's massive scale, it becomes a preferred service design pattern. Pub/sub integration people have known this for decades, it is not news. obdisclosure: I am ex-AWS but all the above is derived from public statements and documentation.
    - pmontra 10y ago
      I'm sincerely interested. The POST I use to submit this comment can be asynchronous, no problem (just do ajax). Same for votes and flags. The GET I use to get the page could be a real HTTP GET or diffs coming over a websocket (but I need a page to start with). I'm not sure how to implement that in the AWS ecosystem because I don't have much experience on it, but is it this what you mean with "asynchronous event-driven model"?
- beat 10y ago
  API Gateway does a number of things. It distributes endpoints via cloudfront, which is pretty awesome. It serves as a unified front end for a collection of back end microservices. It handles client cert authentication (important to us, but not to most people) and api keys for you, and a bunch of other neat stuff as well. And it offers a test harness where you can mock up data easily. In theory, you could do the same thing with nginx and load balancers and multiple ec2 instances, but it'd be a lot more work. And I think the real magic is with integration with Lambda. That's a game-changer.
  - homero 10y ago
    Offloading some calls into lambda does look cool

Touche 10y ago

> but the design intention of Lambda is to invoke it asynchronously via writes to dynamodb, s3, sqs, IoT etc. I wish they (they being AWS or Apex or someone in this space) said this explicitly. Every time I've read about Lambda I've thought it's interesting but am not sure how/where to invoke it in production.

dminor 10y ago
Here's their documentation on the various ways to invoke a lambda function (including the API Gateway, contrary to what was stated above): http://docs.aws.amazon.com/lambda/latest/dg/intro-core-components.html#intro-core-components-event-sources http://docs.aws.amazon.com/lambda/latest/dg/intro-core-compo...

southpolesteve 10y ago

There is nothing wrong with using lambda to synchronously respond to user facing API requests. It actually works really well and I would highly recommend it. Were doing it in production at ~15 million calls/day.

Touche 10y ago
Isn't it expensive though?
- southpolesteve 10y ago
  That is the best part! It is cheaper! We had a node service that went from $2500/mo to $400/mo just by switching form Ec2 to Lambda. That is the most apples to apples comparison I can give you. Other services were split up or rewritten, but honestly my gut feeling tells me they are costing 10x-50x less. IMO Autoscaling ec2 instances to daily traffic is not a trivial problem and you'll never get the granularity you can with Lambda
  - leoalves 10y ago
    You are also saving in not having someone maintaining and making sure the server is always online. Even if it was the same price, not having to worry about the server is a big plus
- skrowl 10y ago
  Cheaper than standing up enough EC2 instances to do the same volume. That's the idea at least.
- zbjornson 10y ago
  Strictly looking at server uptime, yes, it's nearly 5x the cost of comparable t2 on-demand EC2 instances. That can be mitigated if you have spiky traffic where you'd need to turn on several EC2 instances for less than an hour but are stuck paying for the full hour, or if you can scale to zero for extended periods of time. You don't have the ability to serve concurrent requests from a single lambda worker, so if you're waiting for async IO then you're wasting money you wouldn't be wasting with EC2. -- For us lambda was prohibitively expensive at scale. I suspect a fair number of people reporting significant savings vs EC2 had over-provisioned instances.