9 ms·

part of the "magic" of Lambda is making it easy to use from your programming language of choice. If they made a lower-level model, like, for example, "we start

by sreque 9y ago

part of the "magic" of Lambda is making it easy to use from your programming language of choice. If they made a lower-level model, like, for example, "we start up your binary, and it needs to bind to port X, then we'll send HTTP requests to it and it should respond appropriately using the following JSON specification," it would take a lot more work to set up your Lambda function and get it working correctly than it does now. It's certainly possible for them to do it, but I don't think it fits in with their style. Worst case, you can implement this interface yourself now easier than ever. Since golang is statically linked and has super fast startup times, you can build an interface where Lambda launches your golang process, and that process binds to a port and spawns a sub-process that implements the actual Lambda function. The golang process would implement the "socket" based interface for you. People have been doing the above already, but all the existing runtimes, including Node.js, Java, Python, and .NET, have non-trivial startup times and memory usage compared to a golang binary, which should be able to start up in 10s of milliseconds and use less than 1 MB of memory while acting as a proxy.

aaronblohowiak 9y ago
...or they could support FastCGI or CGI which would then have the "low barrier to entry" and "broad support" already handled..
- xienze 9y ago
  I think there's a lot of secret sauce involved in order to make Lambdas perform well. They are definitely not just executing a binary every time a request comes in. You might give them a binary but that's not how it's executed.
  - aaronblohowiak 9y ago
    that's the difference between CGI and FastCGI, though.
    - stock_toaster 9y ago
      I agree. FastCGI or SCGI seem like they would be great solutions, instead of custom rpc formats.
notheguyouthink 9y ago
> If they made a lower-level model, like, for example, "we start up your binary, and it needs to bind to port X, then we'll send HTTP requests to it and it should respond appropriately using the following JSON specification," it would take a lot more work to set up your Lambda function and get it working correctly than it does now. Man, the JSON specification must be annoying/complicated - because the first part of what you said, binding to port X and responding to HTTP, is like.. 1 line in Go. So the first part of what you said is extremely trivial, so the difficulty must lie in the second part? I wonder what makes the JSON specification so difficult? From your wording, I assume the HTTP response is being sent to AWS, not the actual HTTP requester. Regardless, it must be damn clumsy to justify calling a simple HTTP response too complicated for developers. Clearly I've never used Lambda though.
- sreque 9y ago
  I'm only saying that it would take more effort and place more burden on a developer to figure out how to do all that in their programming language rather than just implement a function. I'm surprised this is a contested point. Tell me in Java, without looking up any online resource, how to set up a FastCGI web application. Can you? Probably not. Now tell me how to write a static method in Java. If you have at least the equivalent Java experience of a freshman undergrad, you can do the latter but not the former. If you think Lambda should use FastCGI as its common interface, then by all means create a project on github that deploys a Go-based FastCGI bridge to Lambda and makes it easy for you to bundle up your FastCGI application into a Lambda function. Problem solved! Now you just have to get people to use it.
  - YawningAngel 9y ago
    I'm pretty sure anyone who programs Java for a living can start up Jetty via their favourite framework. It isn't as if lambdas are trivial to package either.
  - SideburnsOfDoom 9y ago
    > I'm only saying that it would take more effort and place more burden on a developer to figure out how to do all that in their programming language rather than just implement a function But the average developer isn't going to do that, and won't need to. Within a week there would be more than one framework up on github to do the binding in their favourite language for them. Look at how many web frameworks there are.
erik_seaberg 9y ago
"Parse JSON from a request that arrived over HTTP" is not a major task in any language that's seen serious use since 2002. And if they had done this they would have support for a dozen better languages literally overnight. The industry has way too many frameworks that should have been protocols.
- philwelch 9y ago
  I’m curious which dozen languages you find better than the ones supported by Lambda.
  - mi100hael 9y ago
    The point is that no one would have to be debating which languages should be supported by Lambda if they made their environment language-agnostic based on a protocol rather than a bunch of individual libraries.
  - electricEmu 9y ago
    .NET Core 2.0, until _very_ recently.
  - eadmund 9y ago
    Common Lisp. Until just now, Go. I hear great things about Java 9. TCL. Rebol. Smalltalk. That's half a dozen (-ish); I'll let someone else complete it.
    - brabel 9y ago
      TCL, Smalltalk, Rebol??? So they could support all 5 users of these languages? Lol
  - deleted 9y ago
    [deleted]
seniorsassycat 9y ago
They've already implemented a lower-level protocol "we fork/exec your binary, it needs to bind to port _LAMBDA_SERVER_PORT, and it must follow gos net/rpc protocol". aws provided a library, aws-lambda-go, that every go implementer must use. Anyone can implement their own 'aws-lambda-go' library in what ever language they want, but the choice to use net/rpc and it's gobs format makes it more difficult than it could be. json is implemented in practically every language. there are binary formats with libraries is most languages if your want to be efficient. Choosing a protocol other than net/rpc would 1. make it easier for AWS to support more runtimes 2. make it easier for the community to support custom runtimes. 3. be as easy to integrate with as aws-lambda-go is.
donatj 9y ago
I don't understand why it can't all be done over standard in / standard out? Why bother with ports at all?
- sriku 9y ago
  Yup. It used to be called CGI ... In the '90s.
- ninkendo 9y ago
  Because then you’d need a (inefficient) fork/execing webserver, or some convoluted (error-prone) way of multiplexing multiple connections over one stream. The OS file descriptor model already handles this for you with listen() and accept(). CGI only works if you’re willing to take the hit of launching an entire process (good luck using any interpreted language, and even go needs some runtime initialization at startup) for every single web request.
  - nivertech 9y ago
    Lambda/FaaS is more like FastCGI https://en.wikipedia.org/wiki/FastCGI https://en.wikipedia.org/wiki/FastCGI
    - ninkendo 9y ago
      Replying way late here, but FastCGI doesn't work by printing to stdout, it talks to a domain socket or an open TCP port. The grandparent post was asking "why can't lambda just use stdout?" The magic of TCP or domain sockets is that calling accept() gives you a unique file descriptor that corresponds with a single client. Just using stdout doesn't work that way, unless your entire process was launched to handle one request, hence CGI being very inefficient.

by sreque 9y ago

aaronblohowiak 9y ago

...or they could support FastCGI or CGI which would then have the "low barrier to entry" and "broad support" already handled..

xienze 9y ago
I think there's a lot of secret sauce involved in order to make Lambdas perform well. They are definitely not just executing a binary every time a request comes in. You might give them a binary but that's not how it's executed.
- aaronblohowiak 9y ago
  that's the difference between CGI and FastCGI, though.
  - stock_toaster 9y ago
    I agree. FastCGI or SCGI seem like they would be great solutions, instead of custom rpc formats.

notheguyouthink 9y ago

> If they made a lower-level model, like, for example, "we start up your binary, and it needs to bind to port X, then we'll send HTTP requests to it and it should respond appropriately using the following JSON specification," it would take a lot more work to set up your Lambda function and get it working correctly than it does now. Man, the JSON specification must be annoying/complicated - because the first part of what you said, binding to port X and responding to HTTP, is like.. 1 line in Go. So the first part of what you said is extremely trivial, so the difficulty must lie in the second part? I wonder what makes the JSON specification so difficult? From your wording, I assume the HTTP response is being sent to AWS, not the actual HTTP requester. Regardless, it must be damn clumsy to justify calling a simple HTTP response too complicated for developers. Clearly I've never used Lambda though.

sreque 9y ago
I'm only saying that it would take more effort and place more burden on a developer to figure out how to do all that in their programming language rather than just implement a function. I'm surprised this is a contested point. Tell me in Java, without looking up any online resource, how to set up a FastCGI web application. Can you? Probably not. Now tell me how to write a static method in Java. If you have at least the equivalent Java experience of a freshman undergrad, you can do the latter but not the former. If you think Lambda should use FastCGI as its common interface, then by all means create a project on github that deploys a Go-based FastCGI bridge to Lambda and makes it easy for you to bundle up your FastCGI application into a Lambda function. Problem solved! Now you just have to get people to use it.
- YawningAngel 9y ago
  I'm pretty sure anyone who programs Java for a living can start up Jetty via their favourite framework. It isn't as if lambdas are trivial to package either.
- SideburnsOfDoom 9y ago
  > I'm only saying that it would take more effort and place more burden on a developer to figure out how to do all that in their programming language rather than just implement a function But the average developer isn't going to do that, and won't need to. Within a week there would be more than one framework up on github to do the binding in their favourite language for them. Look at how many web frameworks there are.

erik_seaberg 9y ago

"Parse JSON from a request that arrived over HTTP" is not a major task in any language that's seen serious use since 2002. And if they had done this they would have support for a dozen better languages literally overnight. The industry has way too many frameworks that should have been protocols.

philwelch 9y ago
I’m curious which dozen languages you find better than the ones supported by Lambda.
- mi100hael 9y ago
  The point is that no one would have to be debating which languages should be supported by Lambda if they made their environment language-agnostic based on a protocol rather than a bunch of individual libraries.
- electricEmu 9y ago
  .NET Core 2.0, until _very_ recently.
- eadmund 9y ago
  Common Lisp. Until just now, Go. I hear great things about Java 9. TCL. Rebol. Smalltalk. That's half a dozen (-ish); I'll let someone else complete it.
  - brabel 9y ago
    TCL, Smalltalk, Rebol??? So they could support all 5 users of these languages? Lol
- deleted 9y ago
  [deleted]

seniorsassycat 9y ago

They've already implemented a lower-level protocol "we fork/exec your binary, it needs to bind to port _LAMBDA_SERVER_PORT, and it must follow gos net/rpc protocol". aws provided a library, aws-lambda-go, that every go implementer must use. Anyone can implement their own 'aws-lambda-go' library in what ever language they want, but the choice to use net/rpc and it's gobs format makes it more difficult than it could be. json is implemented in practically every language. there are binary formats with libraries is most languages if your want to be efficient. Choosing a protocol other than net/rpc would 1. make it easier for AWS to support more runtimes 2. make it easier for the community to support custom runtimes. 3. be as easy to integrate with as aws-lambda-go is.

donatj 9y ago

I don't understand why it can't all be done over standard in / standard out? Why bother with ports at all?

sriku 9y ago
Yup. It used to be called CGI ... In the '90s.
ninkendo 9y ago
Because then you’d need a (inefficient) fork/execing webserver, or some convoluted (error-prone) way of multiplexing multiple connections over one stream. The OS file descriptor model already handles this for you with listen() and accept(). CGI only works if you’re willing to take the hit of launching an entire process (good luck using any interpreted language, and even go needs some runtime initialization at startup) for every single web request.
- nivertech 9y ago
  Lambda/FaaS is more like FastCGI https://en.wikipedia.org/wiki/FastCGI https://en.wikipedia.org/wiki/FastCGI
  - ninkendo 9y ago
    Replying way late here, but FastCGI doesn't work by printing to stdout, it talks to a domain socket or an open TCP port. The grandparent post was asking "why can't lambda just use stdout?" The magic of TCP or domain sockets is that calling accept() gives you a unique file descriptor that corresponds with a single client. Just using stdout doesn't work that way, unless your entire process was launched to handle one request, hence CGI being very inefficient.