8 ms·

Using GraalVM to Run Native Java in AWS Lambda with Golang

152 points by CSDude 8y ago

kodablah 8y ago
You might be able to even compile Go as a static library and embed it in your binary instead of carrying a .so file around. Regardless, Go's FFI story (especially in that direction) is hobbled and you can't do things like share structs easily (for obvious reasons, but still annoying). But yes, as soon as you get to C-level, you can mix and match anything and write about it. Running Go in Rust, or Rust in Go, or Go in JS, or whatever. What would really be neat is to leverage Graal's polyglot iface a bit higher. I am not sure the status of llgo these days, but compile Go to LLVM bitcode and then leverage that from Java and compile to a single binary. Or even expose Go's awesome stdlib to JVM developers that way. But the practicality of doing some of these things becomes a bit lost beyond toy use.
- CSDude 8y ago
  Yeah, Making use of Graal’s polyglot interface would be better, but I did not have much luck to compile Go to LLVM, as I am not so familiar, but would try that soon more intensely.
  - kodablah 8y ago
    Word. Another fun transpiling project I have been mulling over is a Go-to-Kotlin transpiler. With Kotlin coroutines (and accompanying kotlinx lib) I think it has the entire feature set covered though duck typing requires a creative solution but it's not too bad. Keeping on the non-practical, fun note, I wrote a transpiler in the other direction[0]. It compiles Java bytecode to Go code. It works decently actually but I stopped work on it because my approach strains the Go compiler too much[1]. I even started emulating some of the base Java calls as Go calls, but that became tedious too. 0 - https://github.com/cretz/goahead https://github.com/cretz/goahead 1 - https://github.com/golang/go/issues/18602 https://github.com/golang/go/issues/18602
- weberc2 8y ago
  Pretty sure llgo is dead in the water.
benmmurphy 8y ago
the other alternative is to just implement the go-aws-lambda protocol. it is basically a bunch of go RPC calls that AWS makes to a port that you listen on. if you do it correctly it should be safe to run because AWS can't change the protocol in a way that is not backwards compatible with existing go projects. any language that can compile statically and be run on linux can run on AWS lambda because static-go binaries can be deployed to AWS lambda. but this is probably a little less riskier because i'm sure if AWS made a change to the go protocol they would test it against their existing go libraries they have published but they can't test it against your <insert language> implementation.
- lvh 8y ago
  The RPC mechanism uses Gob, not something like gRPC, so first you’d have to write a Gob parser. It’s not clear to me that that’s better.
  - CSDude 8y ago
    Yeah, I already mentioned that in the blog, Although Gob is simple enough, this was better as an experiment. But implementing Gob would be more mature choice in the long run.
- curun1r 8y ago
  > any language that can compile statically and be run on linux Most languages that satisfy this criteria can also speak the C ABI. So why not write a native module for Node, Python or Java? Dealing with those interfaces is faster and they're better standardized. FWIW.../me is running Rust on lambda as a Node native module.
ianamartin 8y ago
Christ, I'm officially old now. Current PhD candidates think cars needed to warm up in the 80s.
- CSDude 8y ago
  Well my car from 93 needed to warm up :(
  - hinkley 8y ago
    When it was new or when it was old? Cars from the 80's only needed to warm up if you lived in the upper Midwest, or when they got old and started falling apart. If you were a kid in the 80's or 90's and your neighborhood wasn't well off you might be forgiven for thinking all cars had this problem, because everyone drove used cars until they fell apart. [Edit] Looks like OP is going to a University in Turkey. The possibility of cultural disconnects with SV and PNW is not surprising at all.
    - CSDude 8y ago
      Well, it was a Renault 21 with a carburetor, and it had only 150K kms for over 20 years and nice use. It used to work great in Izmir where it was warm but when I attended University at Ankara where it was freezing, it required more than 3 ignitions, keeping it above 2.5k RPM and the handle (I do not know its Enligsh name) must be on (https://youtu.be/jLfnwRavuy8?t=1m4s https://youtu.be/jLfnwRavuy8?t=1m4s) which pumped more gas so cold weather was reduced, I guess. But it might happen in kinda in warm weather too, at 5C
- kuwze 8y ago
  Wait, don't cars have to warm up so you don't damage the engine when it's cold outside?
  - roywiggins 8y ago
    Modern cars don't. https://www.washingtonpost.com/news/wonk/wp/2014/12/29/the-biggest-winter-energy-myth-that-you-need-to-idle-your-car-before-driving/ https://www.washingtonpost.com/news/wonk/wp/2014/12/29/the-b...
deleted 8y ago
[deleted]
lvh 8y ago
So, clearly you can't use the real Java runtime because it expects to load an endpoint from a jar, but I wonder what the equivalent Python runtime version would look like. Prior to explicit Go support, people used the Python runtime as a shim to start a Go binary. Is there anything about the way the Go runtime works that makes it fundamentally better than the Python runtime's semantics? Is the entirety of it that you really don't need to do a lot of work but Go is going to be marginally faster at doing that tiny bit of work (dlopen and ferry bytes across)? I think the underlying serialization format in Python is JSON. I have no idea how fast Gob is. Any idea how much you lose on the FFI bridge? (My understanding is you need to copy the Go struct you get out before you can pass it to FFI. I'm very familiar with FFI in general and especially on the JVM, but only superficially with Go's in particular.)
- _wmd 8y ago
  Lambda's design is a little perplexing in this regard.. latencies to invoke a function, even for a warm function are pretty ridiculous, and any on-host copies will be totally invisible given the networking environment. Despite that, e.g. the Python runtime implements RPCs by sharing a memory segment with the host process running outside the container, which seems like massive overkill to me. I played with writing a library that would exec a new binary over the top of Python (for fun), but doing so loses access to the segment. There doesn't seem to be any UNIX API that would allow access to it to be recovered across exec. It might be possible by leaving the Python process idle and mmapping from /proc/parent_pid/mem in a child, but that requires debugging privileges almost certainly absent in a container. The segment is created by mmapping a passed FD, which is then closed by the bootstrap code. I couldn't find any mechanism for moving that segment across processes, or inheriting it across exec. So much simpler if they just implemented HTTP over a socket or suchlike
gw 8y ago
Is there any advantage to this over using Lambda's python runtime to run a binary? That seems to be what this project is doing to run a natively-compiled Clojure project: https://github.com/uwcpdx/clombda https://github.com/uwcpdx/clombda
- maerF0x0 8y ago
  bootstrapping time is the first thing that comes to mind.
heyyyouu 8y ago
Side note: great dek on this story.
ericfrederich 8y ago
Sweet... I feel like someone should do this for a Jython program as a joke. Then you'd have Python code executed by an interpreter written in Java compiled by Graal to produce a native binary which will be registered on AWS as a Go executable and will communicate via Go's net/rpc. ... as opposed to switching the runtime to Python ;-)
Twirrim 8y ago
Okay, I can see what you're doing. I'm a little confused why. This seems like a potential rube goldberg machine that may exhibit some strange and unpredictable behaviour. Maybe I'm crazy, but I much prefer to keep things as simple as possible. Pick a technology stack and use just it. Don't play mix and match games. If java is just not working for you in lambda, due to the startup cost, is it time to consider that Java might not be the right technology at all for you to be using there? If you're determined to stick with lambda, maybe it's time to bite-the-bullet and use Go (or something else with fast startup times). If you're determined to stick with Java, maybe it's time to consider that maybe Lambda isn't the right solution for you? How much benefit are you really seeing vs the engineering cost and complications?
- Yeroc 8y ago
  To be fair, there's a disclaimer right in the first paragraph: "But in this blog, we will introduce you a probably not practical but a cool use case using GraalVM." (sic)
- kjeetgill 8y ago
  I think it's cool hack / proof of concept. Sometimes you need to know something is possible before you start investing if it'll solve your problem.
bitwize 8y ago
> This title might seem like generated by a Markov Chain on Hacker News submissions, but it is real. I smiled. But a more plausible Markov-chain headline might be "Using GraalVM to Run Native Java with Rust, Machine Learning, and the Gut Microbiome".
CSDude 8y ago
FYI There is a blog post for interested, a reverse engine of AWS Lambda for Python, which does not use RPC/network at all, instead uses shared memory to communicate, which could be more of a challenge. https://www.denialof.services/lambda/ https://www.denialof.services/lambda/

152 points by CSDude 8y ago

You might be able to even compile Go as a static library and embed it in your binary instead of carrying a .so file around. Regardless, Go's FFI story (especially in that direction) is hobbled and you can't do things like share structs easily (for obvious reasons, but still annoying). But yes, as soon as you get to C-level, you can mix and match anything and write about it. Running Go in Rust, or Rust in Go, or Go in JS, or whatever. What would really be neat is to leverage Graal's polyglot iface a bit higher. I am not sure the status of llgo these days, but compile Go to LLVM bitcode and then leverage that from Java and compile to a single binary. Or even expose Go's awesome stdlib to JVM developers that way. But the practicality of doing some of these things becomes a bit lost beyond toy use.

CSDude 8y ago
Yeah, Making use of Graal’s polyglot interface would be better, but I did not have much luck to compile Go to LLVM, as I am not so familiar, but would try that soon more intensely.
- kodablah 8y ago
  Word. Another fun transpiling project I have been mulling over is a Go-to-Kotlin transpiler. With Kotlin coroutines (and accompanying kotlinx lib) I think it has the entire feature set covered though duck typing requires a creative solution but it's not too bad. Keeping on the non-practical, fun note, I wrote a transpiler in the other direction[0]. It compiles Java bytecode to Go code. It works decently actually but I stopped work on it because my approach strains the Go compiler too much[1]. I even started emulating some of the base Java calls as Go calls, but that became tedious too. 0 - https://github.com/cretz/goahead https://github.com/cretz/goahead 1 - https://github.com/golang/go/issues/18602 https://github.com/golang/go/issues/18602
weberc2 8y ago
Pretty sure llgo is dead in the water.

benmmurphy 8y ago

the other alternative is to just implement the go-aws-lambda protocol. it is basically a bunch of go RPC calls that AWS makes to a port that you listen on. if you do it correctly it should be safe to run because AWS can't change the protocol in a way that is not backwards compatible with existing go projects. any language that can compile statically and be run on linux can run on AWS lambda because static-go binaries can be deployed to AWS lambda. but this is probably a little less riskier because i'm sure if AWS made a change to the go protocol they would test it against their existing go libraries they have published but they can't test it against your <insert language> implementation.

lvh 8y ago
The RPC mechanism uses Gob, not something like gRPC, so first you’d have to write a Gob parser. It’s not clear to me that that’s better.
- CSDude 8y ago
  Yeah, I already mentioned that in the blog, Although Gob is simple enough, this was better as an experiment. But implementing Gob would be more mature choice in the long run.
curun1r 8y ago
> any language that can compile statically and be run on linux Most languages that satisfy this criteria can also speak the C ABI. So why not write a native module for Node, Python or Java? Dealing with those interfaces is faster and they're better standardized. FWIW.../me is running Rust on lambda as a Node native module.

ianamartin 8y ago

Christ, I'm officially old now. Current PhD candidates think cars needed to warm up in the 80s.

CSDude 8y ago
Well my car from 93 needed to warm up :(
- hinkley 8y ago
  When it was new or when it was old? Cars from the 80's only needed to warm up if you lived in the upper Midwest, or when they got old and started falling apart. If you were a kid in the 80's or 90's and your neighborhood wasn't well off you might be forgiven for thinking all cars had this problem, because everyone drove used cars until they fell apart. [Edit] Looks like OP is going to a University in Turkey. The possibility of cultural disconnects with SV and PNW is not surprising at all.
  - CSDude 8y ago
    Well, it was a Renault 21 with a carburetor, and it had only 150K kms for over 20 years and nice use. It used to work great in Izmir where it was warm but when I attended University at Ankara where it was freezing, it required more than 3 ignitions, keeping it above 2.5k RPM and the handle (I do not know its Enligsh name) must be on (https://youtu.be/jLfnwRavuy8?t=1m4s https://youtu.be/jLfnwRavuy8?t=1m4s) which pumped more gas so cold weather was reduced, I guess. But it might happen in kinda in warm weather too, at 5C
kuwze 8y ago
Wait, don't cars have to warm up so you don't damage the engine when it's cold outside?
- roywiggins 8y ago
  Modern cars don't. https://www.washingtonpost.com/news/wonk/wp/2014/12/29/the-biggest-winter-energy-myth-that-you-need-to-idle-your-car-before-driving/ https://www.washingtonpost.com/news/wonk/wp/2014/12/29/the-b...

deleted 8y ago

[deleted]

lvh 8y ago

So, clearly you can't use the real Java runtime because it expects to load an endpoint from a jar, but I wonder what the equivalent Python runtime version would look like. Prior to explicit Go support, people used the Python runtime as a shim to start a Go binary. Is there anything about the way the Go runtime works that makes it fundamentally better than the Python runtime's semantics? Is the entirety of it that you really don't need to do a lot of work but Go is going to be marginally faster at doing that tiny bit of work (dlopen and ferry bytes across)? I think the underlying serialization format in Python is JSON. I have no idea how fast Gob is. Any idea how much you lose on the FFI bridge? (My understanding is you need to copy the Go struct you get out before you can pass it to FFI. I'm very familiar with FFI in general and especially on the JVM, but only superficially with Go's in particular.)

_wmd 8y ago
Lambda's design is a little perplexing in this regard.. latencies to invoke a function, even for a warm function are pretty ridiculous, and any on-host copies will be totally invisible given the networking environment. Despite that, e.g. the Python runtime implements RPCs by sharing a memory segment with the host process running outside the container, which seems like massive overkill to me. I played with writing a library that would exec a new binary over the top of Python (for fun), but doing so loses access to the segment. There doesn't seem to be any UNIX API that would allow access to it to be recovered across exec. It might be possible by leaving the Python process idle and mmapping from /proc/parent_pid/mem in a child, but that requires debugging privileges almost certainly absent in a container. The segment is created by mmapping a passed FD, which is then closed by the bootstrap code. I couldn't find any mechanism for moving that segment across processes, or inheriting it across exec. So much simpler if they just implemented HTTP over a socket or suchlike

gw 8y ago

Is there any advantage to this over using Lambda's python runtime to run a binary? That seems to be what this project is doing to run a natively-compiled Clojure project: https://github.com/uwcpdx/clombda https://github.com/uwcpdx/clombda

maerF0x0 8y ago
bootstrapping time is the first thing that comes to mind.

heyyyouu 8y ago

Side note: great dek on this story.

ericfrederich 8y ago

Sweet... I feel like someone should do this for a Jython program as a joke. Then you'd have Python code executed by an interpreter written in Java compiled by Graal to produce a native binary which will be registered on AWS as a Go executable and will communicate via Go's net/rpc. ... as opposed to switching the runtime to Python ;-)

Twirrim 8y ago

Okay, I can see what you're doing. I'm a little confused why. This seems like a potential rube goldberg machine that may exhibit some strange and unpredictable behaviour. Maybe I'm crazy, but I much prefer to keep things as simple as possible. Pick a technology stack and use just it. Don't play mix and match games. If java is just not working for you in lambda, due to the startup cost, is it time to consider that Java might not be the right technology at all for you to be using there? If you're determined to stick with lambda, maybe it's time to bite-the-bullet and use Go (or something else with fast startup times). If you're determined to stick with Java, maybe it's time to consider that maybe Lambda isn't the right solution for you? How much benefit are you really seeing vs the engineering cost and complications?

Yeroc 8y ago
To be fair, there's a disclaimer right in the first paragraph: "But in this blog, we will introduce you a probably not practical but a cool use case using GraalVM." (sic)
kjeetgill 8y ago
I think it's cool hack / proof of concept. Sometimes you need to know something is possible before you start investing if it'll solve your problem.

bitwize 8y ago

> This title might seem like generated by a Markov Chain on Hacker News submissions, but it is real. I smiled. But a more plausible Markov-chain headline might be "Using GraalVM to Run Native Java with Rust, Machine Learning, and the Gut Microbiome".

CSDude 8y ago

FYI There is a blog post for interested, a reverse engine of AWS Lambda for Python, which does not use RPC/network at all, instead uses shared memory to communicate, which could be more of a challenge. https://www.denialof.services/lambda/ https://www.denialof.services/lambda/