6 ms·

Lunatic is an Erlang-inspired runtime for WebAssembly

131 points by juancampa 5y ago

RcouF1uZ4gsC 5y ago
The big question that I did not see answered is does it support location transparency. One of the huge benefits of Erlang is that it is easy to take a process and move it to another machine or even another data center. If Lunatic can do that, it would be a game changer for distributed computing.
- nextaccountic 5y ago
  I don't know about Lunatic, but there's a bunch of Rust actor frameworks that run on many machines. I think I saw one that supported moving an actors between machines For example, Kay https://github.com/aeplay/kay https://github.com/aeplay/kay (used in https://aeplay.org/citybound https://aeplay.org/citybound) I'm less sure about Axiom https://docs.rs/axiom/0.2.1/axiom/ https://docs.rs/axiom/0.2.1/axiom/ And then there is Bastion https://crates.io/crates/bastion https://crates.io/crates/bastion that I think allows moving an actor to another machine
  - dlahoda 5y ago
    kay ended single machine. bastion and actix has examples of multi node cluster. wasmcloud is wasm also. riker and some other has no examples of multinode cluster
    - nextaccountic 5y ago
      Thanks for those suggestions! Do you know if some of those multi-node runtimes can move an actor from one node to another? I think I saw some Rust library do this but I can't find it.
      - dlahoda 5y ago
        i am on search myself. seeking orleans like solution. bastion has cluster support. so may be it should be checked. cluster is needed for multi node and move. so for now i am coding raw redis, k8s pods, in mem cache and websockets...
- staticassertion 5y ago
  An interesting capability of webassembly is that you can pass programs across VMs. So if you have service discovery, you basically can have RCE, which I imagine makes the "run anywhere" kinda easy to build yourself.
- bkolobara 5y ago
  Hi! Author of lunatic here. We are working on a distributed lunatic implementation built on top of nats.io.
dlahoda 5y ago
similar, so different angle https://fluence.network/ https://fluence.network/ these have state full actors for public computing in wasm. if actor service dies, your deploy new one via pi calculus topology script
sususu 5y ago
As a complete layman in WebAssembly I always struggled to understand what are the use cases. I think what I don't understand is what the _Web_ part exactly means, is a browse thing? Or could be used to more general use cases?
- reader_mode 5y ago
  Web "means" it's portable and sandboxed, so theoretically you can run untrusted from the web and it only has access to what you expose. This stacks with DOM sandboxed APIs so you get same level of isolation in the browser as JS but better perf and a memory model more suitable to other languages. Outside of browser you still get sandboxed low level VM suitable for running C and other low level languages. Eg. you could compile a C module for say node and ship it compiled as WASM
- kumarvvr 5y ago
  WebAssembly is a game changer. Why? Because it removes the need for JS, and is more performant. Couple that with a secure sand box to run your apps and you have a new distribution model, somewhere between a full native app and a completely online web app. You have also removed the shackles of JS and provided an environment to which you can compile and you get the ability to port a lot of libraries and code to the new platform. How does this help? One, you can use other languages to write apps for the web. Two, you can use existing libraries to help develop your app. Three, by having a permission system to access system resources in a secure way, you can incorporate native application features and performance into your web app. Think about stuff like direct printer access, USB access, etc.
  - dimitrios1 5y ago
    Now can we have the non-hyped up version? As someone who doesn't mind the "shackles" of JS, from what I have seen so far, apps written in WebAssembly are up to two times slower than their JS counterparts. DOM access is abysmal, and from what I've seen in the wild, no one has written a serious business critical application using it yet. What does webassembly do today, and what experience does it provide over javascript? I get the sandbox and distribution, but again, those advantages mostly apply to JS as well.
throwawayboise 5y ago
Aside from some amount of inspiration, this seems to have nothing to do with Erlang. Kind of a gratuitous use of the name IMO.
- staticassertion 5y ago
  > Lunatic's design is all about super lightweight processes. Processes are fast to create, have a small memory footprint and a low scheduling overhead. They are designed for massive concurrency. > All processes running on Lunatic are preemptively scheduled and executed by a work stealing async executor. Sounds like Erlang to me.
  - dnautics 5y ago
    Isn't go that way too? Erlang is special for other reasons.
    - staticassertion 5y ago
      Sure, lots of languages are like Erlang. Actors and CSP are pretty fundamental patterns. Some things that would make it more like Erlang than Go (but that I won't bother to verify): 1. Go isn't really pre-emptive, it just has a lot of hidden yield points afaik 2. Go doesn't enforce message passing, Erlang obviously does 3. When you send a message to a goroutine that has panicked you hang and, potentially at some point, panic. In Erlang an actor dying can send out a final exit message that other actors (likely a supervisor) can subscribe to. So yeah IDK. To me if it's got lightweight processes, pre-emption, and message passing, it's got everything that makes Erlang special. If it says it's inspired by Erlang I'm going to assume that, along with those things, it also shares goals like fault tolerance.
      - brundolf 5y ago
        "Inspired" is also a highly subjective/pretty broad statement to make. You can have the idea for something while thinking about another thing and not be wrong to say it was inspired by that other thing. I don't see any point saying "you need to justify your assertion that you were inspired by Erlang when working on this project".
staticassertion 5y ago
Thanks for posting this on a Friday, this is basically gonna be my weekend now. I love the idea of meshing a lightweight process framework with webassembly.
supermatt 5y ago
This sounds awesome! How does messaging work between processes? What sort of overhead is involved?
- bkolobara 5y ago
  Thanks! Author of lunatic here. Processes don't share any memory, so the message will need to be copied from one process to another. This should be the biggest overhead involved when sending them. Erlang uses a trick for big binary data structures, they are allocated inside a common memory space and reference counted. That way big messages can be sent just by reference. I was thinking of doing something similar, the wasm spec allows you to import multiple memories and one of the memories could be a commonly shared one. The only issue is that most higher level languages, like Rust, don't have a concept of multiple memories. So it would be really hard to use this trick inside them. Once we can run lunatic in a distributed way, some messages will need to be sent over the network and we can't cheat there. From this perspective it's even better if we can force architectures where the messages stay small.
brainless 5y ago
After reading the Elixir implementation of X-Plane MMO backend (which appeared here yesterday, 1), I came across this: https://github.com/lumen/lumen https://github.com/lumen/lumen. It is an alternative BEAM implementation, designed for WebAssembly. I am personally learning Rust and I find it interesting that there are multiple BEAM (Erlang runtime) inspired implementations in Rust gaining traction. 1. https://news.ycombinator.com/item?id=27998323 https://news.ycombinator.com/item?id=27998323
pjmlp 5y ago
> Lunatic builds on WebAssembly's security. We all use unaudited third-party libs that get deployed with our code, Lunatic can use capability based security to limit them. If they happen to load all dependencies into a giant blob of linear memory, C written extensions will take care of corrupting their runtime's memory, regardless of WebAssembly "security".
- malft 5y ago
  Yes, that's the point. If you need to parse a JPEG, you spawn a VM to do so. It is allowed to use n seconds of CPU, k bytes of memory, and send a bitmap back to the caller. (Sure, you could screw it up if the caller accepts a negative-size bitmap and corrupts its heap. Lets assume that process is written in a memory-safe language.)
  - pjmlp 5y ago
    Assumptions is how exploits get triggered.
    - sealeck 5y ago
      It would be unlikely to incorrectly assume that the language you are using is not in fact memory safe no?
aitchnyu 5y ago
Tangential, but can WASM be used to run massive legacy C/C++ codebases in a memory safe manner without slowing down too much? One example is iOS and Android messaging apps previously could parse messages that exploited the OS.

131 points by juancampa 5y ago

The big question that I did not see answered is does it support location transparency. One of the huge benefits of Erlang is that it is easy to take a process and move it to another machine or even another data center. If Lunatic can do that, it would be a game changer for distributed computing.

nextaccountic 5y ago
I don't know about Lunatic, but there's a bunch of Rust actor frameworks that run on many machines. I think I saw one that supported moving an actors between machines For example, Kay https://github.com/aeplay/kay https://github.com/aeplay/kay (used in https://aeplay.org/citybound https://aeplay.org/citybound) I'm less sure about Axiom https://docs.rs/axiom/0.2.1/axiom/ https://docs.rs/axiom/0.2.1/axiom/ And then there is Bastion https://crates.io/crates/bastion https://crates.io/crates/bastion that I think allows moving an actor to another machine
- dlahoda 5y ago
  kay ended single machine. bastion and actix has examples of multi node cluster. wasmcloud is wasm also. riker and some other has no examples of multinode cluster
  - nextaccountic 5y ago
    Thanks for those suggestions! Do you know if some of those multi-node runtimes can move an actor from one node to another? I think I saw some Rust library do this but I can't find it.
    - dlahoda 5y ago
      i am on search myself. seeking orleans like solution. bastion has cluster support. so may be it should be checked. cluster is needed for multi node and move. so for now i am coding raw redis, k8s pods, in mem cache and websockets...
staticassertion 5y ago
An interesting capability of webassembly is that you can pass programs across VMs. So if you have service discovery, you basically can have RCE, which I imagine makes the "run anywhere" kinda easy to build yourself.
bkolobara 5y ago
Hi! Author of lunatic here. We are working on a distributed lunatic implementation built on top of nats.io.

dlahoda 5y ago

similar, so different angle https://fluence.network/ https://fluence.network/ these have state full actors for public computing in wasm. if actor service dies, your deploy new one via pi calculus topology script

sususu 5y ago

As a complete layman in WebAssembly I always struggled to understand what are the use cases. I think what I don't understand is what the _Web_ part exactly means, is a browse thing? Or could be used to more general use cases?

reader_mode 5y ago
Web "means" it's portable and sandboxed, so theoretically you can run untrusted from the web and it only has access to what you expose. This stacks with DOM sandboxed APIs so you get same level of isolation in the browser as JS but better perf and a memory model more suitable to other languages. Outside of browser you still get sandboxed low level VM suitable for running C and other low level languages. Eg. you could compile a C module for say node and ship it compiled as WASM
kumarvvr 5y ago
WebAssembly is a game changer. Why? Because it removes the need for JS, and is more performant. Couple that with a secure sand box to run your apps and you have a new distribution model, somewhere between a full native app and a completely online web app. You have also removed the shackles of JS and provided an environment to which you can compile and you get the ability to port a lot of libraries and code to the new platform. How does this help? One, you can use other languages to write apps for the web. Two, you can use existing libraries to help develop your app. Three, by having a permission system to access system resources in a secure way, you can incorporate native application features and performance into your web app. Think about stuff like direct printer access, USB access, etc.
- dimitrios1 5y ago
  Now can we have the non-hyped up version? As someone who doesn't mind the "shackles" of JS, from what I have seen so far, apps written in WebAssembly are up to two times slower than their JS counterparts. DOM access is abysmal, and from what I've seen in the wild, no one has written a serious business critical application using it yet. What does webassembly do today, and what experience does it provide over javascript? I get the sandbox and distribution, but again, those advantages mostly apply to JS as well.

throwawayboise 5y ago

Aside from some amount of inspiration, this seems to have nothing to do with Erlang. Kind of a gratuitous use of the name IMO.

staticassertion 5y ago
> Lunatic's design is all about super lightweight processes. Processes are fast to create, have a small memory footprint and a low scheduling overhead. They are designed for massive concurrency. > All processes running on Lunatic are preemptively scheduled and executed by a work stealing async executor. Sounds like Erlang to me.
- dnautics 5y ago
  Isn't go that way too? Erlang is special for other reasons.
  - staticassertion 5y ago
    Sure, lots of languages are like Erlang. Actors and CSP are pretty fundamental patterns. Some things that would make it more like Erlang than Go (but that I won't bother to verify): 1. Go isn't really pre-emptive, it just has a lot of hidden yield points afaik 2. Go doesn't enforce message passing, Erlang obviously does 3. When you send a message to a goroutine that has panicked you hang and, potentially at some point, panic. In Erlang an actor dying can send out a final exit message that other actors (likely a supervisor) can subscribe to. So yeah IDK. To me if it's got lightweight processes, pre-emption, and message passing, it's got everything that makes Erlang special. If it says it's inspired by Erlang I'm going to assume that, along with those things, it also shares goals like fault tolerance.
    - brundolf 5y ago
      "Inspired" is also a highly subjective/pretty broad statement to make. You can have the idea for something while thinking about another thing and not be wrong to say it was inspired by that other thing. I don't see any point saying "you need to justify your assertion that you were inspired by Erlang when working on this project".

staticassertion 5y ago

Thanks for posting this on a Friday, this is basically gonna be my weekend now. I love the idea of meshing a lightweight process framework with webassembly.

supermatt 5y ago

This sounds awesome! How does messaging work between processes? What sort of overhead is involved?

bkolobara 5y ago
Thanks! Author of lunatic here. Processes don't share any memory, so the message will need to be copied from one process to another. This should be the biggest overhead involved when sending them. Erlang uses a trick for big binary data structures, they are allocated inside a common memory space and reference counted. That way big messages can be sent just by reference. I was thinking of doing something similar, the wasm spec allows you to import multiple memories and one of the memories could be a commonly shared one. The only issue is that most higher level languages, like Rust, don't have a concept of multiple memories. So it would be really hard to use this trick inside them. Once we can run lunatic in a distributed way, some messages will need to be sent over the network and we can't cheat there. From this perspective it's even better if we can force architectures where the messages stay small.

brainless 5y ago

After reading the Elixir implementation of X-Plane MMO backend (which appeared here yesterday, 1), I came across this: https://github.com/lumen/lumen https://github.com/lumen/lumen. It is an alternative BEAM implementation, designed for WebAssembly. I am personally learning Rust and I find it interesting that there are multiple BEAM (Erlang runtime) inspired implementations in Rust gaining traction. 1. https://news.ycombinator.com/item?id=27998323 https://news.ycombinator.com/item?id=27998323

pjmlp 5y ago

> Lunatic builds on WebAssembly's security. We all use unaudited third-party libs that get deployed with our code, Lunatic can use capability based security to limit them. If they happen to load all dependencies into a giant blob of linear memory, C written extensions will take care of corrupting their runtime's memory, regardless of WebAssembly "security".

malft 5y ago
Yes, that's the point. If you need to parse a JPEG, you spawn a VM to do so. It is allowed to use n seconds of CPU, k bytes of memory, and send a bitmap back to the caller. (Sure, you could screw it up if the caller accepts a negative-size bitmap and corrupts its heap. Lets assume that process is written in a memory-safe language.)
- pjmlp 5y ago
  Assumptions is how exploits get triggered.
  - sealeck 5y ago
    It would be unlikely to incorrectly assume that the language you are using is not in fact memory safe no?

aitchnyu 5y ago

Tangential, but can WASM be used to run massive legacy C/C++ codebases in a memory safe manner without slowing down too much? One example is iOS and Android messaging apps previously could parse messages that exploited the OS.