3 ms·

I think we're coming at things from sufficiently different angles that it's not really necessary to merge our projects, other than perhaps some common parts tha

I think we're coming at things from sufficiently different angles that it's not really necessary to merge our projects, other than perhaps some common parts that could benefit from that kind of sharing. Enigma is really a faithful reimplementation of the VM in Rust; while Lumen is an ahead-of-time compiler, which requires a very different approach. I think the goal of Enigma in making the BEAM architecture easier to understand, and providing a great learning platform for getting involved in working on the BEAM itself, or just on Enigma as an alternative is a great idea, and something I know I wish I had been able to have on hand when I was trying to understand the deep inner workings of the BEAM implementation. I if Enigma was only ever that, it would still be worth the effort spent on it. Lumen does have parts that could likely be shared with projects like Enigma - namely the high-level IR we use in the frontend (EIR, Erlang Intermediate Representation). That IR could be used in any Rust-based VM/compiler targeting Erlang, or an Erlang-derived language, and work there would directly benefit downstream consumers of the IR in terms of better optimization, etc. We've also put work into our term representation, and various parts of the runtime, like reproducing the core parts of the BEAM garbage collector, etc. While some of those things are intertwined with the compiler, much of it could be easily extracted and used elsewhere.

archseer 6y ago
Agreed! I think both projects can coexist, but there could be some components that we could share. I was in talks with Hans last year about potentially supporting EIR opcodes in Enigma. I do think that Lumen is potentially more interesting in the real world usecases though, and I wish you best of luck. Having an alternative Erlang compiler is a lot more complex but allows us to explore new techniques easier that could potentially be ported back to OTP if nothing else.
- bitwalker 6y ago
  Absolutely! I strongly believe we should be building alternative implementations of the Erlang compiler, the runtime, the virtual machine - all of the major components. It provides a lot of useful data that, if nothing else, can be used to improve the original implementations in the BEAM. Enigma is a great example of how reimplementing such a core piece of that infrastructure can provide better learning opportunities for the community, as well as a way to clean the slate and start with a fresh look at the problem with modern tools. If nothing else I'd really like to see projects like ours demonstrate the value to the core Erlang/OTP team in addressing the lack of documentation in some areas - ideally in the form of one or more specifications. Erlang deserves a specification at this point - it is very stable, and a spec would at the very least provide additional structure for future evolution. Core Erlang had a specification, but it is very much out of date at this point - considering how widely it is used as an IR for BEAM languages in general, it's disappointing it hasn't been kept up to date.

archseer 6y ago

Agreed! I think both projects can coexist, but there could be some components that we could share. I was in talks with Hans last year about potentially supporting EIR opcodes in Enigma. I do think that Lumen is potentially more interesting in the real world usecases though, and I wish you best of luck. Having an alternative Erlang compiler is a lot more complex but allows us to explore new techniques easier that could potentially be ported back to OTP if nothing else.

bitwalker 6y ago
Absolutely! I strongly believe we should be building alternative implementations of the Erlang compiler, the runtime, the virtual machine - all of the major components. It provides a lot of useful data that, if nothing else, can be used to improve the original implementations in the BEAM. Enigma is a great example of how reimplementing such a core piece of that infrastructure can provide better learning opportunities for the community, as well as a way to clean the slate and start with a fresh look at the problem with modern tools. If nothing else I'd really like to see projects like ours demonstrate the value to the core Erlang/OTP team in addressing the lack of documentation in some areas - ideally in the form of one or more specifications. Erlang deserves a specification at this point - it is very stable, and a spec would at the very least provide additional structure for future evolution. Core Erlang had a specification, but it is very much out of date at this point - considering how widely it is used as an IR for BEAM languages in general, it's disappointing it hasn't been kept up to date.