4 ms·

Is Lisp + message passing a bridge too far? ie https://lfe.io/ https://lfe.io/ Does Lisp on the BEAM hold any appeal for you?

lisper 4y ago
I don't know what Lisp on the BEAM is (and neither does DDG). With regards to LFE, I don't really see the point. But that might just be because of the kinds of coding I do. I've never written high performance massively parallel code. Message passing might have a benefit there (that is what the big win with Erlang is supposed to be) but I'm generally skeptical of designing the language semantics around the needs of the compiler rather than the programmer. In particular, I don't see what Erlang could possibly do that could not be done by a CL compiler that noticed when a method was dispatching only on the first argument. The semantics of that are equivalent to the semantics of message passing, and so the code that such a compiler emits could be identical in both cases. But I don't actually know Erlang so it's possible I'm missing something.
- samatman 4y ago
  BEAM is the Erlang VM so you covered the question nicely.
- GenericJam 4y ago
  With Erlang it's necessary to separate the compiler from the VM. You are working around what the VM wants but the compiler is not a problem. The VM acts a bit like an operating system that can schedule its own processes and processes can send each other messages. In order to receive a message a process has to have a receive block and be capable of receiving the message (with an applicable pattern match). When a process is waiting for a message, it is set aside by the VM so it consumes very few resources (just a bit of memory). It is woken up when it receives a message. This way a single 'program' can have many things going on at once and it is never blocking. From the programmer's perspective it feels like you are writing synchronous code but you are getting async behaviour 'for free'. wrt LFE specifically, Robert Virding, one of the founders of Erlang, really likes languages and Lisp in particular so he wrote one for the BEAM. It just has some special accommodations to be able to send/receive (and I think pattern recognition too).
  - lisper 4y ago
    That actually sounds pretty cool as a runtime environment. Is the VM programmed in byte code, or is it native code? What back ends are available? What is the runtime written in?
    - GenericJam 4y ago
      The BEAM (runtime?) is written in C. There is also an effort to rewrite it in Rust (https://github.com/lumen/lumen https://github.com/lumen/lumen). Some functions are built into the VM but most of the supporting 'standard library' (OTP / Open Telecom Platform) is written in Erlang. The (main) compiler is written in C. So it's all C or Erlang afaik. It is ported to every major flavour of OS. I don't know what 'back end' means in this context.
      - lisper 4y ago
        You can compile a high-level language down to a byte code which looks like machine code but which does not correspond to any actual hardware. Instead, the byte code is interpreted. Python and Java both work this way. You can also compile a high level language down to machine code that runs on actual hardware, like an x86 or an ARM. For languages that run on more than on processor, the compilation process usually consists of an architecture-independent phase which produces some sort of intermediate representation (which may be a byte code, or it might be something else, like LLVM) and then an architecture-specific pass that transforms the intermediate representation into machine code for the target architecture. The code that implements that second pass is called the "back end." I have no idea whether Erlang compiles to native code or byte code. In addition to all that, there can also be a run-time environment that is required to run the resulting code. For byte code, this environment necessarily includes an interpreter for the byte code, and might also include other things. For native code this environment might include things like a garbage collector or a standard library that provides an interface to an operating system or something like that.
        GenericJam 4y ago
- GenericJam 4y ago
  Another thing about the BEAM that may resonate for you is the abstract syntax tree (AST) for the BEAM is very reminiscent of Lisp. Some people call Elixir which is a popular reworking of Erlang which also runs on the BEAM a secret Lisp as it gives direct access to the AST via macros. A friend wrote on the topic here: https://dorgan.netlify.app/posts/2021/04/the_elixir_ast/ https://dorgan.netlify.app/posts/2021/04/the_elixir_ast/
- abecedarius 4y ago
  Hi Ron, I'd recommend Joe Armstrong's thesis https://erlang.org/download/armstrong_thesis_2003.pdf https://erlang.org/download/armstrong_thesis_2003.pdf to get an idea what's special about Erlang. Briefly, it's designed around keeping systems running live even when they have bugs and other faults. Straight Lisp with single-dispatch wouldn't isolate effects, though of course you could build something in Lisp that did. Initially Erlang was built on top of Prolog. (This is Darius. I never really learned your system with related goals for robots -- as you know, I didn't stick around at JPL. About 10 years later I did a bit of Erlang for Yahoo.)
  - lisper 4y ago
    Hi Darius! :-)

lisper 4y ago

I don't know what Lisp on the BEAM is (and neither does DDG). With regards to LFE, I don't really see the point. But that might just be because of the kinds of coding I do. I've never written high performance massively parallel code. Message passing might have a benefit there (that is what the big win with Erlang is supposed to be) but I'm generally skeptical of designing the language semantics around the needs of the compiler rather than the programmer. In particular, I don't see what Erlang could possibly do that could not be done by a CL compiler that noticed when a method was dispatching only on the first argument. The semantics of that are equivalent to the semantics of message passing, and so the code that such a compiler emits could be identical in both cases. But I don't actually know Erlang so it's possible I'm missing something.

samatman 4y ago
BEAM is the Erlang VM so you covered the question nicely.
GenericJam 4y ago
With Erlang it's necessary to separate the compiler from the VM. You are working around what the VM wants but the compiler is not a problem. The VM acts a bit like an operating system that can schedule its own processes and processes can send each other messages. In order to receive a message a process has to have a receive block and be capable of receiving the message (with an applicable pattern match). When a process is waiting for a message, it is set aside by the VM so it consumes very few resources (just a bit of memory). It is woken up when it receives a message. This way a single 'program' can have many things going on at once and it is never blocking. From the programmer's perspective it feels like you are writing synchronous code but you are getting async behaviour 'for free'. wrt LFE specifically, Robert Virding, one of the founders of Erlang, really likes languages and Lisp in particular so he wrote one for the BEAM. It just has some special accommodations to be able to send/receive (and I think pattern recognition too).
- lisper 4y ago
  That actually sounds pretty cool as a runtime environment. Is the VM programmed in byte code, or is it native code? What back ends are available? What is the runtime written in?
  - GenericJam 4y ago
    The BEAM (runtime?) is written in C. There is also an effort to rewrite it in Rust (https://github.com/lumen/lumen https://github.com/lumen/lumen). Some functions are built into the VM but most of the supporting 'standard library' (OTP / Open Telecom Platform) is written in Erlang. The (main) compiler is written in C. So it's all C or Erlang afaik. It is ported to every major flavour of OS. I don't know what 'back end' means in this context.
    - lisper 4y ago
      You can compile a high-level language down to a byte code which looks like machine code but which does not correspond to any actual hardware. Instead, the byte code is interpreted. Python and Java both work this way. You can also compile a high level language down to machine code that runs on actual hardware, like an x86 or an ARM. For languages that run on more than on processor, the compilation process usually consists of an architecture-independent phase which produces some sort of intermediate representation (which may be a byte code, or it might be something else, like LLVM) and then an architecture-specific pass that transforms the intermediate representation into machine code for the target architecture. The code that implements that second pass is called the "back end." I have no idea whether Erlang compiles to native code or byte code. In addition to all that, there can also be a run-time environment that is required to run the resulting code. For byte code, this environment necessarily includes an interpreter for the byte code, and might also include other things. For native code this environment might include things like a garbage collector or a standard library that provides an interface to an operating system or something like that.
      - GenericJam 4y ago
GenericJam 4y ago
Another thing about the BEAM that may resonate for you is the abstract syntax tree (AST) for the BEAM is very reminiscent of Lisp. Some people call Elixir which is a popular reworking of Erlang which also runs on the BEAM a secret Lisp as it gives direct access to the AST via macros. A friend wrote on the topic here: https://dorgan.netlify.app/posts/2021/04/the_elixir_ast/ https://dorgan.netlify.app/posts/2021/04/the_elixir_ast/
abecedarius 4y ago
Hi Ron, I'd recommend Joe Armstrong's thesis https://erlang.org/download/armstrong_thesis_2003.pdf https://erlang.org/download/armstrong_thesis_2003.pdf to get an idea what's special about Erlang. Briefly, it's designed around keeping systems running live even when they have bugs and other faults. Straight Lisp with single-dispatch wouldn't isolate effects, though of course you could build something in Lisp that did. Initially Erlang was built on top of Prolog. (This is Darius. I never really learned your system with related goals for robots -- as you know, I didn't stick around at JPL. About 10 years later I did a bit of Erlang for Yahoo.)
- lisper 4y ago
  Hi Darius! :-)