2 ms·

I didn't realize that Kernel ends up being necessarily very slow. Can you give a quick explanation of why?

reitzensteinm 6y ago
Because everything is a fexpr, with lambda built on top of it, when you call a function you're not passing it values, you're passing it code and a first class environment to use to execute it. Further, because macro like features are first class, they're executed each time they're used as opposed to generating code. I believe there are decent implementations but I bet they put a fair amount of work in to making them fast. The naive reference implementation is extremely slow. While my toy project is a bit different and probably even more wasteful (with a list calling convention and immutability), it faces similar challenges. Take a read through core.clj and try to spot where the 88 million function calls are spent.
- mst 6y ago
  Yah, my kernel-like thing ... oh gods the number of reductions. lambda (wrap) basically has to be VM provided code, and that's just the start. But for sheer flexibility it's beautiful and I don't feel like giving up yet :)
- sparkie 6y ago
  klisp is quite efficient (though still slow compared to other Lisps). Bronze-age-lisp optimizes some of the most critical parts in assembly language. This is probably the fastest Kernel you will find. Some details on the design were published in 4 posts to this group: https://groups.google.com/g/klisp https://groups.google.com/g/klisp
mst 6y ago
A naive implementation has to do a ridiculous number of scope lookups, even before you get to anything else. Also instead of compile time macros, one has runtime fexprs, which means the equivalent of macro expansion happens every invocation. However, one can have kernel style semantics with at least moderately reasonable performance by basically applying a combination of constant folding and expansion caching. It's a bunch of extra effort though, and non-trivial to make work, though I'm currently experimenting myself with something where I'm willing to do that. A key part is to be able to recognise fexprs that are 'really' just macros and pre-expand them and cache the expansion - without at least that you're screwed. I'm also experimenting with what else I can cache given the principle that everything is a lexical environment. I'm rather enjoying myself ... but understand the risks.

Because everything is a fexpr, with lambda built on top of it, when you call a function you're not passing it values, you're passing it code and a first class environment to use to execute it. Further, because macro like features are first class, they're executed each time they're used as opposed to generating code. I believe there are decent implementations but I bet they put a fair amount of work in to making them fast. The naive reference implementation is extremely slow. While my toy project is a bit different and probably even more wasteful (with a list calling convention and immutability), it faces similar challenges. Take a read through core.clj and try to spot where the 88 million function calls are spent.

mst 6y ago
Yah, my kernel-like thing ... oh gods the number of reductions. lambda (wrap) basically has to be VM provided code, and that's just the start. But for sheer flexibility it's beautiful and I don't feel like giving up yet :)
sparkie 6y ago
klisp is quite efficient (though still slow compared to other Lisps). Bronze-age-lisp optimizes some of the most critical parts in assembly language. This is probably the fastest Kernel you will find. Some details on the design were published in 4 posts to this group: https://groups.google.com/g/klisp https://groups.google.com/g/klisp

mst 6y ago

A naive implementation has to do a ridiculous number of scope lookups, even before you get to anything else. Also instead of compile time macros, one has runtime fexprs, which means the equivalent of macro expansion happens every invocation. However, one can have kernel style semantics with at least moderately reasonable performance by basically applying a combination of constant folding and expansion caching. It's a bunch of extra effort though, and non-trivial to make work, though I'm currently experimenting myself with something where I'm willing to do that. A key part is to be able to recognise fexprs that are 'really' just macros and pre-expand them and cache the expansion - without at least that you're screwed. I'm also experimenting with what else I can cache given the principle that everything is a lexical environment. I'm rather enjoying myself ... but understand the risks.