8 ms·

Call for support for Lisp in WebAssembly development

223 points by patrickmay 11y ago

lisper 11y ago
> some support from other people in the lisp community seems necessary. A clearer call to action would be helpful here. What exactly should members of the Lisp community who care about this do?
- pjlegato 11y ago
  Agree. I support having Lisp support in WebAssembly! Now what?
  - vmorgulis 11y ago
    My knowledge of LLVM an SBCL is limited but I know a bit emscripten and how it works. I will look around "multivalues" and "power of two memory access" in LLVM.
- zimbatm 11y ago
  Actually getting the support of other language communities would be even more useful. If you can find use-cases of the feature for other languages then it's going to be much more compelling.
klodolph 11y ago
From digging, it looks like the issue here (or one of the issues) is that the Web Assembly encoding for some Lisp uses cases (multiple value return) is not very compact. Making the proposed change would presumably reduce the size of Lisp code once compiled to Web Assembly, it would not really affect the behavior of the program once compiled to native code. I can relate to the Web Assembly team's reluctance to add features which are only really wanted by a small subset of users, especially when they only affect the binary size. These features, if implemented, may suffer from poor test coverage. My own preference is that compact binaries are nice but if you're going to use a high-level language, an increase in binary size is just expected (say, an order of magnitude) unless the encoding/VM and language were designed in concert (Java + JVM, or C# + CIL are two examples). Heck, C++ binaries can be enormous. Then again, I didn't dig deep enough to really understand the nuances of the argument. Perhaps someone could elaborate.
- kuschku 11y ago
  Then again, if the WebAssembly team makes that argument, we might as well just use JavaScript + ASM.js
  - rubber_duck 11y ago
    Not really, aside from smaller binary size parsing a binary bytecode vs parsinga a JS pseudo bytecode is a huge difference in load time + they get to optimize it for efficient translation vs trying to hack JS constructs to get desired semantics and then trying to recognize those in the JIT Asm.js is a hack made with ducktape and chewing gum, wasm is a solution designed/engineered to solve the general problem - not just smaller asm.js
  - dietrichepp 11y ago
    ASM.js builds can be quite large, even 10s of MB is not uncommon. Reducing the binary size isn't just "nice to have", it changes the viability of the platform.
    - kuschku 11y ago
      But wasn’t the argument of the wasm team against the LISP features that binary size isn’t relevant and just a "nice to have" feature?
      - dietrichepp 11y ago
        I don't think anyone is arguing that binary size isn't relevant. It just has to be weighed against the other parameters we want to optimize, like implementation complexity.
- vmorgulis 11y ago
  The target prevails over the language. The implicit intent is to convert LLVM bitcode. "Please keep in mind that Wasm is not a user-facing language, it is a compilation target. To justify the extra complexity of this feature for its own sake, you would need to come up with convincing evidence that compilers would significantly benefit from it. I doubt that." https://github.com/WebAssembly/spec/pull/215#issuecomment-176836966 https://github.com/WebAssembly/spec/pull/215#issuecomment-17...
vmorgulis 11y ago
The misunderstandings are related to the use of the word "AST". wasm looks like an AST but in fact it's a bytecode with stackframes.
Sanddancer 11y ago
The memory allocation feature feels more like it would be part of whatever memory allocation library is used than something that should be baked into the language. For example, jemalloc allows for the kind of alignment that is discussed here and is done at runtime, and doesn't require specific behavior from the lower levels. Any language is going to need a runtime because you can't put in every feature that every user will need, and a malloc doesn't seem like a huge issue, especially with one already written that an implementation can crib from.
- nabla9 11y ago
  Memory allocation feature asked is not a language feature. It has to be baked into web assembly if you want to use portable byte masking with pointers. It's very low level implementation level detail that enables fast execution of dynamically typed languages. Boxing has high cost and it consumes memory. Type tags embedded in pointers can be very fast. For that to work, you need objects that are aligned with power-of-two byte boundaries. Adding this feature enables efficient execution strategy for scripting and dynamic languages.
  - Sanddancer 11y ago
    The primary target of WebAssembly is strongly-typed pre-compiled languages, where the kinds of features you want would just lead to slowdowns and excessive memory consumption. There is no hardware currently out there that is a tagged architecture, so expecting them to bend backwards is not a realistic option.
    - nabla9 11y ago
      You don't need tagged architecture if you allocate memory by power-of-two regions.
dschiptsov 11y ago
Code generating backend for SBCL, like one in LLVM?
dschiptsov 11y ago
BTW, language implementations which rely on LLVM for code generation would get it for free. Well, for much less pain. BTW2, time to appreciate how LLVM's approach is superior to JVM madness/religion (and how Golang's is even more clever - do less by doing it right - essentiaistic/minimalistic ascetism)
- dgellow 11y ago
  Could you elaborate on you BTW2?
  - dschiptsov 11y ago
    Modern CPU+OS is a good-enough hardware VM and a target platform. Process isolation under an OS is a right level of abstraction. A VM as user-level OS process which tries to do an OS job and reimplement everything inside a VM is simply ridiculous. Javascript follows the same madeness. Multi-threading for imperative code is a big mistake, which breaks isolation and results in lock-hell, context-switching nightmare and layers of unnecessary complexity which is impossible to reason about. Golang and Swift guys got it.
    - xaduha 11y ago
      How would you explain that? https://www.techempower.com/benchmarks/#section=data-r11&hw=peak&test=json https://www.techempower.com/benchmarks/#section=data-r11&hw=... Swift is nowhere to be seen and Go is nowhere near the top. Once it gets going JVM is a beast.
      - dschiptsov 11y ago
        At cost of wasting of almost as much resources that it serves. Top is about popularity, not quality. Junk-food is also popular. My analysis was about the first principles, not abstract ones, but grounded in reality. Those who got the principles right wins in the long run. Erlang (where VM is not a byte-code interpreter), Golang, Haskell (except when monads are abused by idiots), etc are designs based on the right principles. Java was a primitive religion based on superstitions (the fear of pointers) from the start.
pmarreck 11y ago
I support having the Erlang BEAM in WebAssembly! Any takers?
CyberDildonics 11y ago
webasm is a strongly typed AST with manual memory management, it is not meant to be a direct analog to lisp or a lisp interpreter.
zaro 11y ago
"This masking strategy would in turn require a power-of-two related memory size, and there has been a lot of resistance to this too." A more appropriate title will be "WebAssembly team doesn't want to listen my ideas on how WebAssembly should work".
- junke 11y ago
  From https://www.w3.org/community/webassembly https://www.w3.org/community/webassembly : > The mission of this group is to promote early-stage cross-browser collaboration on a new, portable, size- and load-time-efficient format suitable for compilation to the web. See also https://www.w3.org/community/council/wiki/Templates/CG_Charter#Decision_Process https://www.w3.org/community/council/wiki/Templates/CG_Chart.... This is a collaborative work where people can make suggestions (I cannot judge if the proposal was fairly evaluated or not). > WebAssembly team doesn't want to listen my ideas on how WebAssembly should work. Your title implies that the WebAssembly team has the best knowledge and/or expertise to develop WebAssembly. They are probably expert in their own domain but are willing to take advice from other contributors.
  - zaro 11y ago
    Please read this quote again: "This masking strategy would in turn require a power-of-two related memory size, and there has been a lot of resistance to this too." And try to think about the implications it has on the memory model of the VM that is going to execute/JIT Webassemly. A power of two memory model, isn't really viable at this level I think. And you don't need to think a lot about it, to figure out that jumping to 256MB memory, just because your app/page needs 130MB is a bit of counter-optimization :) The resistance is the sensible thing to do in this case :)
finchisko 11y ago
So what is this webassembly about? Allowing programming for web in any language (java, c, lisp) and compile to webasm, as some kind of runtime env?
- _alexander_ 11y ago
  Actually, anybody does not know what is it ;) but many paople think that webassembly will kill JS.
- qwertyuiop924 11y ago
  Basically. At least, that's the theory.
sandra_saltlake 11y ago
I don't expect it to use the hardware in a sensible manner.

223 points by patrickmay 11y ago

> some support from other people in the lisp community seems necessary. A clearer call to action would be helpful here. What exactly should members of the Lisp community who care about this do?

pjlegato 11y ago
Agree. I support having Lisp support in WebAssembly! Now what?
- vmorgulis 11y ago
  My knowledge of LLVM an SBCL is limited but I know a bit emscripten and how it works. I will look around "multivalues" and "power of two memory access" in LLVM.
zimbatm 11y ago
Actually getting the support of other language communities would be even more useful. If you can find use-cases of the feature for other languages then it's going to be much more compelling.

klodolph 11y ago

From digging, it looks like the issue here (or one of the issues) is that the Web Assembly encoding for some Lisp uses cases (multiple value return) is not very compact. Making the proposed change would presumably reduce the size of Lisp code once compiled to Web Assembly, it would not really affect the behavior of the program once compiled to native code. I can relate to the Web Assembly team's reluctance to add features which are only really wanted by a small subset of users, especially when they only affect the binary size. These features, if implemented, may suffer from poor test coverage. My own preference is that compact binaries are nice but if you're going to use a high-level language, an increase in binary size is just expected (say, an order of magnitude) unless the encoding/VM and language were designed in concert (Java + JVM, or C# + CIL are two examples). Heck, C++ binaries can be enormous. Then again, I didn't dig deep enough to really understand the nuances of the argument. Perhaps someone could elaborate.

kuschku 11y ago
Then again, if the WebAssembly team makes that argument, we might as well just use JavaScript + ASM.js
- rubber_duck 11y ago
  Not really, aside from smaller binary size parsing a binary bytecode vs parsinga a JS pseudo bytecode is a huge difference in load time + they get to optimize it for efficient translation vs trying to hack JS constructs to get desired semantics and then trying to recognize those in the JIT Asm.js is a hack made with ducktape and chewing gum, wasm is a solution designed/engineered to solve the general problem - not just smaller asm.js
- dietrichepp 11y ago
  ASM.js builds can be quite large, even 10s of MB is not uncommon. Reducing the binary size isn't just "nice to have", it changes the viability of the platform.
  - kuschku 11y ago
    But wasn’t the argument of the wasm team against the LISP features that binary size isn’t relevant and just a "nice to have" feature?
    - dietrichepp 11y ago
      I don't think anyone is arguing that binary size isn't relevant. It just has to be weighed against the other parameters we want to optimize, like implementation complexity.
vmorgulis 11y ago
The target prevails over the language. The implicit intent is to convert LLVM bitcode. "Please keep in mind that Wasm is not a user-facing language, it is a compilation target. To justify the extra complexity of this feature for its own sake, you would need to come up with convincing evidence that compilers would significantly benefit from it. I doubt that." https://github.com/WebAssembly/spec/pull/215#issuecomment-176836966 https://github.com/WebAssembly/spec/pull/215#issuecomment-17...

vmorgulis 11y ago

The misunderstandings are related to the use of the word "AST". wasm looks like an AST but in fact it's a bytecode with stackframes.

Sanddancer 11y ago

The memory allocation feature feels more like it would be part of whatever memory allocation library is used than something that should be baked into the language. For example, jemalloc allows for the kind of alignment that is discussed here and is done at runtime, and doesn't require specific behavior from the lower levels. Any language is going to need a runtime because you can't put in every feature that every user will need, and a malloc doesn't seem like a huge issue, especially with one already written that an implementation can crib from.

nabla9 11y ago
Memory allocation feature asked is not a language feature. It has to be baked into web assembly if you want to use portable byte masking with pointers. It's very low level implementation level detail that enables fast execution of dynamically typed languages. Boxing has high cost and it consumes memory. Type tags embedded in pointers can be very fast. For that to work, you need objects that are aligned with power-of-two byte boundaries. Adding this feature enables efficient execution strategy for scripting and dynamic languages.
- Sanddancer 11y ago
  The primary target of WebAssembly is strongly-typed pre-compiled languages, where the kinds of features you want would just lead to slowdowns and excessive memory consumption. There is no hardware currently out there that is a tagged architecture, so expecting them to bend backwards is not a realistic option.
  - nabla9 11y ago
    You don't need tagged architecture if you allocate memory by power-of-two regions.

dschiptsov 11y ago

Code generating backend for SBCL, like one in LLVM?

dschiptsov 11y ago

BTW, language implementations which rely on LLVM for code generation would get it for free. Well, for much less pain. BTW2, time to appreciate how LLVM's approach is superior to JVM madness/religion (and how Golang's is even more clever - do less by doing it right - essentiaistic/minimalistic ascetism)

dgellow 11y ago
Could you elaborate on you BTW2?
- dschiptsov 11y ago
  Modern CPU+OS is a good-enough hardware VM and a target platform. Process isolation under an OS is a right level of abstraction. A VM as user-level OS process which tries to do an OS job and reimplement everything inside a VM is simply ridiculous. Javascript follows the same madeness. Multi-threading for imperative code is a big mistake, which breaks isolation and results in lock-hell, context-switching nightmare and layers of unnecessary complexity which is impossible to reason about. Golang and Swift guys got it.
  - xaduha 11y ago
    How would you explain that? https://www.techempower.com/benchmarks/#section=data-r11&hw=peak&test=json https://www.techempower.com/benchmarks/#section=data-r11&hw=... Swift is nowhere to be seen and Go is nowhere near the top. Once it gets going JVM is a beast.
    - dschiptsov 11y ago
      At cost of wasting of almost as much resources that it serves. Top is about popularity, not quality. Junk-food is also popular. My analysis was about the first principles, not abstract ones, but grounded in reality. Those who got the principles right wins in the long run. Erlang (where VM is not a byte-code interpreter), Golang, Haskell (except when monads are abused by idiots), etc are designs based on the right principles. Java was a primitive religion based on superstitions (the fear of pointers) from the start.

pmarreck 11y ago

I support having the Erlang BEAM in WebAssembly! Any takers?

CyberDildonics 11y ago

webasm is a strongly typed AST with manual memory management, it is not meant to be a direct analog to lisp or a lisp interpreter.

zaro 11y ago

"This masking strategy would in turn require a power-of-two related memory size, and there has been a lot of resistance to this too." A more appropriate title will be "WebAssembly team doesn't want to listen my ideas on how WebAssembly should work".

junke 11y ago
From https://www.w3.org/community/webassembly https://www.w3.org/community/webassembly : > The mission of this group is to promote early-stage cross-browser collaboration on a new, portable, size- and load-time-efficient format suitable for compilation to the web. See also https://www.w3.org/community/council/wiki/Templates/CG_Charter#Decision_Process https://www.w3.org/community/council/wiki/Templates/CG_Chart.... This is a collaborative work where people can make suggestions (I cannot judge if the proposal was fairly evaluated or not). > WebAssembly team doesn't want to listen my ideas on how WebAssembly should work. Your title implies that the WebAssembly team has the best knowledge and/or expertise to develop WebAssembly. They are probably expert in their own domain but are willing to take advice from other contributors.
- zaro 11y ago
  Please read this quote again: "This masking strategy would in turn require a power-of-two related memory size, and there has been a lot of resistance to this too." And try to think about the implications it has on the memory model of the VM that is going to execute/JIT Webassemly. A power of two memory model, isn't really viable at this level I think. And you don't need to think a lot about it, to figure out that jumping to 256MB memory, just because your app/page needs 130MB is a bit of counter-optimization :) The resistance is the sensible thing to do in this case :)

finchisko 11y ago

So what is this webassembly about? Allowing programming for web in any language (java, c, lisp) and compile to webasm, as some kind of runtime env?

_alexander_ 11y ago
Actually, anybody does not know what is it ;) but many paople think that webassembly will kill JS.
qwertyuiop924 11y ago
Basically. At least, that's the theory.

sandra_saltlake 11y ago

I don't expect it to use the hardware in a sensible manner.