5 ms·

Show HN: A tiny WASM compiler and runtime for demonstration purposes

48 points by thomscoder 4y ago

Hello, I'm building a WebAssembly compiler and documenting the process in the meantime. This aims to be a tool for demonstration and education purpose. It doesn't aim to do fancy stuff nor to replace existing ones. I'm building this to learn and to improve and hopefully it can be useful for others as well :) From last time I've presented the idea, I've implemented more basic features on Luna and I've built a custom and tiny runtime for Luna! (documenting it of course). Any feedback is really appreciated :) Repo: https://github.com/thomscoder/luna https://github.com/thomscoder/luna

deleted 4y ago
[deleted]
droobles 4y ago
This is awesome! I too have wanted to conquer the WASM dungeon but haven't dedicated the time myself to approach the project. I'm eager to look through this repo and learn a thing or two from it.
- thomscoder 4y ago
  Thank you very much! :) I'm also writing blog articles. (i have to decide if doing a series or a one for the compiler and one for the runtime)
chubs 4y ago
Thanks for showing this! Today I learned that wasm has a text format thanks to you :)
- thomscoder 4y ago
  thank you :) happy to be useful!
haberman 4y ago
I'm curious how small a conforming WASM implementation can be (Luna looks extremely small, but not conforming). In particular I'm curious how feasible it is to create WASM polyfills for various languages. Imagine you have a nice C kernel that you want to wrap in various languages. In many cases you can use native extensions to wrap your C kernel through an FFI, and on the web you can use WASM in supported browsers. But what about cases where people can't use your native extension? Suppose they are using a browser that doesn't support WASM. It's nice to be able to offer a polyfill that at least works, even if it is slower. The feasibility of this idea hinges on how easily you can write a conforming implementation of WASM in a bunch of different languages. But I don't know the WASM spec well enough to know how big of an undertaking this is.
- thomscoder 4y ago
  helloo! what a nice problem you've brought up. I honestly do not know...yet. I'm as well studying the WASM spec as I'm making Luna (it still misses concepts like linear memory for example). Have not tackled the polyfills yet, I'm sorry. Anyway for the specs: https://webassembly.github.io/spec/core/intro/index.html https://webassembly.github.io/spec/core/intro/index.html this is where I get my infos from
ushakov 4y ago
I'm upset WASM doesn't have any types other than numbers, so you can't export any real-world functions without relying on glue code, which converts strings and objects to numbers and the other way around this is also the reason why the "hello world" of WASM is a function adding two numbers - because there are no strings in WASM https://webassembly.github.io/spec/core/syntax/types.html https://webassembly.github.io/spec/core/syntax/types.html
- nitsky 4y ago
  Check out the component model proposal, which adds a stable ABI for rich types like strings, arrays, structs, etc. https://github.com/WebAssembly/component-model https://github.com/WebAssembly/component-model
  - josephg 4y ago
    Whats the status on this? How long before the component model is something we can use?
  - nuc1e0n 4y ago
    Not a great proposal IMHO and it misses the point of wasm in general. Strings and other high level data structures are distinctions the high level languages that output wasm should make. Anyone who's had to do java to .net clr interop will say how problematic it was. The only resonable way to do it was via text de/serialisation on the database or via a web based api, even on the same computer. That's crazy when you think about it. So arrays of numbers will work and have predictable performance from one wasm runtime to another. Personally, I'd prefer to see wasm not repeat the same disasterous mistakes Java made.
- nuc1e0n 4y ago
  Strings are only an array of numbers (codepoints). Wasm is intended as a compiler target that can be easily implemented with good performance. The text format is not a high level language on purpose. The proper term for this tool is not a compiler but an assembler because the text format is a 1-to-1 mapping of the binary output.
gnabgib 4y ago
You only just posted this (12 days ago, 75 points, 2 comments[0]), the vercel app seems to look the same, and still references the repo for instructions. This seems way to soon to be making a duplicate post. [0]: https://news.ycombinator.com/item?id=33424354 https://news.ycombinator.com/item?id=33424354
- deleted 4y ago
  [deleted]
- pvg 4y ago
  They're only HN dupes if they generate significant discussion and this one hadn't.
kasajian 4y ago
I think it's interesting the amount of justification given for why this is being built. Twice in the same paragraph. And no doubt there will be hundreds more wondering why do create something that already exists. Software developers are too fixated on pragmatism. If you're writing code, the resulting code better be something useful OR ELSE. Do you ever ask someone why they're doing a crossword puzzle? Playing a video game? Trying to get of an Escape Room? No. But if someone say writes their own Lisp as an exercise, oh the vitriol from the pragmatists. I've personally stopped answering that question.
- thomscoder 4y ago
  hmm :( i'm sorry if I bored you with explanation. it was not my aim
  - urbandw311er 4y ago
    My reading of the parent comment is that it is actually sympathetic to you. I think it’s saying you shouldn’t have to feel you need to justify building this —- it’s a great, worthy project just to do something to see if it’s within your capability.
    - deleted 4y ago
      [deleted]
    - thomscoder 4y ago
      Oh, ops. -.- thank you guys!
ethanwillis 4y ago
Please take these as suggestions and not criticism :) https://github.com/thomscoder/luna/blob/009b20c9afd88d0a6572838c4ba2b5ec879be21b/runtime/runtime/start.js#L9 https://github.com/thomscoder/luna/blob/009b20c9afd88d0a6572... The list splitting here is currently un-necessary. Your example for users is const result = startAeonRuntime(wasmBinary, "addNumbers", n1, n2, n3); So you can just do something like const startAeonRuntime(wasm, funcName, ...params) => { const ast = createAST(wasm); return invokeFunction(ast, funcName, ...params); } However, this leaves me with questions. Why should we need to create/parse the AST each time we want to invoke a function? A wasmBinary can provide multiple functions. Creating the AST each time seems extremely wasteful, but also a bit confusing since I don't think it's necessary. My personal opinion is the function "startAeonRuntime" should be more something like "createAeonRuntime" that returns a partially parameterized version of invokeFunction that I can re-use over and over without it needing to create an AST each time. So something like In user code I have const runtime = createAeonRuntime(wasmBinary); const additionResult = runtime("addNumbers", 1, 2, 3); const multiplicationResult = runtime("multiplyNumbers, 2, 4, 6); Then in the library for the runtime the implementation looks like. const createAeonRuntime = (wasmBinary) => { const ast = createAST(wasmBinary); return (funcName, params) => invokeFunction(ast, funcName, params); }
- thomscoder 4y ago
  Very nice suggestion. Yes looks like a blunder ahah Nice optimization, I'll implement it :) (or if you want to open a PR feel free to do it).

48 points by thomscoder 4y ago

deleted 4y ago

[deleted]

droobles 4y ago

This is awesome! I too have wanted to conquer the WASM dungeon but haven't dedicated the time myself to approach the project. I'm eager to look through this repo and learn a thing or two from it.

thomscoder 4y ago
Thank you very much! :) I'm also writing blog articles. (i have to decide if doing a series or a one for the compiler and one for the runtime)

chubs 4y ago

Thanks for showing this! Today I learned that wasm has a text format thanks to you :)

thomscoder 4y ago
thank you :) happy to be useful!

haberman 4y ago

I'm curious how small a conforming WASM implementation can be (Luna looks extremely small, but not conforming). In particular I'm curious how feasible it is to create WASM polyfills for various languages. Imagine you have a nice C kernel that you want to wrap in various languages. In many cases you can use native extensions to wrap your C kernel through an FFI, and on the web you can use WASM in supported browsers. But what about cases where people can't use your native extension? Suppose they are using a browser that doesn't support WASM. It's nice to be able to offer a polyfill that at least works, even if it is slower. The feasibility of this idea hinges on how easily you can write a conforming implementation of WASM in a bunch of different languages. But I don't know the WASM spec well enough to know how big of an undertaking this is.

thomscoder 4y ago
helloo! what a nice problem you've brought up. I honestly do not know...yet. I'm as well studying the WASM spec as I'm making Luna (it still misses concepts like linear memory for example). Have not tackled the polyfills yet, I'm sorry. Anyway for the specs: https://webassembly.github.io/spec/core/intro/index.html https://webassembly.github.io/spec/core/intro/index.html this is where I get my infos from

ushakov 4y ago

I'm upset WASM doesn't have any types other than numbers, so you can't export any real-world functions without relying on glue code, which converts strings and objects to numbers and the other way around this is also the reason why the "hello world" of WASM is a function adding two numbers - because there are no strings in WASM https://webassembly.github.io/spec/core/syntax/types.html https://webassembly.github.io/spec/core/syntax/types.html

nitsky 4y ago
Check out the component model proposal, which adds a stable ABI for rich types like strings, arrays, structs, etc. https://github.com/WebAssembly/component-model https://github.com/WebAssembly/component-model
- josephg 4y ago
  Whats the status on this? How long before the component model is something we can use?
- nuc1e0n 4y ago
  Not a great proposal IMHO and it misses the point of wasm in general. Strings and other high level data structures are distinctions the high level languages that output wasm should make. Anyone who's had to do java to .net clr interop will say how problematic it was. The only resonable way to do it was via text de/serialisation on the database or via a web based api, even on the same computer. That's crazy when you think about it. So arrays of numbers will work and have predictable performance from one wasm runtime to another. Personally, I'd prefer to see wasm not repeat the same disasterous mistakes Java made.
nuc1e0n 4y ago
Strings are only an array of numbers (codepoints). Wasm is intended as a compiler target that can be easily implemented with good performance. The text format is not a high level language on purpose. The proper term for this tool is not a compiler but an assembler because the text format is a 1-to-1 mapping of the binary output.

gnabgib 4y ago

You only just posted this (12 days ago, 75 points, 2 comments[0]), the vercel app seems to look the same, and still references the repo for instructions. This seems way to soon to be making a duplicate post. [0]: https://news.ycombinator.com/item?id=33424354 https://news.ycombinator.com/item?id=33424354

deleted 4y ago
[deleted]
pvg 4y ago
They're only HN dupes if they generate significant discussion and this one hadn't.

kasajian 4y ago

I think it's interesting the amount of justification given for why this is being built. Twice in the same paragraph. And no doubt there will be hundreds more wondering why do create something that already exists. Software developers are too fixated on pragmatism. If you're writing code, the resulting code better be something useful OR ELSE. Do you ever ask someone why they're doing a crossword puzzle? Playing a video game? Trying to get of an Escape Room? No. But if someone say writes their own Lisp as an exercise, oh the vitriol from the pragmatists. I've personally stopped answering that question.

thomscoder 4y ago
hmm :( i'm sorry if I bored you with explanation. it was not my aim
- urbandw311er 4y ago
  My reading of the parent comment is that it is actually sympathetic to you. I think it’s saying you shouldn’t have to feel you need to justify building this —- it’s a great, worthy project just to do something to see if it’s within your capability.
  - deleted 4y ago
    [deleted]
  - thomscoder 4y ago
    Oh, ops. -.- thank you guys!

ethanwillis 4y ago

Please take these as suggestions and not criticism :) https://github.com/thomscoder/luna/blob/009b20c9afd88d0a6572838c4ba2b5ec879be21b/runtime/runtime/start.js#L9 https://github.com/thomscoder/luna/blob/009b20c9afd88d0a6572... The list splitting here is currently un-necessary. Your example for users is const result = startAeonRuntime(wasmBinary, "addNumbers", n1, n2, n3); So you can just do something like const startAeonRuntime(wasm, funcName, ...params) => { const ast = createAST(wasm); return invokeFunction(ast, funcName, ...params); } However, this leaves me with questions. Why should we need to create/parse the AST each time we want to invoke a function? A wasmBinary can provide multiple functions. Creating the AST each time seems extremely wasteful, but also a bit confusing since I don't think it's necessary. My personal opinion is the function "startAeonRuntime" should be more something like "createAeonRuntime" that returns a partially parameterized version of invokeFunction that I can re-use over and over without it needing to create an AST each time. So something like In user code I have const runtime = createAeonRuntime(wasmBinary); const additionResult = runtime("addNumbers", 1, 2, 3); const multiplicationResult = runtime("multiplyNumbers, 2, 4, 6); Then in the library for the runtime the implementation looks like. const createAeonRuntime = (wasmBinary) => { const ast = createAST(wasmBinary); return (funcName, params) => invokeFunction(ast, funcName, params); }

thomscoder 4y ago
Very nice suggestion. Yes looks like a blunder ahah Nice optimization, I'll implement it :) (or if you want to open a PR feel free to do it).