9 ms·

Compiling Rust to Web Assembly

289 points by grey-area 10y ago

adamnemecek 10y ago
Three rust articles on the front page? Not complaining, it definitely seems like the popularity has really jumped up in the last three months.
- hacker_9 10y ago
  Rust and Go are always on the front page.
  - rch 10y ago
    Also Python, JS frameworks, CLR and JVM languages, C/C++, LLVM, notably obscure languages... It's like there's some kind of theme.
  - lacampbell 10y ago
    If someone makes a Go compiler in Rust hacker news will have to shut down. Game over, there is nothing left to do in computing.
    - flamedoge 10y ago
      Rust compiler written in Go?
    - slowmovintarget 10y ago
      Clojure-on-Rust Granted, it would require writing a garbage collector in Rust...
      - saghm 10y ago
        > Granted, it would require writing a garbage collector in Rust... Why would that be a problem? There are compilers written in C that implement garbage collectors. Now, writing a garbage collector for Rust, that's another problem entirely
        Manishearth 10y ago
        > garbage collector in Rust.. http://users.cecs.anu.edu.au/~steveb/downloads/pdf/rust-ismm-2016.pdf http://users.cecs.anu.edu.au/~steveb/downloads/pdf/rust-ismm... > Now, writing a garbage collector for Rust, http://manishearth.github.io/blog/2015/09/01/designing-a-gc-in-rust/ http://manishearth.github.io/blog/2015/09/01/designing-a-gc-... (mostly a toy GC, but some folks are using it) In the future we may have http://manishearth.github.io/blog/2016/08/18/gc-support-in-rust-api-design/ http://manishearth.github.io/blog/2016/08/18/gc-support-in-r...
        saghm 10y ago
        During my senior year of college (which ended last spring), I worked with two other students on implementing a server-side JavaScript interpreter in Rust. One of us, a grad student (not me!), wrote the garbage collector for it (https://github.com/rustoscript/french-press https://github.com/rustoscript/french-press). We didn't end up finishing the entire language, but we got a sizable chunk of the language features implemented, and more than enough to know that Rust is very well suited to writing programming languages in, including garbage collectors. By far, the most complicated component of getting the garbage collector right was dealing with closures, but that's mostly because JavaScript is a very "unique" language, to put it nicely. > In the future we may have http://manishearth.github.io/blog/2016/08/18/gc-support-in-rust-api-design/ http://manishearth.github.io/blog/2016/08/18/gc-support-in-r... I read that back when you first posted it, which is exactly what came to mind when I was writing my comment. At least to my naive understanding, implementing a garbage collector for Rust is several orders of magnitude more difficult than merely implementing one in Rust.
        Manishearth 10y ago
        The JS interpreter is pretty cool! We've had problems with closures before in Servo, I know that pain :) > implementing a garbage collector for Rust is several orders of magnitude more difficult than merely implementing one in Rust. Well, I mean it depends on what you're aiming for. Making a safe optional GC in Rust isn't too hard; rust-gc exists and is not very complicated. Making the compiler support arbitrary pluggable GCs is trickier. GC algorithms themselves can get so complicated that the state of the art research probably eclipses whatever has been done with rust-gc and similar "GCs for rust" so far. If you want a GC for Rust that contains a world-class collection algorithm the complexity is additive but the source of most of the complexity isn't the "for Rust" part, it's the "in Rust" part (A GC for rust will also typically be in Rust). But also "optional" GCs like those that would be useful in Rust haven't been explored much either, so world class algorithms may not even make sense in this case. It's ... an interesting set of problems, and very hard to make judgement about what is difficult :)
      - lenkite 10y ago
        There is a lisp dialect on rust https://github.com/murarth/ketos https://github.com/murarth/ketos and it doesn't need GC!
    - solidsnack9000 10y ago
      https://github.com/yberreby/rgo https://github.com/yberreby/rgo "A Go compiler, written in Rust."
- swsieber 10y ago
  As far as I know it's because the Rust is filling in the gaps - two of the three articles are about relatively new features. And the next Rust release should close the stable/unstable gap by stabilizing some macro/compiler plugin stuff.
  - pekk 10y ago
    It is a marketing push. Marketing pushes don't have to correlate to massive improvements in the product.
    - Manishearth 10y ago
      Certainly not a coordinated marketing push. All of the recent Rust posts on HN don't come from any official venues. It's just random community members writing about Rust, and HN upvoting it. There was a flurry of Rust posts about "why Rust" near the end of the year, which was somewhat "coordinated". In the sense that Steve wrote a post and this triggered a chain reaction where all kinds of folks wrote about related topics. Completely unintentional, just happened. Most of those posts did not make it to HN.
    - coldtea 10y ago
      It's just random people discovering Rust due to increased popularity and posting about it (which has a circular effect). No marketing plan involved, and nobody coordinates these posts.
- lacampbell 10y ago
  TBH it's getting a bit much at this point. The rust advocates here (not sure if they're paid or just fanboys) really need to tone it down. The hyperbole, and the constant "rust solves this unrelated problem, and has no flaws!" is beginning to grate. I've coded in it a bit and it seems like cool language, they get a lot right and have some new and interesting ideas. But it's not a panacea (or indeed "salvation" as one recent blog post put it) for everything wrong in software. We don't need to all immediately re-write everything we do rust. Not everything new needs to be written in rust. And your re-writes of programs that have already been written in other languages and work well aren't that compelling. If you like rust, then write some new, interesting and useful software in the language, where what language it's written in isn't the main selling point. For example, I love the look of this http://nickel.rs/ http://nickel.rs/. TL;DR - the corrosive nature of rust advocates on HN is starting to eat away at my patience, and I am tired of having to galvanize against them and their absurd claims.
  - burntsushi 10y ago
    Yes, we had a recent explosion of marketing materials that went a little over the top. That's OK. It happens. I don't think anyone really believes Rust solves every problem, so I think you're attacking a straw man. There are plenty of trade offs at play. But the problems we believe it does solve makes it exciting for a lot of us (and for perspective users).
    - adiabatty 10y ago
      Plus it's not really the fault of Rust developers if https://blog.ntpsec.org/2017/01/03/getting-past-c.html https://blog.ntpsec.org/2017/01/03/getting-past-c.html gets to the front page; that's not really marketing material for the language.
  - hacker_9 10y ago
    As a counter, stuff is posted to new every minute, but 90% of it doesn't reach the front page. In order to reach the front, you often have to do make up silly titles such as the ones you mentioned, which cause an emotional response (good or bad, as long as it gets clicks). HN also has a problem with brigading, which is why you see some utterly random articles make it to the top too. Sadly not much you can do about this, but you could try browsing /r/programming or elsewhere instead.
- ruster 10y ago
  and what's the argument here?
- flukus 10y ago
  Quite a few of us made learning rust our new years resolution in recent threads.
- anamoulous 10y ago
  More Rust discussion and less "future of media/ad supported businesses" discussion works for me.
  - flukus 10y ago
    Not to mention fake news and politics in general.
tps5 10y ago
Nice guide. I did this over the holidays just to see it in action. It was a pain to get the emscripten sdk installed (especially on a slow connection) but it's quite cool to see it in action. There's also a asm.js target which you can add like this: $ rustup target add asmjs-unknown-emscripten I'm not sure about browser support for asm.js but recent versions of node support it.
- marcoperaza 10y ago
  Every browser supports asm.js, since it's just a subset of standard JS. Some browsers do optimize it, though. https://en.wikipedia.org/wiki/Asm.js#Implementations https://en.wikipedia.org/wiki/Asm.js#Implementations
kibwen 10y ago
This is great, I've actually never seen the steps to compiling WASM laid out before. What all does the Emscripten SDK provide?
- hackcasual 10y ago
  An HTML shell that can handle resource loading, plus a bunch of JS methods for providing IO to WebAssembly. In the future it will be possible to do something like https://github.com/brson/mir2wasm https://github.com/brson/mir2wasm to go directly from LLVM to WASM (though IO functionality will need to be supplied separately).
rafinha 10y ago
Is there a way to debug apps compiled to Web assembly? (Like a gdb.js :D ) I suppose using standard dev tools won't be useful.
- MayeulC 10y ago
  That's one area where I would like to see web browsers cooperate a bit. At least Firefox and Chromium. This would give better tools to everyone. I won't bet on it, however.
- empath75 10y ago
  https://github.com/WebAssembly/design/blob/master/TextFormat.md https://github.com/WebAssembly/design/blob/master/TextFormat...
rl3 10y ago
What's the binary size like compared to C/C++? I've read the overhead of Rust's standard library is about 100kB. Not sure if putting it through the Emscripten toolchain changes that significantly.
- steveklabnik 10y ago
  In general, https://lifthrasiir.github.io/rustlog/why-is-a-rust-executable-large.html https://lifthrasiir.github.io/rustlog/why-is-a-rust-executab... For this specifically, I made some numbers when it first landed: https://www.reddit.com/r/rust/comments/575pp6/compiling_to_the_web_with_rust_and_emscripten/d8pkki0/?context=1 https://www.reddit.com/r/rust/comments/575pp6/compiling_to_t... Regardless, this is in a "oh my god it works" state, not a super serious best-we-can-do state. My understanding is that it's larger than it has to be at the moment. We'll get there.
  - rl3 10y ago
    >-rw-r--r-- 1 steve steve 159K Oct 12 20:00 sup.wasm It's looking pretty good already. Thanks for the reply.
sunng 10y ago
Here is an example that using Rust-native library for web development: https://github.com/tcr/rust-todomvc https://github.com/tcr/rust-todomvc Also I forked it to use handlebars: https://github.com/sunng87/rust-todomvc https://github.com/sunng87/rust-todomvc Using asmjs, the compiled js file is about 20MB and can be slow in browser. While wasm32 one is just ~500KB.
- k__ 10y ago
  Can wasm be chunked up and loaded on demand via AJAX, like JS?
ponytech 10y ago
Has anyone heard of python compiled to Web Assembly ?
- coldtea 10y ago
  Python needs all the extra runtime and interpreter cannot just be compiled that easily, except with dropped features.
  - Freak_NL 10y ago
    That reminds me of Google's GWT, which compiled Java to JavaScript and supported a limited subset of the JDK. Maintaining a legacy GWT codebase is not a very pleasant experience, because you are stuck in a Java-like abstraction layer on top of the actual browser features and the DOM. I guess compiling a language like Rust to WebAssembly solves these issues for a part?
    - k__ 10y ago
      I don't know, but if Web Assembly allows something like dynamic linking, it would enable, for example, Oracle to supply a WASM-JVM via CDN, that JVM based languages could target. But this would have to be a rather cut-down version of the JVM, I think, so probably not that viable.
    - flohofwoe 10y ago
      GWT is very different because it directly 'transpiles' Java to a high-level Javascript, you basically get the worst of both worlds ;) asm.js and WebAssembly are much lower level, they are generated from LLVM bitcode, theoretically any language that can be compiled through LLVM can be used with asm.js / WebAssembly (as long as the language runtime libraries are ported to run in a browser environment).
- k__ 10y ago
  As far as I know, if you don't want to add a runtime, you can only compile languages that compile to native code to Web Assembly. C, C++, D, Rust, Nim, etc. So it's more for low level lanuages to make performance easier on the Web, not for replacing JavaScript in general.
marcosscriven 10y ago
I wasn't a fan of having to compile the dev branch of both emscripten itself, and clang, each time I wanted to try the latest (hence the 'wait 2 hours' in the blog) I thus asked about providing a Docker image - the person that handles the builds and the SDKs says that nightly builds are on the way: https://github.com/kripken/emscripten/issues/4682#issuecomment-260159020 https://github.com/kripken/emscripten/issues/4682#issuecomme...
- brianshaler 10y ago
  I read the guide and came here looking to see if there was a docker image. I just don't want to blindly pipe-sh on my host machine (would need to review what rustup is doing first). On top of that, 'wait 2 hours' is not something I want to do when I want to kick the tires on a Hello World. This is a great example of docker-for-dev. Don't bother setting up the specific versions of compilers and then worry about clean-up afterward. The guide should be 2 lines: docker pull, then docker run.
microo8 10y ago
Still waiting for an article named 'Compiling Go to Web Assembly' https://imgflip.com/s/meme/Ill-Just-Wait-Here.jpg https://imgflip.com/s/meme/Ill-Just-Wait-Here.jpg

289 points by grey-area 10y ago

Three rust articles on the front page? Not complaining, it definitely seems like the popularity has really jumped up in the last three months.

hacker_9 10y ago
Rust and Go are always on the front page.
- rch 10y ago
  Also Python, JS frameworks, CLR and JVM languages, C/C++, LLVM, notably obscure languages... It's like there's some kind of theme.
- lacampbell 10y ago
  If someone makes a Go compiler in Rust hacker news will have to shut down. Game over, there is nothing left to do in computing.
  - flamedoge 10y ago
    Rust compiler written in Go?
  - slowmovintarget 10y ago
    Clojure-on-Rust Granted, it would require writing a garbage collector in Rust...
    - saghm 10y ago
      > Granted, it would require writing a garbage collector in Rust... Why would that be a problem? There are compilers written in C that implement garbage collectors. Now, writing a garbage collector for Rust, that's another problem entirely
      - Manishearth 10y ago
        > garbage collector in Rust.. http://users.cecs.anu.edu.au/~steveb/downloads/pdf/rust-ismm-2016.pdf http://users.cecs.anu.edu.au/~steveb/downloads/pdf/rust-ismm... > Now, writing a garbage collector for Rust, http://manishearth.github.io/blog/2015/09/01/designing-a-gc-in-rust/ http://manishearth.github.io/blog/2015/09/01/designing-a-gc-... (mostly a toy GC, but some folks are using it) In the future we may have http://manishearth.github.io/blog/2016/08/18/gc-support-in-rust-api-design/ http://manishearth.github.io/blog/2016/08/18/gc-support-in-r...
        saghm 10y ago
        During my senior year of college (which ended last spring), I worked with two other students on implementing a server-side JavaScript interpreter in Rust. One of us, a grad student (not me!), wrote the garbage collector for it (https://github.com/rustoscript/french-press https://github.com/rustoscript/french-press). We didn't end up finishing the entire language, but we got a sizable chunk of the language features implemented, and more than enough to know that Rust is very well suited to writing programming languages in, including garbage collectors. By far, the most complicated component of getting the garbage collector right was dealing with closures, but that's mostly because JavaScript is a very "unique" language, to put it nicely. > In the future we may have http://manishearth.github.io/blog/2016/08/18/gc-support-in-rust-api-design/ http://manishearth.github.io/blog/2016/08/18/gc-support-in-r... I read that back when you first posted it, which is exactly what came to mind when I was writing my comment. At least to my naive understanding, implementing a garbage collector for Rust is several orders of magnitude more difficult than merely implementing one in Rust.
        Manishearth 10y ago
        The JS interpreter is pretty cool! We've had problems with closures before in Servo, I know that pain :) > implementing a garbage collector for Rust is several orders of magnitude more difficult than merely implementing one in Rust. Well, I mean it depends on what you're aiming for. Making a safe optional GC in Rust isn't too hard; rust-gc exists and is not very complicated. Making the compiler support arbitrary pluggable GCs is trickier. GC algorithms themselves can get so complicated that the state of the art research probably eclipses whatever has been done with rust-gc and similar "GCs for rust" so far. If you want a GC for Rust that contains a world-class collection algorithm the complexity is additive but the source of most of the complexity isn't the "for Rust" part, it's the "in Rust" part (A GC for rust will also typically be in Rust). But also "optional" GCs like those that would be useful in Rust haven't been explored much either, so world class algorithms may not even make sense in this case. It's ... an interesting set of problems, and very hard to make judgement about what is difficult :)
    - lenkite 10y ago
      There is a lisp dialect on rust https://github.com/murarth/ketos https://github.com/murarth/ketos and it doesn't need GC!
  - solidsnack9000 10y ago
    https://github.com/yberreby/rgo https://github.com/yberreby/rgo "A Go compiler, written in Rust."
swsieber 10y ago
As far as I know it's because the Rust is filling in the gaps - two of the three articles are about relatively new features. And the next Rust release should close the stable/unstable gap by stabilizing some macro/compiler plugin stuff.
- pekk 10y ago
  It is a marketing push. Marketing pushes don't have to correlate to massive improvements in the product.
  - Manishearth 10y ago
    Certainly not a coordinated marketing push. All of the recent Rust posts on HN don't come from any official venues. It's just random community members writing about Rust, and HN upvoting it. There was a flurry of Rust posts about "why Rust" near the end of the year, which was somewhat "coordinated". In the sense that Steve wrote a post and this triggered a chain reaction where all kinds of folks wrote about related topics. Completely unintentional, just happened. Most of those posts did not make it to HN.
  - coldtea 10y ago
    It's just random people discovering Rust due to increased popularity and posting about it (which has a circular effect). No marketing plan involved, and nobody coordinates these posts.
lacampbell 10y ago
TBH it's getting a bit much at this point. The rust advocates here (not sure if they're paid or just fanboys) really need to tone it down. The hyperbole, and the constant "rust solves this unrelated problem, and has no flaws!" is beginning to grate. I've coded in it a bit and it seems like cool language, they get a lot right and have some new and interesting ideas. But it's not a panacea (or indeed "salvation" as one recent blog post put it) for everything wrong in software. We don't need to all immediately re-write everything we do rust. Not everything new needs to be written in rust. And your re-writes of programs that have already been written in other languages and work well aren't that compelling. If you like rust, then write some new, interesting and useful software in the language, where what language it's written in isn't the main selling point. For example, I love the look of this http://nickel.rs/ http://nickel.rs/. TL;DR - the corrosive nature of rust advocates on HN is starting to eat away at my patience, and I am tired of having to galvanize against them and their absurd claims.
- burntsushi 10y ago
  Yes, we had a recent explosion of marketing materials that went a little over the top. That's OK. It happens. I don't think anyone really believes Rust solves every problem, so I think you're attacking a straw man. There are plenty of trade offs at play. But the problems we believe it does solve makes it exciting for a lot of us (and for perspective users).
  - adiabatty 10y ago
    Plus it's not really the fault of Rust developers if https://blog.ntpsec.org/2017/01/03/getting-past-c.html https://blog.ntpsec.org/2017/01/03/getting-past-c.html gets to the front page; that's not really marketing material for the language.
- hacker_9 10y ago
  As a counter, stuff is posted to new every minute, but 90% of it doesn't reach the front page. In order to reach the front, you often have to do make up silly titles such as the ones you mentioned, which cause an emotional response (good or bad, as long as it gets clicks). HN also has a problem with brigading, which is why you see some utterly random articles make it to the top too. Sadly not much you can do about this, but you could try browsing /r/programming or elsewhere instead.
ruster 10y ago
and what's the argument here?
flukus 10y ago
Quite a few of us made learning rust our new years resolution in recent threads.
anamoulous 10y ago
More Rust discussion and less "future of media/ad supported businesses" discussion works for me.
- flukus 10y ago
  Not to mention fake news and politics in general.

tps5 10y ago

Nice guide. I did this over the holidays just to see it in action. It was a pain to get the emscripten sdk installed (especially on a slow connection) but it's quite cool to see it in action. There's also a asm.js target which you can add like this: $ rustup target add asmjs-unknown-emscripten I'm not sure about browser support for asm.js but recent versions of node support it.

marcoperaza 10y ago
Every browser supports asm.js, since it's just a subset of standard JS. Some browsers do optimize it, though. https://en.wikipedia.org/wiki/Asm.js#Implementations https://en.wikipedia.org/wiki/Asm.js#Implementations

kibwen 10y ago

This is great, I've actually never seen the steps to compiling WASM laid out before. What all does the Emscripten SDK provide?

hackcasual 10y ago
An HTML shell that can handle resource loading, plus a bunch of JS methods for providing IO to WebAssembly. In the future it will be possible to do something like https://github.com/brson/mir2wasm https://github.com/brson/mir2wasm to go directly from LLVM to WASM (though IO functionality will need to be supplied separately).

rafinha 10y ago

Is there a way to debug apps compiled to Web assembly? (Like a gdb.js :D ) I suppose using standard dev tools won't be useful.

MayeulC 10y ago
That's one area where I would like to see web browsers cooperate a bit. At least Firefox and Chromium. This would give better tools to everyone. I won't bet on it, however.
empath75 10y ago
https://github.com/WebAssembly/design/blob/master/TextFormat.md https://github.com/WebAssembly/design/blob/master/TextFormat...

rl3 10y ago

What's the binary size like compared to C/C++? I've read the overhead of Rust's standard library is about 100kB. Not sure if putting it through the Emscripten toolchain changes that significantly.

steveklabnik 10y ago
In general, https://lifthrasiir.github.io/rustlog/why-is-a-rust-executable-large.html https://lifthrasiir.github.io/rustlog/why-is-a-rust-executab... For this specifically, I made some numbers when it first landed: https://www.reddit.com/r/rust/comments/575pp6/compiling_to_the_web_with_rust_and_emscripten/d8pkki0/?context=1 https://www.reddit.com/r/rust/comments/575pp6/compiling_to_t... Regardless, this is in a "oh my god it works" state, not a super serious best-we-can-do state. My understanding is that it's larger than it has to be at the moment. We'll get there.
- rl3 10y ago
  >-rw-r--r-- 1 steve steve 159K Oct 12 20:00 sup.wasm It's looking pretty good already. Thanks for the reply.

sunng 10y ago

Here is an example that using Rust-native library for web development: https://github.com/tcr/rust-todomvc https://github.com/tcr/rust-todomvc Also I forked it to use handlebars: https://github.com/sunng87/rust-todomvc https://github.com/sunng87/rust-todomvc Using asmjs, the compiled js file is about 20MB and can be slow in browser. While wasm32 one is just ~500KB.

k__ 10y ago
Can wasm be chunked up and loaded on demand via AJAX, like JS?

ponytech 10y ago

Has anyone heard of python compiled to Web Assembly ?

coldtea 10y ago
Python needs all the extra runtime and interpreter cannot just be compiled that easily, except with dropped features.
- Freak_NL 10y ago
  That reminds me of Google's GWT, which compiled Java to JavaScript and supported a limited subset of the JDK. Maintaining a legacy GWT codebase is not a very pleasant experience, because you are stuck in a Java-like abstraction layer on top of the actual browser features and the DOM. I guess compiling a language like Rust to WebAssembly solves these issues for a part?
  - k__ 10y ago
    I don't know, but if Web Assembly allows something like dynamic linking, it would enable, for example, Oracle to supply a WASM-JVM via CDN, that JVM based languages could target. But this would have to be a rather cut-down version of the JVM, I think, so probably not that viable.
  - flohofwoe 10y ago
    GWT is very different because it directly 'transpiles' Java to a high-level Javascript, you basically get the worst of both worlds ;) asm.js and WebAssembly are much lower level, they are generated from LLVM bitcode, theoretically any language that can be compiled through LLVM can be used with asm.js / WebAssembly (as long as the language runtime libraries are ported to run in a browser environment).
k__ 10y ago
As far as I know, if you don't want to add a runtime, you can only compile languages that compile to native code to Web Assembly. C, C++, D, Rust, Nim, etc. So it's more for low level lanuages to make performance easier on the Web, not for replacing JavaScript in general.

marcosscriven 10y ago

I wasn't a fan of having to compile the dev branch of both emscripten itself, and clang, each time I wanted to try the latest (hence the 'wait 2 hours' in the blog) I thus asked about providing a Docker image - the person that handles the builds and the SDKs says that nightly builds are on the way: https://github.com/kripken/emscripten/issues/4682#issuecomment-260159020 https://github.com/kripken/emscripten/issues/4682#issuecomme...

brianshaler 10y ago
I read the guide and came here looking to see if there was a docker image. I just don't want to blindly pipe-sh on my host machine (would need to review what rustup is doing first). On top of that, 'wait 2 hours' is not something I want to do when I want to kick the tires on a Hello World. This is a great example of docker-for-dev. Don't bother setting up the specific versions of compilers and then worry about clean-up afterward. The guide should be 2 lines: docker pull, then docker run.

microo8 10y ago

Still waiting for an article named 'Compiling Go to Web Assembly' https://imgflip.com/s/meme/Ill-Just-Wait-Here.jpg https://imgflip.com/s/meme/Ill-Just-Wait-Here.jpg