10 ms·

Speedy Web Compiler – A Rust port of Babel and Closure Compiler

382 points by kn8 8y ago

calineczka 8y ago
How is it so much faster exactly? Is it due to parallelization or being written in a compiled language or something entirely different?
- steveklabnik 8y ago
  I don’t know much about how Babel is implemented, but this uses the excellent rayon crate, which makes data parallelism awesome and easy. Rust is also generally fast, especially when you put effort into performance, which (from the creator talking about this on reddit) was a big goal. This is the kind of thing Rust should be excellent at.
  - peterhunt 8y ago
    I bet if you compared single threaded performance you’d still blow Babel out of the water. Even with the best JITs, idiomatic js code is so much harder/impossible to optimize than idiomatic rust or c/c++ code.
    - steveklabnik 8y ago
      This might be true, but Babel is a big, important project. I’d have expected performance to be something they’ve invested in. That may be wrong.
      - acemarke 8y ago
        The Babel team is great.... and they're also tiny, overworked, and underfunded. (See some of Henry Zhu's recent conference talks on dealing with being an OSS maintainer for background.) I think Babel 7 had some perf improvements (including tweaks contributed by Benedikt Meurer from the v8 team), but I don't think they've ever had time to focus on just improving perf.
        hzoo 8y ago
        Yep there's been various PRs for perf over the years but it's never been a focus per-se, and we've had help from various people including the v8 team https://twitter.com/left_pad/status/927554660508028929 https://twitter.com/left_pad/status/927554660508028929. Haha yeah no one wants to hear about open source sustainability in a thread about asking that tools be correct/fast/easy to use at the same time :D It's true, speed is not really a concern on my mind at the moment - rather that we have enough people even willing to contribute back at all
        steveklabnik 8y ago
        Trust me, I am well aware that even big open source projects don’t get the resourcing they deserve. Everyone always has to make do. Babel is a fantastic project.
        hzoo 8y ago
        haha oops I totally wasn't looking at who I commenting with, definitely agree! Thanks!
        steveklabnik 8y ago
        No oops required! I should have been more careful in my initial phrasing. I certainly don’t want to imply that you should be doing even more work!
        steveklabnik 8y ago
        Yeah, those kind of one-off contributions was what I was meaning. You are 100% right with regards to expectations, and I don’t mean to imply that they should have. Only that it’s a bit more likely than some random projetct that nobody uses.
  - purple_ducks 8y ago
    > which makes data parallelism awesome and easy There's still going to be a finite number of cores on a machine. I can't see gaining parallelism being that big a boost (as opposed to gaining concurrency(if not already there))
    - zozbot123 8y ago
      You'd be surprised, even consumer-grade CPUs are fast approaching manycore territory these days. Just look e.g. at AMD's Threadripper, which BTW is being targeted at hobbyists and enthusiasts, not just professionals. (There are also ARM-based 'desktops'/'workstations' with comparable number of cores.) If we want to make anywhere near full use of these capabilities, we'll need easy parallelization (for applicable problems) to become essentially ubiquitous. Rust and Rayon are perfect for that.
    - steveklabnik 8y ago
      I would expect this task to be CPU bound, not IO bound. Maybe that’s incorrect.
  - Matthias247 8y ago
    > This is the kind of thing Rust should be excellent at. I would actually think this (compilers) is a thing where Rust doesn’t have that big of an advantage. Compilers are very special. They mostly deal with some kind of tree transformation. And as soon as those trees use owned nodes (E.g. in the form of Box of Rc) things are not that far from what a speedy managed language would do. Compilers are also non realtime at all. It doesn’t really matter if there’s a GC pause or not, and whether individual allocations are quick or slow. The only thing that matters is the overall throughput/performance. Now this statement shouldn’t mean that Rust is bad at doing compilers (enough projects tell otherwise). Only that in other domains (E.g. realtime and low level code, graphics, audio, etc) it’s advantages might be even bigger.
    - girvo 8y ago
      I agree - which is why I’ve kept an eye on Fastpack as a replacement for my biggest bottleneck which is bundling — which is really just a special case of a compiler in a lot of ways. OCaml is brilliant for these tasks. https://github.com/fastpack/fastpack/blob/master/README.md https://github.com/fastpack/fastpack/blob/master/README.md
  - vram22 8y ago
    >this uses the excellent rayon crate, which makes data parallelism awesome and easy. Is that somewhat similar to D's std.parallelism module? A simple example of its use here: https://jugad2.blogspot.com/2016/12/simple-parallel-processing-in-d-with.html https://jugad2.blogspot.com/2016/12/simple-parallel-processi...
    - steveklabnik 8y ago
      A similar idea, yes. Details may be a bit different.
      - vram22 8y ago
        Got it, thanks.
- TeMPOraL 8y ago
  Did Babel contributors put much effort into optimization? JS isn't exactly high-performance by default.
  - hzoo 8y ago
    We certaintly have PRs around perf https://github.com/babel/babel/labels/area%3A%20perf https://github.com/babel/babel/labels/area%3A%20perf but we don't have explicit benchmarking for it, and have had work from v8 team and others https://twitter.com/rauchg/status/924349334346276864?s=20 https://twitter.com/rauchg/status/924349334346276864?s=20.
ajross 8y ago
Seems like transcompilation is an area where you really DON'T want be overfocusing on performance. Correctness is king here. I mean, compiler bugs are hard. Really hard. Deep voodoo hard. No one who's ever dealt with a significant code generation bug on a large project ever wants to repeat that experience. That's not to say that babel isn't too slow or that this project isn't correct, just that it would scare me to try to work with it. Does babel have a regression suite? Does this pass it?
- peterhunt 8y ago
  Js packing performance (including compilation) is probably the slowest part of the iteration cycle for frontend devs today. When combined with Babel’s and ecmas extensive test suites I think something like this could be wildly successful.
  - rcavezza 8y ago
    Can you elaborate more on this? I've worked with very large codebases in React and Angular and have not seen this issue. I'm guessing it's either because all of my applications were able to hot-reload or we're not thinking of the same thing.
    - the_mitsuhiko 8y ago
      I don’t know if we’re doing something wrong but initial startup time on sentry’s webpack setup is something like 30 seconds and iterative changes around 5 seconds. Still beats our rust or python iteration times though :)
      - orf 8y ago
        Have you looked into why the Sentry Django app takes so long to boot? Are you doing queries on app init or something?
    - peterhunt 8y ago
      a lot of codebases don't support hot reload, but even the ones that do often require multi-second incremental compile times during hot reload if they're big enough [1]. [1] current project is very slow w/ 250kloc (2.5mm kloc w/ dependencies). even seen it be slow on my last project which was only 40kloc.
kmlx 8y ago
for me these kinds of tools are only as good as the time spent debugging when something goes awry. as such, the biggest risk i see is that it isn't built in javascript. i simply don't see myself debugging rust while trying to compile javascript. i really don't. this being said, i do wish this project (and any others like it) great success. we definitely do need something faster when compiling js. (this tool reminds me of a similar project for a small client. they required a fast js compiler. the project was a failure because no-one wanted to debug non-js code when the build failed)
- deleted 8y ago
  [deleted]
- sam0x17 8y ago
  This is why I'd really like to see some projects that compile to readable js when not in production mode. Throw in some auto-generated comments above every method that tell you the file and line number in the Rust/Go/Crystal/Whatever code that this comes from, and debugging will be fairly easy.
rattray 8y ago
Does this work with JSX and TypeScript/Flow? These language extensions, when enabled, have nontrivial performance impact. On that note, they don't say what plugins/presets were used in the Babel comparison. Was it the same set of features that are supported in swc?
- hzoo 8y ago
  Yeah I've had a difficult time figuring out how this was tested: I would recommend using the benchmark at https://github.com/v8/web-tooling-benchmark https://github.com/v8/web-tooling-benchmark in the future if it's currently not possible since currently it only seems to be testing ES5 (jquery.min.js) for the parser and a short ES6 snippet for the transforms.
  - rattray 8y ago
    (I think when you say "tested" you mean "benchmarked" – your comment at https://news.ycombinator.com/item?id=18746905 https://news.ycombinator.com/item?id=18746905 gave a nice link to how it is tested).
rattray 8y ago
Where are the test cases? Babel's parser alone has quite a formidable suite of fixtures (a few thousand as I recall): https://github.com/babel/babel/tree/master/packages/babel-parser/test/fixtures https://github.com/babel/babel/tree/master/packages/babel-pa... The nice thing about implementing a JS-JS compiler is that there is already a huge number of tests out there, like these, that you can use to TDD your compiler's edge-cases. SWC's CONTRIBUTING mentions this: > Include tests that cover all non-trivial code. The existing tests in test/ provide templates on how to test swc's behavior in a sandbox-environment. The internal crate testing provides a vast amount of helpers to minimize boilerplate. See [testing/lib.rs] for an introduction to writing tests. But I cannot find a `/test` directory, and it does not appear in the .gitignore either. EDIT: Ah, the tests are in `/tests.rs` with JS embedded inside rust code. This makes it a bit harder to compare directly with babel's suite, but claims to be lifted from test262, which babel also based many of their tests on (albeit recategorized). hzoo's comment here has details: https://news.ycombinator.com/item?id=18746905 https://news.ycombinator.com/item?id=18746905
- steveklabnik 8y ago
  https://news.ycombinator.com/item?id=18746754 https://news.ycombinator.com/item?id=18746754 https://news.ycombinator.com/item?id=18746905 https://news.ycombinator.com/item?id=18746905
  - rattray 8y ago
    Thanks Steve! Missed those; race condition perhaps. Updated.
rattray 8y ago
Perhaps the best thing about Babel is its plugin-based architecture. As of Babel 7, even its parser accepts plugins. This not only makes it easy to keep up to the latest of ECMA's standards and extensions like JSX, Flow, TypeScript – it also makes it surprisingly easy to implement your own syntax extensions to JS (eg; https://wcjohnson.github.io/lightscript/ https://wcjohnson.github.io/lightscript/, which I have worked on). Curious how this project is thinking about plugins and staying up-to-date sustainably.
- mintplant 8y ago
  > As of Babel 7, even its parser accepts plugins. Does it really? Last time I looked into this, the "parser plugins" were all fake: single-line modules which simply switched on hidden config flags in Babel to enable functionality already hard-coded into the core parser. Left a bad taste in my mouth that the developers weren't up-front about the true (non-)extensibility of the system.
LinaLauneBaer 8y ago
I never had any problems with Babel's performance to be honest... Also a quick check with google trends does seem to support that feeling. https://trends.google.com/trends/explore?cat=31&date=all&q=Swift%20slow,Babel%20slow https://trends.google.com/trends/explore?cat=31&date=all&q=S... Swift is probably used a lot less than Babel but it yields a lot more spikes...
rattray 8y ago
Those interested in fast JS parsers may also be interested in cherow – https://github.com/cherow/cherow https://github.com/cherow/cherow – though it does not provide compilation.
losvedir 8y ago
This is really cool. I understand why all the front end tooling is written in JS but it really is a pain to deal with in my experience, especially when it comes to building and deploying. Since they're essentially just scripts you have to also install the right versions of node and npm and use npm to install the dependencies (and watch out for native extensions!). It's fairly tedious to get it all right, whenever I try it, since I'm not a pure frontend dev, and things have always changed since the last time I did it. But the promise of this is just dropping in a binary and having it all work! I actually like JS as a programming language for the web, I just don't like scripting languages in general for command line tools.
- alangpierce 8y ago
  Funny, I actually view it as the opposite. swc is distributed via npm as a native extension, so you need to have the right rust version installed in order for it to work. I tried `yarn add --dev swc` like in the instructions, but it failed. I already had rust installed but I guess it wasn't the right rust. I'd be hesitant to tell my team to use it because of the extra dev environment setup. JS is a universal binary format that tends to just work from my experience. You do occasionally run into issues if you're on an old node version, but other than that it has been pretty reliable for me. But it may just be a difference in familiarity.
  - steveklabnik 8y ago
    This is one area where wasm may shine; you can distribute it pre-compiled, nobody using JS needs a rust toolchain installed. This is something the webpack team has indicated interest in, for example. Of course, wasm will need threading first for this project.
    - jahewson 8y ago
      Ooh I like it - yet another reason to get excited about wasm, thanks Steve!
    - ori_b 8y ago
      How do you verify that the distribution matches the source?
JeremyBanks 8y ago
Closure Compiler is a poorly specified pile of edge cases and bugs. (Embarassingly poor quality for something Google relies on so much.) Is this attempting to support its type system, minification moded, or what? I'd be impressed if it managed a significant subset.
alangpierce 8y ago
I've been working on a very similar tool, interesting to see some competition. :-) https://github.com/alangpierce/sucrase https://github.com/alangpierce/sucrase In my case, it's still running as JS, but rearchitected to solve the more straightforward problem where you don't need to compile to ES5. I've thought about rewriting it in Rust to see how much faster it gets, though currently I'm trying to get it running in WebAssembly via AssemblyScript ( https://github.com/AssemblyScript/assemblyscript https://github.com/AssemblyScript/assemblyscript ), which has been promising. I'm curious about the about the Babel performance comparison. Benchmarking JS is tricky because, from my observations, the performance improves by a factor of ~20 if you give it 5-10 seconds for the JIT to fully optimize everything. https://github.com/alangpierce/sucrase/issues/216 https://github.com/alangpierce/sucrase/issues/216 . Rust and WebAssembly both have a significant advantage in that sense when running on small datasets.
- azakai 8y ago
  The AssemblyScript conversion sounds very interesting! Is there an issue or PR where I can follow that?
  - alangpierce 8y ago
    Yep! Issue (already linked above): https://github.com/alangpierce/sucrase/issues/216 https://github.com/alangpierce/sucrase/issues/216 Most recent prototype branch (doesn't include some patches to AssemblyScript to get it working): https://github.com/alangpierce/sucrase/commits/assemblyscript-prototype https://github.com/alangpierce/sucrase/commits/assemblyscrip... I've also chatted about it a little on the AssemblyScript Slack.
- natural219 8y ago
  Hey man, I just started using sucrase in production at work. Great work with it; just the right features to let me kill Babel, simplify build docs, and still get use some modern niceties (I use it alongside gulp/rollup)
  - alangpierce 8y ago
PascalW 8y ago
There's also Pax [1], also written in Rust. Pax however is only a bundler, I don't think it actually transpiles code. [1] https://github.com/nathan/pax https://github.com/nathan/pax
pookeh 8y ago
Honestly I don't think Babel is the performance bottleneck in most dev setups. Most slow frontend dev setups are because of webpack and it's core architecture. Even the ts compiler is quite fast.
- alangpierce 8y ago
  I've looked into dev tooling performance quite a bit and I think your comment is half-true. Babel is just part of the performance story, and in some situations, speeding up Babel doesn't help at all. One complication is the fact that Babel results are often cached, so with a reliable cache, Babel performance barely matters at all. However, every caching approach I've seen has reliability issues in practice. Certainly a tool is more pleasant to use without a cache, since "maybe I have a bad cache" is never a worry when running into strange behavior. It also depends a lot on your scale, how you compile your code, and what operations you want to be fast. When switching to Sucrase (my own Babel alternative project, mentioned elsewhere) on a large codebase, webpack startup went from 40 seconds to 20 seconds, though incremental builds weren't any different. Running tests (in node with a require hook) became much faster, from minutes to seconds when uncached and about a 2x speedup when Babel pulled from cache. In both of these cases, the remaining time is now in other things: webpack processing the files, node running the imports, etc. So there's a lot more besides Babel, but Babel still is a non-trivial component in the cases I've run into.
Solar19 8y ago
How is it Rust project if it's installed via Node?
- steveklabnik 8y ago
  You can put anything you want in an npm package.

382 points by kn8 8y ago

How is it so much faster exactly? Is it due to parallelization or being written in a compiled language or something entirely different?

steveklabnik 8y ago
I don’t know much about how Babel is implemented, but this uses the excellent rayon crate, which makes data parallelism awesome and easy. Rust is also generally fast, especially when you put effort into performance, which (from the creator talking about this on reddit) was a big goal. This is the kind of thing Rust should be excellent at.
- peterhunt 8y ago
  I bet if you compared single threaded performance you’d still blow Babel out of the water. Even with the best JITs, idiomatic js code is so much harder/impossible to optimize than idiomatic rust or c/c++ code.
  - steveklabnik 8y ago
    This might be true, but Babel is a big, important project. I’d have expected performance to be something they’ve invested in. That may be wrong.
    - acemarke 8y ago
      The Babel team is great.... and they're also tiny, overworked, and underfunded. (See some of Henry Zhu's recent conference talks on dealing with being an OSS maintainer for background.) I think Babel 7 had some perf improvements (including tweaks contributed by Benedikt Meurer from the v8 team), but I don't think they've ever had time to focus on just improving perf.
      - hzoo 8y ago
        Yep there's been various PRs for perf over the years but it's never been a focus per-se, and we've had help from various people including the v8 team https://twitter.com/left_pad/status/927554660508028929 https://twitter.com/left_pad/status/927554660508028929. Haha yeah no one wants to hear about open source sustainability in a thread about asking that tools be correct/fast/easy to use at the same time :D It's true, speed is not really a concern on my mind at the moment - rather that we have enough people even willing to contribute back at all
        steveklabnik 8y ago
        Trust me, I am well aware that even big open source projects don’t get the resourcing they deserve. Everyone always has to make do. Babel is a fantastic project.
        hzoo 8y ago
        haha oops I totally wasn't looking at who I commenting with, definitely agree! Thanks!
        steveklabnik 8y ago
        No oops required! I should have been more careful in my initial phrasing. I certainly don’t want to imply that you should be doing even more work!
      - steveklabnik 8y ago
        Yeah, those kind of one-off contributions was what I was meaning. You are 100% right with regards to expectations, and I don’t mean to imply that they should have. Only that it’s a bit more likely than some random projetct that nobody uses.
- purple_ducks 8y ago
  > which makes data parallelism awesome and easy There's still going to be a finite number of cores on a machine. I can't see gaining parallelism being that big a boost (as opposed to gaining concurrency(if not already there))
  - zozbot123 8y ago
    You'd be surprised, even consumer-grade CPUs are fast approaching manycore territory these days. Just look e.g. at AMD's Threadripper, which BTW is being targeted at hobbyists and enthusiasts, not just professionals. (There are also ARM-based 'desktops'/'workstations' with comparable number of cores.) If we want to make anywhere near full use of these capabilities, we'll need easy parallelization (for applicable problems) to become essentially ubiquitous. Rust and Rayon are perfect for that.
  - steveklabnik 8y ago
    I would expect this task to be CPU bound, not IO bound. Maybe that’s incorrect.
- Matthias247 8y ago
  > This is the kind of thing Rust should be excellent at. I would actually think this (compilers) is a thing where Rust doesn’t have that big of an advantage. Compilers are very special. They mostly deal with some kind of tree transformation. And as soon as those trees use owned nodes (E.g. in the form of Box of Rc) things are not that far from what a speedy managed language would do. Compilers are also non realtime at all. It doesn’t really matter if there’s a GC pause or not, and whether individual allocations are quick or slow. The only thing that matters is the overall throughput/performance. Now this statement shouldn’t mean that Rust is bad at doing compilers (enough projects tell otherwise). Only that in other domains (E.g. realtime and low level code, graphics, audio, etc) it’s advantages might be even bigger.
  - girvo 8y ago
    I agree - which is why I’ve kept an eye on Fastpack as a replacement for my biggest bottleneck which is bundling — which is really just a special case of a compiler in a lot of ways. OCaml is brilliant for these tasks. https://github.com/fastpack/fastpack/blob/master/README.md https://github.com/fastpack/fastpack/blob/master/README.md
- vram22 8y ago
  >this uses the excellent rayon crate, which makes data parallelism awesome and easy. Is that somewhat similar to D's std.parallelism module? A simple example of its use here: https://jugad2.blogspot.com/2016/12/simple-parallel-processing-in-d-with.html https://jugad2.blogspot.com/2016/12/simple-parallel-processi...
  - steveklabnik 8y ago
    A similar idea, yes. Details may be a bit different.
    - vram22 8y ago
      Got it, thanks.
TeMPOraL 8y ago
Did Babel contributors put much effort into optimization? JS isn't exactly high-performance by default.
- hzoo 8y ago
  We certaintly have PRs around perf https://github.com/babel/babel/labels/area%3A%20perf https://github.com/babel/babel/labels/area%3A%20perf but we don't have explicit benchmarking for it, and have had work from v8 team and others https://twitter.com/rauchg/status/924349334346276864?s=20 https://twitter.com/rauchg/status/924349334346276864?s=20.

ajross 8y ago

Seems like transcompilation is an area where you really DON'T want be overfocusing on performance. Correctness is king here. I mean, compiler bugs are hard. Really hard. Deep voodoo hard. No one who's ever dealt with a significant code generation bug on a large project ever wants to repeat that experience. That's not to say that babel isn't too slow or that this project isn't correct, just that it would scare me to try to work with it. Does babel have a regression suite? Does this pass it?

peterhunt 8y ago
Js packing performance (including compilation) is probably the slowest part of the iteration cycle for frontend devs today. When combined with Babel’s and ecmas extensive test suites I think something like this could be wildly successful.
- rcavezza 8y ago
  Can you elaborate more on this? I've worked with very large codebases in React and Angular and have not seen this issue. I'm guessing it's either because all of my applications were able to hot-reload or we're not thinking of the same thing.
  - the_mitsuhiko 8y ago
    I don’t know if we’re doing something wrong but initial startup time on sentry’s webpack setup is something like 30 seconds and iterative changes around 5 seconds. Still beats our rust or python iteration times though :)
    - orf 8y ago
      Have you looked into why the Sentry Django app takes so long to boot? Are you doing queries on app init or something?
  - peterhunt 8y ago
    a lot of codebases don't support hot reload, but even the ones that do often require multi-second incremental compile times during hot reload if they're big enough [1]. [1] current project is very slow w/ 250kloc (2.5mm kloc w/ dependencies). even seen it be slow on my last project which was only 40kloc.

kmlx 8y ago

for me these kinds of tools are only as good as the time spent debugging when something goes awry. as such, the biggest risk i see is that it isn't built in javascript. i simply don't see myself debugging rust while trying to compile javascript. i really don't. this being said, i do wish this project (and any others like it) great success. we definitely do need something faster when compiling js. (this tool reminds me of a similar project for a small client. they required a fast js compiler. the project was a failure because no-one wanted to debug non-js code when the build failed)

deleted 8y ago
[deleted]
sam0x17 8y ago
This is why I'd really like to see some projects that compile to readable js when not in production mode. Throw in some auto-generated comments above every method that tell you the file and line number in the Rust/Go/Crystal/Whatever code that this comes from, and debugging will be fairly easy.

rattray 8y ago

Does this work with JSX and TypeScript/Flow? These language extensions, when enabled, have nontrivial performance impact. On that note, they don't say what plugins/presets were used in the Babel comparison. Was it the same set of features that are supported in swc?

hzoo 8y ago
Yeah I've had a difficult time figuring out how this was tested: I would recommend using the benchmark at https://github.com/v8/web-tooling-benchmark https://github.com/v8/web-tooling-benchmark in the future if it's currently not possible since currently it only seems to be testing ES5 (jquery.min.js) for the parser and a short ES6 snippet for the transforms.
- rattray 8y ago
  (I think when you say "tested" you mean "benchmarked" – your comment at https://news.ycombinator.com/item?id=18746905 https://news.ycombinator.com/item?id=18746905 gave a nice link to how it is tested).

rattray 8y ago

Where are the test cases? Babel's parser alone has quite a formidable suite of fixtures (a few thousand as I recall): https://github.com/babel/babel/tree/master/packages/babel-parser/test/fixtures https://github.com/babel/babel/tree/master/packages/babel-pa... The nice thing about implementing a JS-JS compiler is that there is already a huge number of tests out there, like these, that you can use to TDD your compiler's edge-cases. SWC's CONTRIBUTING mentions this: > Include tests that cover all non-trivial code. The existing tests in test/ provide templates on how to test swc's behavior in a sandbox-environment. The internal crate testing provides a vast amount of helpers to minimize boilerplate. See [testing/lib.rs] for an introduction to writing tests. But I cannot find a `/test` directory, and it does not appear in the .gitignore either. EDIT: Ah, the tests are in `/tests.rs` with JS embedded inside rust code. This makes it a bit harder to compare directly with babel's suite, but claims to be lifted from test262, which babel also based many of their tests on (albeit recategorized). hzoo's comment here has details: https://news.ycombinator.com/item?id=18746905 https://news.ycombinator.com/item?id=18746905

steveklabnik 8y ago
https://news.ycombinator.com/item?id=18746754 https://news.ycombinator.com/item?id=18746754 https://news.ycombinator.com/item?id=18746905 https://news.ycombinator.com/item?id=18746905
- rattray 8y ago
  Thanks Steve! Missed those; race condition perhaps. Updated.

rattray 8y ago

Perhaps the best thing about Babel is its plugin-based architecture. As of Babel 7, even its parser accepts plugins. This not only makes it easy to keep up to the latest of ECMA's standards and extensions like JSX, Flow, TypeScript – it also makes it surprisingly easy to implement your own syntax extensions to JS (eg; https://wcjohnson.github.io/lightscript/ https://wcjohnson.github.io/lightscript/, which I have worked on). Curious how this project is thinking about plugins and staying up-to-date sustainably.

mintplant 8y ago
> As of Babel 7, even its parser accepts plugins. Does it really? Last time I looked into this, the "parser plugins" were all fake: single-line modules which simply switched on hidden config flags in Babel to enable functionality already hard-coded into the core parser. Left a bad taste in my mouth that the developers weren't up-front about the true (non-)extensibility of the system.

LinaLauneBaer 8y ago

I never had any problems with Babel's performance to be honest... Also a quick check with google trends does seem to support that feeling. https://trends.google.com/trends/explore?cat=31&date=all&q=Swift%20slow,Babel%20slow https://trends.google.com/trends/explore?cat=31&date=all&q=S... Swift is probably used a lot less than Babel but it yields a lot more spikes...

rattray 8y ago

Those interested in fast JS parsers may also be interested in cherow – https://github.com/cherow/cherow https://github.com/cherow/cherow – though it does not provide compilation.

losvedir 8y ago

This is really cool. I understand why all the front end tooling is written in JS but it really is a pain to deal with in my experience, especially when it comes to building and deploying. Since they're essentially just scripts you have to also install the right versions of node and npm and use npm to install the dependencies (and watch out for native extensions!). It's fairly tedious to get it all right, whenever I try it, since I'm not a pure frontend dev, and things have always changed since the last time I did it. But the promise of this is just dropping in a binary and having it all work! I actually like JS as a programming language for the web, I just don't like scripting languages in general for command line tools.

alangpierce 8y ago
Funny, I actually view it as the opposite. swc is distributed via npm as a native extension, so you need to have the right rust version installed in order for it to work. I tried `yarn add --dev swc` like in the instructions, but it failed. I already had rust installed but I guess it wasn't the right rust. I'd be hesitant to tell my team to use it because of the extra dev environment setup. JS is a universal binary format that tends to just work from my experience. You do occasionally run into issues if you're on an old node version, but other than that it has been pretty reliable for me. But it may just be a difference in familiarity.
- steveklabnik 8y ago
  This is one area where wasm may shine; you can distribute it pre-compiled, nobody using JS needs a rust toolchain installed. This is something the webpack team has indicated interest in, for example. Of course, wasm will need threading first for this project.
  - jahewson 8y ago
    Ooh I like it - yet another reason to get excited about wasm, thanks Steve!
  - ori_b 8y ago
    How do you verify that the distribution matches the source?

JeremyBanks 8y ago

Closure Compiler is a poorly specified pile of edge cases and bugs. (Embarassingly poor quality for something Google relies on so much.) Is this attempting to support its type system, minification moded, or what? I'd be impressed if it managed a significant subset.

alangpierce 8y ago

I've been working on a very similar tool, interesting to see some competition. :-) https://github.com/alangpierce/sucrase https://github.com/alangpierce/sucrase In my case, it's still running as JS, but rearchitected to solve the more straightforward problem where you don't need to compile to ES5. I've thought about rewriting it in Rust to see how much faster it gets, though currently I'm trying to get it running in WebAssembly via AssemblyScript ( https://github.com/AssemblyScript/assemblyscript https://github.com/AssemblyScript/assemblyscript ), which has been promising. I'm curious about the about the Babel performance comparison. Benchmarking JS is tricky because, from my observations, the performance improves by a factor of ~20 if you give it 5-10 seconds for the JIT to fully optimize everything. https://github.com/alangpierce/sucrase/issues/216 https://github.com/alangpierce/sucrase/issues/216 . Rust and WebAssembly both have a significant advantage in that sense when running on small datasets.

azakai 8y ago
The AssemblyScript conversion sounds very interesting! Is there an issue or PR where I can follow that?
- alangpierce 8y ago
  Yep! Issue (already linked above): https://github.com/alangpierce/sucrase/issues/216 https://github.com/alangpierce/sucrase/issues/216 Most recent prototype branch (doesn't include some patches to AssemblyScript to get it working): https://github.com/alangpierce/sucrase/commits/assemblyscript-prototype https://github.com/alangpierce/sucrase/commits/assemblyscrip... I've also chatted about it a little on the AssemblyScript Slack.
natural219 8y ago
Hey man, I just started using sucrase in production at work. Great work with it; just the right features to let me kill Babel, simplify build docs, and still get use some modern niceties (I use it alongside gulp/rollup)
- alangpierce 8y ago

PascalW 8y ago

There's also Pax [1], also written in Rust. Pax however is only a bundler, I don't think it actually transpiles code. [1] https://github.com/nathan/pax https://github.com/nathan/pax

pookeh 8y ago

Honestly I don't think Babel is the performance bottleneck in most dev setups. Most slow frontend dev setups are because of webpack and it's core architecture. Even the ts compiler is quite fast.

alangpierce 8y ago
I've looked into dev tooling performance quite a bit and I think your comment is half-true. Babel is just part of the performance story, and in some situations, speeding up Babel doesn't help at all. One complication is the fact that Babel results are often cached, so with a reliable cache, Babel performance barely matters at all. However, every caching approach I've seen has reliability issues in practice. Certainly a tool is more pleasant to use without a cache, since "maybe I have a bad cache" is never a worry when running into strange behavior. It also depends a lot on your scale, how you compile your code, and what operations you want to be fast. When switching to Sucrase (my own Babel alternative project, mentioned elsewhere) on a large codebase, webpack startup went from 40 seconds to 20 seconds, though incremental builds weren't any different. Running tests (in node with a require hook) became much faster, from minutes to seconds when uncached and about a 2x speedup when Babel pulled from cache. In both of these cases, the remaining time is now in other things: webpack processing the files, node running the imports, etc. So there's a lot more besides Babel, but Babel still is a non-trivial component in the cases I've run into.

Solar19 8y ago

How is it Rust project if it's installed via Node?

steveklabnik 8y ago
You can put anything you want in an npm package.