5 ms·

Show HN: Sucrase, 20x faster smaller-scoped alternative to Babel

88 points by alangpierce 8y ago

fold_left 8y ago
I could imagine using the jest plugin, could be useful as a presumably faster alternative to babel-jest.
- alangpierce 8y ago
  That's the hope! To be honest, I haven't worked with Jest a lot, and the plugin is sort of a first pass to get things working on the Apollo codebase (which uses Jest), so e.g. it may not be as smart as babel-jest for things like caching. Contributions welcome, though! My specific use case so far has been running Mocha tests on a large codebase, so it's replacing babel-register (which uses a cache) with sucrase/register (which doesn't). For that use case, it seems to significantly speed up test startup time with a cold cache and somewhat speed up startup time with a warm cache. Just loading and saving the babel-register cache takes a fair amount of time, at least in Babel 6. It's good to keep caching in mind when thinking about this stuff. In many cases, Sucrase won't be faster than loading results from cache. Sucrase helps with initial startup time and avoids the cache fragility issues that I've seen with lots of caching systems. And, of course, you could cache Sucrase results and get something at least as fast as cached Babel.
btown 8y ago
Is there a good way to use this from Webpack? Edit: https://github.com/alangpierce/sucrase/blob/master/integrations/webpack-loader/ https://github.com/alangpierce/sucrase/blob/master/integrati...
- alangpierce 8y ago
  Yep, looks like you found the plugin! Note that if you're using Webpack 4, you won't need webpack-object-rest-spread-plugin anymore, since it uses the newest Acorn, which should handle object rest/spread syntax without needing a plugin. Annoyingly, Webpack still spends a lot of time parsing the JS files with its own parser, so you cut down on the transpile time but still need to do a relatively slow parse of all files. I've thought about making Webpack (or similar) use a fast parser like the one Sucrase uses, although it certainly seems like it would be a lot of work.
  - the_duke 8y ago
    Do you have any benchmarks showing how much time is spent in on webpack plumbing vs babel, comparing to sucrase? As in, how much does this actually buy you in a real world scenario?
    - alangpierce 8y ago
      I'm not very optimistic about webpack build times getting dramatically better with Sucrase in its current state. I've seen more clear benefits when running tests using sucrase/register. That said, I think it depends quite a bit on your webpack config. Certainly, if you run Babel single-threaded and don't cache the output, then you'll probably get a significant benefit, but that's likely not a realistic scenario. Some specific numbers I just measured for my codebase at work: Babel/TypeScript, cold cache: 66 seconds Babel/TypeScript, warm cache: 55 seconds Sucrase, cold cache: 52 seconds Sucrase, warm cache: 49 seconds This is on a codebase of about 400,000 lines of code (some JS compiled with Babel, some TypeScript compiled with tsc). Transpilation is parallelized using happypack (and I'm running it on a 4-core machine), but webpack parsing/processing is all single-threaded, so it ends up taking more time. I've done a little prototyping on using a Sucrase-like approach to speed up webpack (most of it is indeed parse time), but it would certainly be a project to get it working in practice. With a warm cache, it's not running sucrase/babel/typescript at all. I think the running time difference there is because Babel and TypeScript are emitting ES5, which is a little more code for webpack to parse. Probably a more correct comparison would be to configure Babel and TypeScript to target newer JS, but these should all be seen as rough numbers anyway.
nicoburns 8y ago
Does anyone have any idea how much performance is being left on the table by writing these compile-to-js compilers in JS? Could we get another 10x improvement by writing the transpilers in something like Go/Rust/C++? Frontend compile performance is beginning to get painful with large codebases...
- tjallingt 8y ago
  Some examples of "transpilers" targeting Javascript but writen in other languages: https://github.com/google/closure-compiler https://github.com/google/closure-compiler https://github.com/BuckleScript/bucklescript https://github.com/BuckleScript/bucklescript https://github.com/fastpack/fastpack https://github.com/fastpack/fastpack Seems like on the average it does offer a boost in performance. And there is some aditional work providing javascript parsers for rust (which you could build tools like babel on top of): https://github.com/dherman/esprit https://github.com/dherman/esprit
  - kevinb7 8y ago
    I think part of BuckleScript's speed may come from the source language, OCaml, being easier to parse. There may also be fewer passes necessary to transform OCaml to JavaScript.
    - chenglou 8y ago
      BuckleScript using Reason (with a similar syntax than JS, slower parser than vanilla OCaml) is still one of the fastest around. Mostly a case of very careful and dedicated engineering here (I work on Reason with the author of BuckleScript). Additionally, we try to work smart and delegate most of the build process to Ninja, which itself is one of the fastests around. But I believe the topic here was about runtime performance of using a language to compile JS, not about the build speed of working in that language itself. In which case you’ll still get some wins writing a JS toolchain in BuckleScript (compiled to JS), just from the JiT-friendliness of the BuckleScript JS output. But realistically, you’d be compiling to native OCaml through the same codebase. We did see a 10-25x perf jump from converting a part of a Babel pipeline to native OCaml. I mean, these languages are basically designed over decades with AST manipulation in mind, so that’s not surprising.
fiatjaf 8y ago
Thank you for making this. Babel really needs some alternatives. It would be great to have a Browserify plugin (I'm not asking for anything, just speaking my thoughts).
- alangpierce 8y ago
  Yep, hopefully this will motivate more of a performance focus in general. To be clear, Sucrase's parser is a slimmed-down fork of Babel's parser, so Sucrase wouldn't be possible without Babel. Also, Babel has a much broader scope as a pluginizable code transformation tool, so it's much better suited for things like prototyping future language features and doing other nontrivial transformations. Regarding Browserify support, PRs are welcome. :-) https://github.com/alangpierce/sucrase/tree/master/integrations https://github.com/alangpierce/sucrase/tree/master/integrati...
bpierre 8y ago
How does it compare to Bublé? https://buble.surge.sh/guide/ https://buble.surge.sh/guide/
- rich_harris 8y ago
  Bublé creator here. The obvious big difference is that this supports TypeScript and Flow, which Bublé doesn't. Bublé is designed to compile a subset of ES6+ into ES5 (the subset of ES6+ that transpiles well without causing bloat, which excludes some features like generators), whereas this seems to be focused on more modern targets. Technically there are a few interesting differences under the hood, such as Sucrase not wasting time generating a full-blown AST (which Bublé needs to do in order to handle things like block scoping). Anyway, I won't waffle on any further as I'm sure Alan can explain the differences better — I just wanted to chime in to say that I'm excited about Sucrase. I've written a [Rollup plugin][1] for it, and I'm planning to use it with my TypeScript projects. [1]: https://github.com/rollup/rollup-plugin-sucrase https://github.com/rollup/rollup-plugin-sucrase
- alangpierce 8y ago
  Adding to what Rich said, Sucrase takes a fair amount of inspiration from Bublé, and has a similar high-level approach of "pick a well-scoped subset of what Babel solves, and focus on doing that really well". In Bublé's case, that's compiling many (but not all) ES6 features down to ES5. In Sucrase's case, that's compiling language extensions (plus some upcoming JS features like class fields) to modern JS. They solve different problems, so you would never pick between the two. Architecturally, they both skip a bunch of work that Babel does (AST transformations and AST formatting), but Sucrase is a bit more ambitious. Bublé does a normal parse step and then does replacements using magic-string. The original hope was that Sucrase would be able to skip parsing and just tokenize the JS and do all of the necessary transforms from the tokens. It turns out that that's basically impossible, but Sucrase still tries to find a middle ground where it does something reminiscent of parsing but doesn't need to produce an AST at the end. It's still able to resolve variable scopes (which is needed for the TypeScript and imports transforms) by providing an array of scopes in addition to an array of tokens, and it provides enough useful context in the tokens to put the pieces together if necessary.
gymshoes 8y ago
Would targeting modern Js runtimes yield faster performance in browser as well? I am guessing some loading time would be improved if code converted from ES2105 to ES5 is not required

88 points by alangpierce 8y ago

fold_left 8y ago

I could imagine using the jest plugin, could be useful as a presumably faster alternative to babel-jest.

alangpierce 8y ago
That's the hope! To be honest, I haven't worked with Jest a lot, and the plugin is sort of a first pass to get things working on the Apollo codebase (which uses Jest), so e.g. it may not be as smart as babel-jest for things like caching. Contributions welcome, though! My specific use case so far has been running Mocha tests on a large codebase, so it's replacing babel-register (which uses a cache) with sucrase/register (which doesn't). For that use case, it seems to significantly speed up test startup time with a cold cache and somewhat speed up startup time with a warm cache. Just loading and saving the babel-register cache takes a fair amount of time, at least in Babel 6. It's good to keep caching in mind when thinking about this stuff. In many cases, Sucrase won't be faster than loading results from cache. Sucrase helps with initial startup time and avoids the cache fragility issues that I've seen with lots of caching systems. And, of course, you could cache Sucrase results and get something at least as fast as cached Babel.

btown 8y ago

Is there a good way to use this from Webpack? Edit: https://github.com/alangpierce/sucrase/blob/master/integrations/webpack-loader/ https://github.com/alangpierce/sucrase/blob/master/integrati...

alangpierce 8y ago
Yep, looks like you found the plugin! Note that if you're using Webpack 4, you won't need webpack-object-rest-spread-plugin anymore, since it uses the newest Acorn, which should handle object rest/spread syntax without needing a plugin. Annoyingly, Webpack still spends a lot of time parsing the JS files with its own parser, so you cut down on the transpile time but still need to do a relatively slow parse of all files. I've thought about making Webpack (or similar) use a fast parser like the one Sucrase uses, although it certainly seems like it would be a lot of work.
- the_duke 8y ago
  Do you have any benchmarks showing how much time is spent in on webpack plumbing vs babel, comparing to sucrase? As in, how much does this actually buy you in a real world scenario?
  - alangpierce 8y ago
    I'm not very optimistic about webpack build times getting dramatically better with Sucrase in its current state. I've seen more clear benefits when running tests using sucrase/register. That said, I think it depends quite a bit on your webpack config. Certainly, if you run Babel single-threaded and don't cache the output, then you'll probably get a significant benefit, but that's likely not a realistic scenario. Some specific numbers I just measured for my codebase at work: Babel/TypeScript, cold cache: 66 seconds Babel/TypeScript, warm cache: 55 seconds Sucrase, cold cache: 52 seconds Sucrase, warm cache: 49 seconds This is on a codebase of about 400,000 lines of code (some JS compiled with Babel, some TypeScript compiled with tsc). Transpilation is parallelized using happypack (and I'm running it on a 4-core machine), but webpack parsing/processing is all single-threaded, so it ends up taking more time. I've done a little prototyping on using a Sucrase-like approach to speed up webpack (most of it is indeed parse time), but it would certainly be a project to get it working in practice. With a warm cache, it's not running sucrase/babel/typescript at all. I think the running time difference there is because Babel and TypeScript are emitting ES5, which is a little more code for webpack to parse. Probably a more correct comparison would be to configure Babel and TypeScript to target newer JS, but these should all be seen as rough numbers anyway.

nicoburns 8y ago

Does anyone have any idea how much performance is being left on the table by writing these compile-to-js compilers in JS? Could we get another 10x improvement by writing the transpilers in something like Go/Rust/C++? Frontend compile performance is beginning to get painful with large codebases...

tjallingt 8y ago
Some examples of "transpilers" targeting Javascript but writen in other languages: https://github.com/google/closure-compiler https://github.com/google/closure-compiler https://github.com/BuckleScript/bucklescript https://github.com/BuckleScript/bucklescript https://github.com/fastpack/fastpack https://github.com/fastpack/fastpack Seems like on the average it does offer a boost in performance. And there is some aditional work providing javascript parsers for rust (which you could build tools like babel on top of): https://github.com/dherman/esprit https://github.com/dherman/esprit
- kevinb7 8y ago
  I think part of BuckleScript's speed may come from the source language, OCaml, being easier to parse. There may also be fewer passes necessary to transform OCaml to JavaScript.
  - chenglou 8y ago
    BuckleScript using Reason (with a similar syntax than JS, slower parser than vanilla OCaml) is still one of the fastest around. Mostly a case of very careful and dedicated engineering here (I work on Reason with the author of BuckleScript). Additionally, we try to work smart and delegate most of the build process to Ninja, which itself is one of the fastests around. But I believe the topic here was about runtime performance of using a language to compile JS, not about the build speed of working in that language itself. In which case you’ll still get some wins writing a JS toolchain in BuckleScript (compiled to JS), just from the JiT-friendliness of the BuckleScript JS output. But realistically, you’d be compiling to native OCaml through the same codebase. We did see a 10-25x perf jump from converting a part of a Babel pipeline to native OCaml. I mean, these languages are basically designed over decades with AST manipulation in mind, so that’s not surprising.

fiatjaf 8y ago

Thank you for making this. Babel really needs some alternatives. It would be great to have a Browserify plugin (I'm not asking for anything, just speaking my thoughts).

alangpierce 8y ago
Yep, hopefully this will motivate more of a performance focus in general. To be clear, Sucrase's parser is a slimmed-down fork of Babel's parser, so Sucrase wouldn't be possible without Babel. Also, Babel has a much broader scope as a pluginizable code transformation tool, so it's much better suited for things like prototyping future language features and doing other nontrivial transformations. Regarding Browserify support, PRs are welcome. :-) https://github.com/alangpierce/sucrase/tree/master/integrations https://github.com/alangpierce/sucrase/tree/master/integrati...

bpierre 8y ago

How does it compare to Bublé? https://buble.surge.sh/guide/ https://buble.surge.sh/guide/

rich_harris 8y ago
Bublé creator here. The obvious big difference is that this supports TypeScript and Flow, which Bublé doesn't. Bublé is designed to compile a subset of ES6+ into ES5 (the subset of ES6+ that transpiles well without causing bloat, which excludes some features like generators), whereas this seems to be focused on more modern targets. Technically there are a few interesting differences under the hood, such as Sucrase not wasting time generating a full-blown AST (which Bublé needs to do in order to handle things like block scoping). Anyway, I won't waffle on any further as I'm sure Alan can explain the differences better — I just wanted to chime in to say that I'm excited about Sucrase. I've written a [Rollup plugin][1] for it, and I'm planning to use it with my TypeScript projects. [1]: https://github.com/rollup/rollup-plugin-sucrase https://github.com/rollup/rollup-plugin-sucrase
alangpierce 8y ago
Adding to what Rich said, Sucrase takes a fair amount of inspiration from Bublé, and has a similar high-level approach of "pick a well-scoped subset of what Babel solves, and focus on doing that really well". In Bublé's case, that's compiling many (but not all) ES6 features down to ES5. In Sucrase's case, that's compiling language extensions (plus some upcoming JS features like class fields) to modern JS. They solve different problems, so you would never pick between the two. Architecturally, they both skip a bunch of work that Babel does (AST transformations and AST formatting), but Sucrase is a bit more ambitious. Bublé does a normal parse step and then does replacements using magic-string. The original hope was that Sucrase would be able to skip parsing and just tokenize the JS and do all of the necessary transforms from the tokens. It turns out that that's basically impossible, but Sucrase still tries to find a middle ground where it does something reminiscent of parsing but doesn't need to produce an AST at the end. It's still able to resolve variable scopes (which is needed for the TypeScript and imports transforms) by providing an array of scopes in addition to an array of tokens, and it provides enough useful context in the tokens to put the pieces together if necessary.

gymshoes 8y ago

Would targeting modern Js runtimes yield faster performance in browser as well? I am guessing some loading time would be improved if code converted from ES2105 to ES5 is not required