3 ms·

Sucrase is written in JS and boasts the highest line throughput of any competing transpiler https://github.com/alangpierce/sucrase https://github.com/alangpier

Sucrase is written in JS and boasts the highest line throughput of any competing transpiler https://github.com/alangpierce/sucrase https://github.com/alangpierce/sucrase Time Speed Sucrase 0.57 seconds 636975 lines per second swc 1.19 seconds 304526 lines per second esbuild 1.45 seconds 248692 lines per second TypeScript 8.98 seconds 40240 lines per second Babel 9.18 seconds 39366 lines per second

merb 4y ago
but the benchmark is stupid: https://github.com/alangpierce/sucrase/blob/main/benchmark/benchmark.ts https://github.com/alangpierce/sucrase/blob/main/benchmark/b... > Like all JavaScript code run in V8, Sucrase runs more slowly at first, then gets faster as the just-in-time compiler applies more optimizations. From a rough measurement, Sucrase is about 2x faster after running for 3 seconds than after running for 1 second. swc (written in Rust) and esbuild (written in Go) don't have this effect because they're pre-compiled, so comparing them with Sucrase gets significantly different results depending on how large of a codebase is being tested and whether each compiler is allowed a "warm up" period before the benchmark is run. (worse it disables esbuild and swc's multi-threading... https://github.com/alangpierce/sucrase/blob/main/benchmark/benchmark.ts#L257 https://github.com/alangpierce/sucrase/blob/main/benchmark/b... https://github.com/alangpierce/sucrase/blob/main/benchmark/benchmark.ts#L231 https://github.com/alangpierce/sucrase/blob/main/benchmark/b...) fake it till ya make it. it's like saying "if I disable everything and wait for 5 minutes it's faster"
- RyanCavanaugh 4y ago
  I don't see how this is a rebuttal to the claim. 636,975 is more than 2x 304,526, so assuming the quoted paragraph is correct, sucrase is still the highest-throughput transpiler even during its warm-up phase. Probably this isn't true for the first 100 milliseconds of execution during the very first warm-up stages, but if the transpile phase is that short, it's basically irrelevant anyway.
  - merb 4y ago
    the warm-up phase is the whole reason why esbuild and swc exists btw. and also the sample is basically a small hello world. any real project would do a little more and the jit would not optimize as much. also 2x 300k is not really the way how multi-threading/concurrency/parallelism works... especially not golang, which should not be run with GOMAXPROCS=1....
    - cxr 4y ago
      As far as I can follow your argument, it seems to be that the native tools perform better on the small inputs (that only ever take a second or three to complete), and that the creators ("maintainers"?) of Sucrase have their thumb on the scale or something, since they emphasize in their results the very large inputs that will involve longer running jobs where their tool has had a chance to warm up. In other words, the tools you're defending only do better when it doesn't really matter, and the tool you're downplaying outperforms the competition on the jobs where it actually does matter. If anything is backwards (or "stupid", as you put it) then that seems to be it. Maybe I'm misunderstanding. In that case, please provide a better benchmark.
- alangpierce 4y ago
  Hi, Sucrase author here. To be clear, the benchmark in the README does not allow JIT warm-up. The Sucrase numbers would be better if it did. From testing just now (add `warmUp: true` to `benchmarkJest`), Sucrase is a little over 3x faster than swc if you allow warm-up, but it seemed unfair to disregard warm-up for the comparison in the README. It's certainly fair to debate whether 360k lines of code is a realistic codebase size for the benchmark; the higher-scale the test case, the better Sucrase looks. > worse it disables esbuild and swc's multi-threading At some point I'm hoping to update the README benchmark to run all tools in parallel, which should be more convincing despite the added variability: https://github.com/alangpierce/sucrase/issues/730 https://github.com/alangpierce/sucrase/issues/730 . In an ideal environment, the results are pretty much the same as a per-core benchmark, but I do expect that Node's parallelism overhead and the JIT warm-up cost across many cores would make Sucrase less competitive than the current numbers.
KingOfCoders 4y ago
"Tools run in single-threaded mode without warm-up." Ithought it was 2022. I have a 12 core machine and my next machine will probably have 22 cores. But I'm amazed, transpiling 636975 lines in <1 second is nice. [Edit] What I do not understand is "Sucrase does not check your code for errors." So it's not a type checker? Or does it check type errors? Why would I used it for Typescript when the reason to use TS is to add types to JS to prevent errors? Is this more like Rust check for continous work and then use tsc from time to time to check for errors?
- alangpierce 4y ago
  Like any transpiler, Sucrase can be run in parallel by having the build system send different files to different threads/processes. Sucrase itself it more of a primitive, just a plain function from input code to output code. > What I do not understand is "Sucrase does not check your code for errors." So it's not a type checker? That's correct, Sucrase, swc, esbuild, and Babel are all just transpilers that transform TypeScript syntax into plain JavaScript (plus other transformations). The usual way you set things up is to use the transpiler to run and build your TS code, and you separately run type checking using the official TypeScript package.

merb 4y ago

but the benchmark is stupid: https://github.com/alangpierce/sucrase/blob/main/benchmark/benchmark.ts https://github.com/alangpierce/sucrase/blob/main/benchmark/b... > Like all JavaScript code run in V8, Sucrase runs more slowly at first, then gets faster as the just-in-time compiler applies more optimizations. From a rough measurement, Sucrase is about 2x faster after running for 3 seconds than after running for 1 second. swc (written in Rust) and esbuild (written in Go) don't have this effect because they're pre-compiled, so comparing them with Sucrase gets significantly different results depending on how large of a codebase is being tested and whether each compiler is allowed a "warm up" period before the benchmark is run. (worse it disables esbuild and swc's multi-threading... https://github.com/alangpierce/sucrase/blob/main/benchmark/benchmark.ts#L257 https://github.com/alangpierce/sucrase/blob/main/benchmark/b... https://github.com/alangpierce/sucrase/blob/main/benchmark/benchmark.ts#L231 https://github.com/alangpierce/sucrase/blob/main/benchmark/b...) fake it till ya make it. it's like saying "if I disable everything and wait for 5 minutes it's faster"

RyanCavanaugh 4y ago
I don't see how this is a rebuttal to the claim. 636,975 is more than 2x 304,526, so assuming the quoted paragraph is correct, sucrase is still the highest-throughput transpiler even during its warm-up phase. Probably this isn't true for the first 100 milliseconds of execution during the very first warm-up stages, but if the transpile phase is that short, it's basically irrelevant anyway.
- merb 4y ago
  the warm-up phase is the whole reason why esbuild and swc exists btw. and also the sample is basically a small hello world. any real project would do a little more and the jit would not optimize as much. also 2x 300k is not really the way how multi-threading/concurrency/parallelism works... especially not golang, which should not be run with GOMAXPROCS=1....
  - cxr 4y ago
    As far as I can follow your argument, it seems to be that the native tools perform better on the small inputs (that only ever take a second or three to complete), and that the creators ("maintainers"?) of Sucrase have their thumb on the scale or something, since they emphasize in their results the very large inputs that will involve longer running jobs where their tool has had a chance to warm up. In other words, the tools you're defending only do better when it doesn't really matter, and the tool you're downplaying outperforms the competition on the jobs where it actually does matter. If anything is backwards (or "stupid", as you put it) then that seems to be it. Maybe I'm misunderstanding. In that case, please provide a better benchmark.
alangpierce 4y ago
Hi, Sucrase author here. To be clear, the benchmark in the README does not allow JIT warm-up. The Sucrase numbers would be better if it did. From testing just now (add `warmUp: true` to `benchmarkJest`), Sucrase is a little over 3x faster than swc if you allow warm-up, but it seemed unfair to disregard warm-up for the comparison in the README. It's certainly fair to debate whether 360k lines of code is a realistic codebase size for the benchmark; the higher-scale the test case, the better Sucrase looks. > worse it disables esbuild and swc's multi-threading At some point I'm hoping to update the README benchmark to run all tools in parallel, which should be more convincing despite the added variability: https://github.com/alangpierce/sucrase/issues/730 https://github.com/alangpierce/sucrase/issues/730 . In an ideal environment, the results are pretty much the same as a per-core benchmark, but I do expect that Node's parallelism overhead and the JIT warm-up cost across many cores would make Sucrase less competitive than the current numbers.

KingOfCoders 4y ago

"Tools run in single-threaded mode without warm-up." Ithought it was 2022. I have a 12 core machine and my next machine will probably have 22 cores. But I'm amazed, transpiling 636975 lines in <1 second is nice. [Edit] What I do not understand is "Sucrase does not check your code for errors." So it's not a type checker? Or does it check type errors? Why would I used it for Typescript when the reason to use TS is to add types to JS to prevent errors? Is this more like Rust check for continous work and then use tsc from time to time to check for errors?

alangpierce 4y ago
Like any transpiler, Sucrase can be run in parallel by having the build system send different files to different threads/processes. Sucrase itself it more of a primitive, just a plain function from input code to output code. > What I do not understand is "Sucrase does not check your code for errors." So it's not a type checker? That's correct, Sucrase, swc, esbuild, and Babel are all just transpilers that transform TypeScript syntax into plain JavaScript (plus other transformations). The usual way you set things up is to use the transpiler to run and build your TS code, and you separately run type checking using the official TypeScript package.