8 ms·

Turbopack Performance Benchmarks

99 points by palmdeezy 4y ago

pyrolistical 4y ago
Good on them make a correction
- swyx 4y ago
  how is it a correction? the main controversy was the 10x and 700x faster thing... the 0.01s thing was just a typo really
  - ZephyrBlu 4y ago
    From Evan You: > There were number rounding issues in the original numbers for the 1k component case - Turbopack's 15ms was rounded down to 0.01s while Vite's 87ms was rounded up to 0.09s. This further got marketed as a 10x advantage when the original numbers were close to 6x. I guess they 'typo-ed' Vite's numbers as well. https://github.com/yyx990803/vite-vs-next-turbo-hmr/discussions/8 https://github.com/yyx990803/vite-vs-next-turbo-hmr/discussi...
    - jridgewell 4y ago
      `0.015.toFixed(2) === "0.01"`, and `0.087.toFixed(2) === "0.09"`
      - zoover2020 4y ago
        At this level of detail it _is_ super misleading however to say "a 10x improvement" whilst in reality it's more like 6x
Bilal_io 4y ago
I am very excited for Turbopack. I hope to see this considered by the Angular team. Angular is tightly coupled with webpack, however, they've been experimenting with ESBuild, which is included in Angular 14. Vite has been tested by an Angular team member and reported some promising results. I just hope they don't tightly couple the implementation with ESBuild and fall into the same issue again. Back to Turbopack: > Turbopack is up to 10x and 700x faster than existing approaches. The 700x speed gain compared to webpack is for dev mode changing 30000 files at once.. I understand that it scales very well. But is it realistic to boast about unrealistic scenarios? It erks me.
- tough 4y ago
  Marketing always overrides engineering on final copy
- 8n4vidtmkvmk 4y ago
  is it that unrealistic? dev mode is what i care most about. its ok if deploys are a little slower. 30k files probably means editing 1 file which triggers rebuilding the whole bundle... if that includes node modules it can probably hit 30k pretty easily
  - wonnage 4y ago
    Or if you just start with an empty cache, or if you come back from vacation and pull several weeks worth of changes… certainly a real use case
  - Bilal_io 4y ago
    The benchmark is measuring HMR specifically. You've already dev-served your app, now you make a change... Which builder is faster? It's important to get all the speed gain we can, Angular (webpack) is really slow at rebuilding when you save changes, this would be a life-saver, but no dev is ever watching 30k files at once.
encryptluks2 4y ago
Has this even been released? I went to look for a release on the repo and website and only saw instructions for using it with Next js. Might as well call it Next.js if it can't easily be used with anything else. The only way I found to build it was from the master branch. The latest tip in master is not a release.
- ice_cold_etoh 4y ago
  > As of today, Turbopack can be used in Next.js v13. In the future we will be releasing a standalone CLI, plugin API, and support for other frameworks such as Svelte and Vue. It is not ready for general usage for the moment, and it is still in alpha stage that isn't production ready and missing some crucial function such as PostCSS support. Nonetheless it is still exciting to see all the new innovation and healthy competition around JS bundler ecosystem.
  - leerob 4y ago
    Correct, still very early. While the happy path right now is through Next.js, there's a hacky workaround for general use outside Next.js. As we move forward in stability, we'll be publishing more guidance for using it as a general bundler with any framework. Next.js is helping dogfood Turbopack prior to that, while in alpha.
    - encryptluks2 4y ago
      So then calling it a release is misleading. I don't mind it being called a component or module of Next.js in the meantime, but release makes it seem like it is standalone.
liuliu 4y ago
> Turbopack and Next.js 13.0.1 are out addressing a regression that snuck in prior to public release and after the initial benchmarks were taken. We also fixed an incorrect rounding bug on our website (0.01s → 15ms). We appreciate Evan You's work that helped us identify and correct this. Does this mean they don't have a CI to run benchmarks to gate regressions?
jamescostian 4y ago
Here's the bulk of the code used to generate the code they're building: https://github.com/vercel/turbo/blob/main/crates/turbopack-create-test-app/src/test_app_builder.rs https://github.com/vercel/turbo/blob/main/crates/turbopack-c... They're building basically the same thing over and over again. This surprised me, given their intro post: https://vercel.com/blog/turbopack https://vercel.com/blog/turbopack Relevant quote: > Turbopack is built on Turbo: an open-source, incremental memoization framework for Rust. Turbo can cache the result of any function in the program. When the program is run again, functions won't re-run unless their inputs have changed. This granular architecture enables your program to skip large amounts of work, at the level of the function. I'd be curious to see if a real-world app (or even one generated with more variety in components) showed comparable performance numbers
- tough 4y ago
  turborepo was the first -turbo- on the family to use this strategy to cache a lot of dependencies on building monorepos. I think they might be applying the same strategy everywhere by decoupling -turbo- from the implementations that use the strategy at each layer of the stack. Expect a -turbo- script doing this and the first real complete replacement for typescript compilation, probably in go, not rust, tho!
  - HJain13 4y ago
    > Expect a -turbo- script doing this and the first real complete replacement for typescript compilation, probably in go, not rust, tho! There is stc which is for now just a PoC (from the developer of swc who is on Vercel's payroll) which is shifting from Go to Rust
Touche 4y ago
Weird that rebuild is absent in these benchmarks.
- leerob 4y ago
  Turbopack is alpha – currently only for local dev (e.g. dev server with hot reloading). Support for production builds hasn't landed yet. Once that lands, we'll add some benchmarks there as well.
colinchartier 4y ago
Another benchmark by maintainer of Vue and Vite, Evan You: https://github.com/yyx990803/vite-vs-next-turbo-hmr https://github.com/yyx990803/vite-vs-next-turbo-hmr He seems to imply that Turbopack is very close in performance to existing tooling like Vite in his benchmark, and not 10x better in common cases Edit: his thoughts as of this hour: https://github.com/yyx990803/vite-vs-next-turbo-hmr/discussions/8 https://github.com/yyx990803/vite-vs-next-turbo-hmr/discussi...
- rektide 4y ago
  Close but faster. And in some cases ("leaf") faster. The way you phrase it makes it sound like Vite is still faster & Turbopack is getting there. These tweets seem to indicate Turbopack is faster.
  - rk06 4y ago
    Turbopack is indeed faster, which makes it more confusing. Like why not use the actual verifiable "apple to apple" benchmark results? Even in this article, they do not refer to that acknowledgement, which because it was from a competiting OSS author, would have a lot more weight
- leerob 4y ago
  Evan is credited in the post (this tweet is from three days ago). > Turbopack and Next.js 13.0.1 are out addressing a regression that snuck in prior to public release and after the initial benchmarks were taken. We also fixed an incorrect rounding bug on our website (0.01s → 15ms). We appreciate Evan You's work that helped us identify and correct this.
  - cjblomqvist 4y ago
    Yes, but the post doesn't comment on all his notes (actually most are not mentioned). Specifically, his note on swc should really be brought up commented I think (after all, they're the ones who compared themselves with Vite, not the other way around).
- deleted 4y ago
QuadrupleA 4y ago
As a dev who avoids build steps entirely in web stacks, hearing that 1.1 sec startup is excellent, or 10-700x faster than the norm seems strange - I've gotten so used to stuff being instant. If you haven't tried coding a little closer to the "metal" (vanilla js/css/html) I definitely recommend it. Browsers give you a lot out of the box now, and life without builds is sweet - fast iteration time, perfect in-browser dev tool support, and vastly reduced codebase complexity.
- cosmotic 4y ago
  Though I'm totally with you on that approach, I'm unaware of native/vanilla tools that offer hot reloading, which is pretty awesome.
  - getcrunk 4y ago
    If you are not bundling your code to begin with which you wouldn't be without a build step; I think there are ways to reload when a file is changed. Like an extension that speaks to a file system watcher.
    - fenomas 4y ago
      Hot [module] reloading refers to dynamically swapping in an updated module or asset without reloading the page. When building nontrivial stuff HMR is pretty huge for productivity. E.g. I do a lot of procedural music via WebAudio, and being able to modify the code and hear the results as the song continues to play is obviously pretty useful.
- earthboundkid 4y ago
  I’ve done both. Vanilla is fine, but people use frameworks for a reason. They really are easier, particularly for inexperienced developers, but in general for experienced developers as well. I don’t see any reason to be snooty about not using a tool because you want to keep things simple.
- nop_slide 4y ago
  Alpine JS plus parcel has been really fun
quaunaut 4y ago
Something that isn't clear to me: Why, exactly, is it so much faster than even its own build tool(swc)? In other words, what's speeding up here isn't build speed, but the ability to download the updated changes. Right? Or am I wrong?
deleted 4y ago
[deleted]
8n4vidtmkvmk 4y ago
will this work with existing webpack configs? and with babel? i need babel..
- julianlam 4y ago
  No, right now I believe it's tied to NextJS
  - 8n4vidtmkvmk 4y ago
    Right, but is webpack compatibility a goal?
newbieuser 4y ago
it doesn't make much sense to compare a tool that only works with nextjs with webpack and vite

99 points by palmdeezy 4y ago

pyrolistical 4y ago

Good on them make a correction

swyx 4y ago
how is it a correction? the main controversy was the 10x and 700x faster thing... the 0.01s thing was just a typo really
- ZephyrBlu 4y ago
  From Evan You: > There were number rounding issues in the original numbers for the 1k component case - Turbopack's 15ms was rounded down to 0.01s while Vite's 87ms was rounded up to 0.09s. This further got marketed as a 10x advantage when the original numbers were close to 6x. I guess they 'typo-ed' Vite's numbers as well. https://github.com/yyx990803/vite-vs-next-turbo-hmr/discussions/8 https://github.com/yyx990803/vite-vs-next-turbo-hmr/discussi...
  - jridgewell 4y ago
    `0.015.toFixed(2) === "0.01"`, and `0.087.toFixed(2) === "0.09"`
    - zoover2020 4y ago
      At this level of detail it _is_ super misleading however to say "a 10x improvement" whilst in reality it's more like 6x

Bilal_io 4y ago

I am very excited for Turbopack. I hope to see this considered by the Angular team. Angular is tightly coupled with webpack, however, they've been experimenting with ESBuild, which is included in Angular 14. Vite has been tested by an Angular team member and reported some promising results. I just hope they don't tightly couple the implementation with ESBuild and fall into the same issue again. Back to Turbopack: > Turbopack is up to 10x and 700x faster than existing approaches. The 700x speed gain compared to webpack is for dev mode changing 30000 files at once.. I understand that it scales very well. But is it realistic to boast about unrealistic scenarios? It erks me.

tough 4y ago
Marketing always overrides engineering on final copy
8n4vidtmkvmk 4y ago
is it that unrealistic? dev mode is what i care most about. its ok if deploys are a little slower. 30k files probably means editing 1 file which triggers rebuilding the whole bundle... if that includes node modules it can probably hit 30k pretty easily
- wonnage 4y ago
  Or if you just start with an empty cache, or if you come back from vacation and pull several weeks worth of changes… certainly a real use case
- Bilal_io 4y ago
  The benchmark is measuring HMR specifically. You've already dev-served your app, now you make a change... Which builder is faster? It's important to get all the speed gain we can, Angular (webpack) is really slow at rebuilding when you save changes, this would be a life-saver, but no dev is ever watching 30k files at once.

encryptluks2 4y ago

Has this even been released? I went to look for a release on the repo and website and only saw instructions for using it with Next js. Might as well call it Next.js if it can't easily be used with anything else. The only way I found to build it was from the master branch. The latest tip in master is not a release.

ice_cold_etoh 4y ago
> As of today, Turbopack can be used in Next.js v13. In the future we will be releasing a standalone CLI, plugin API, and support for other frameworks such as Svelte and Vue. It is not ready for general usage for the moment, and it is still in alpha stage that isn't production ready and missing some crucial function such as PostCSS support. Nonetheless it is still exciting to see all the new innovation and healthy competition around JS bundler ecosystem.
- leerob 4y ago
  Correct, still very early. While the happy path right now is through Next.js, there's a hacky workaround for general use outside Next.js. As we move forward in stability, we'll be publishing more guidance for using it as a general bundler with any framework. Next.js is helping dogfood Turbopack prior to that, while in alpha.
  - encryptluks2 4y ago
    So then calling it a release is misleading. I don't mind it being called a component or module of Next.js in the meantime, but release makes it seem like it is standalone.

liuliu 4y ago

> Turbopack and Next.js 13.0.1 are out addressing a regression that snuck in prior to public release and after the initial benchmarks were taken. We also fixed an incorrect rounding bug on our website (0.01s → 15ms). We appreciate Evan You's work that helped us identify and correct this. Does this mean they don't have a CI to run benchmarks to gate regressions?

jamescostian 4y ago

Here's the bulk of the code used to generate the code they're building: https://github.com/vercel/turbo/blob/main/crates/turbopack-create-test-app/src/test_app_builder.rs https://github.com/vercel/turbo/blob/main/crates/turbopack-c... They're building basically the same thing over and over again. This surprised me, given their intro post: https://vercel.com/blog/turbopack https://vercel.com/blog/turbopack Relevant quote: > Turbopack is built on Turbo: an open-source, incremental memoization framework for Rust. Turbo can cache the result of any function in the program. When the program is run again, functions won't re-run unless their inputs have changed. This granular architecture enables your program to skip large amounts of work, at the level of the function. I'd be curious to see if a real-world app (or even one generated with more variety in components) showed comparable performance numbers

tough 4y ago
turborepo was the first -turbo- on the family to use this strategy to cache a lot of dependencies on building monorepos. I think they might be applying the same strategy everywhere by decoupling -turbo- from the implementations that use the strategy at each layer of the stack. Expect a -turbo- script doing this and the first real complete replacement for typescript compilation, probably in go, not rust, tho!
- HJain13 4y ago
  > Expect a -turbo- script doing this and the first real complete replacement for typescript compilation, probably in go, not rust, tho! There is stc which is for now just a PoC (from the developer of swc who is on Vercel's payroll) which is shifting from Go to Rust

Touche 4y ago

Weird that rebuild is absent in these benchmarks.

leerob 4y ago
Turbopack is alpha – currently only for local dev (e.g. dev server with hot reloading). Support for production builds hasn't landed yet. Once that lands, we'll add some benchmarks there as well.

colinchartier 4y ago

Another benchmark by maintainer of Vue and Vite, Evan You: https://github.com/yyx990803/vite-vs-next-turbo-hmr https://github.com/yyx990803/vite-vs-next-turbo-hmr He seems to imply that Turbopack is very close in performance to existing tooling like Vite in his benchmark, and not 10x better in common cases Edit: his thoughts as of this hour: https://github.com/yyx990803/vite-vs-next-turbo-hmr/discussions/8 https://github.com/yyx990803/vite-vs-next-turbo-hmr/discussi...

rektide 4y ago
Close but faster. And in some cases ("leaf") faster. The way you phrase it makes it sound like Vite is still faster & Turbopack is getting there. These tweets seem to indicate Turbopack is faster.
- rk06 4y ago
  Turbopack is indeed faster, which makes it more confusing. Like why not use the actual verifiable "apple to apple" benchmark results? Even in this article, they do not refer to that acknowledgement, which because it was from a competiting OSS author, would have a lot more weight
leerob 4y ago
Evan is credited in the post (this tweet is from three days ago). > Turbopack and Next.js 13.0.1 are out addressing a regression that snuck in prior to public release and after the initial benchmarks were taken. We also fixed an incorrect rounding bug on our website (0.01s → 15ms). We appreciate Evan You's work that helped us identify and correct this.
- cjblomqvist 4y ago
  Yes, but the post doesn't comment on all his notes (actually most are not mentioned). Specifically, his note on swc should really be brought up commented I think (after all, they're the ones who compared themselves with Vite, not the other way around).
deleted 4y ago

QuadrupleA 4y ago

As a dev who avoids build steps entirely in web stacks, hearing that 1.1 sec startup is excellent, or 10-700x faster than the norm seems strange - I've gotten so used to stuff being instant. If you haven't tried coding a little closer to the "metal" (vanilla js/css/html) I definitely recommend it. Browsers give you a lot out of the box now, and life without builds is sweet - fast iteration time, perfect in-browser dev tool support, and vastly reduced codebase complexity.

cosmotic 4y ago
Though I'm totally with you on that approach, I'm unaware of native/vanilla tools that offer hot reloading, which is pretty awesome.
- getcrunk 4y ago
  If you are not bundling your code to begin with which you wouldn't be without a build step; I think there are ways to reload when a file is changed. Like an extension that speaks to a file system watcher.
  - fenomas 4y ago
    Hot [module] reloading refers to dynamically swapping in an updated module or asset without reloading the page. When building nontrivial stuff HMR is pretty huge for productivity. E.g. I do a lot of procedural music via WebAudio, and being able to modify the code and hear the results as the song continues to play is obviously pretty useful.
earthboundkid 4y ago
I’ve done both. Vanilla is fine, but people use frameworks for a reason. They really are easier, particularly for inexperienced developers, but in general for experienced developers as well. I don’t see any reason to be snooty about not using a tool because you want to keep things simple.
nop_slide 4y ago
Alpine JS plus parcel has been really fun

quaunaut 4y ago

Something that isn't clear to me: Why, exactly, is it so much faster than even its own build tool(swc)? In other words, what's speeding up here isn't build speed, but the ability to download the updated changes. Right? Or am I wrong?

deleted 4y ago

[deleted]

8n4vidtmkvmk 4y ago

will this work with existing webpack configs? and with babel? i need babel..

julianlam 4y ago
No, right now I believe it's tied to NextJS
- 8n4vidtmkvmk 4y ago
  Right, but is webpack compatibility a goal?

newbieuser 4y ago

it doesn't make much sense to compare a tool that only works with nextjs with webpack and vite