9 ms·

10k Bounty for Rewriting Prettier in Rust

89 points by hardwaregeek 3y ago

sneed_chucker 3y ago
Sick. Hope someone does it.
adiabatty 3y ago
This would probably be a good thing on net, but how would plugins for Rust work? I use <https://github.com/NiklasPor/prettier-plugin-go-template https://github.com/NiklasPor/prettier-plugin-go-template> a bit and I'm not sure what one would have to do to make a plugin for a Rust binary.
- veidr 3y ago
  Well they could work any way the author wanted, but if it were me, I'd make the plugins WASM. That's how my favorite code formatter (dprint) does it. For that matter though, it's how I would implement just about any plugin, these days, except those that need to exchange large info rapidly (e.g. an editor plugin that needs to operate on every key press). It's dead easy to implement on the app side, and plugin authors can use an increasing number of higher-level languages to build plugins.
  - mtsr 3y ago
    And on top of that it’s also easy to mostly guarantee the plug-in can only access the APIs you give it. Which is quite nice for third party code.
  - CWIZO 3y ago
    Would love to learn more if you have any resources available?
  - rob74 3y ago
    ...which would have the advantage that plugin authors could also use other languages that compile to WASM.
  - crabmusket 3y ago
    > except those that need to exchange large info rapidly I've seen attempts at zero-copy memory sharing between JS and WASM, but I'm not sure if the same approach would allow different plugins to share memory and communicate via e.g. Cap'n Proto? https://observablehq.com/@kylebarron/zero-copy-apache-arrow-with-webassembly https://observablehq.com/@kylebarron/zero-copy-apache-arrow-...
lacker 3y ago
Looks like the bounty is at 20k now with contribution from @rauchg https://twitter.com/rauchg/status/1723400569392656771 https://twitter.com/rauchg/status/1723400569392656771
downvotetruth 3y ago
"I care that people can have a faster version of prettier, not how it's implemented." & "... for any project written in Rust" are contradictory.
- topspin 3y ago
  That was in the context of a question about whether external crates may be used. Rust is clearly implied.
maxloh 3y ago
Prettier runs on a single thread only. Running it on each thread could make it significantly faster [0]. [0]: https://github.com/microsoft/parallel-prettier https://github.com/microsoft/parallel-prettier
- Alifatisk 3y ago
  The performance gain is massive!
zImPatrick 3y ago
Why does everything need to be written in rust these days?
- ramon156 3y ago
  As a rust dev I'm just as perplexed. I've seen preprocessors and stuff like tailwind being rewritten in Rust aswell. Why though (
  - mtsr 3y ago
    Some dev workflow tools really benefit from being very fast. And parsing and ast building and the like are actually things that rust is quite good for. That doesn’t mean it couldn’t be done in other languages, but it’s a weighing of trade-offs. And of course, groups of people, such as developers on socials, are quite sensitive to popularity.
    - vsnf 3y ago
      > And parsing and ast building and the like are actually things that rust is quite good for. Having built a compiler and typechecker in Rust, I don't know if I'd say that it was "quite good" for it. Going back, I'd still do it again, but it wasn't exactly a cakewalk. Rust's lifetime rules and visceral hatred of referential structs did not make my life easy at all. I definitely took the easy way out with a lot of string copies too.
    - dbrgn 3y ago
      Just as an example: Our eslint checks take 2 minutes to complete. Improving this would considerably improve developer productivity.
  - fullspectrumdev 3y ago
    If it speeds up a build/CI pipeline it’s a worthwhile investment for a lot of folks
    - RamblingCTO 3y ago
deleted 3y ago
[deleted]
pyrolistical 3y ago
It would be problem just as fast if rewritten in any compiled language. Maybe I should rewrite all the rust implementation in zig and see if it’s just as fast
- hu3 3y ago
  I just imagined a transpiler from Rust to Zig.
__alexs 3y ago
Biome already gets pretty close.
lakpan 3y ago
I don’t really get it. Prettier isn’t something that you need to run that often on the entire repo; you’re probably just running it on save in a handful of milliseconds or on a pre-commit on hopefully less than 10 files. There are plenty other tools that often run on the entire repo, like tests, lint, build, type-checking, etc. focus on those. Bun showed that there’s a lot of space for improvement.
- biorach 3y ago
  Many organisations have a CI step that runs a linter or formatter on all the code and fails if the tool makes changes. The idea is to catch cases where contributors failed to run the tool on save. On large code bases this can be time consuming.
  - zegl 3y ago
    The trick to make CI in large codebases fast, or how to make any system fast, is to only test what’s changed. In CI you only need to lint the files that have changed, or run the tests that depend on code that has changed etc. This way the time it takes to execute the tests scales with the amount of changes, and not with the total amount of code.
    - DougBTX 3y ago
      The difficulty with that is trust, if there’s any chance it will miss things by accident then I’d rather eat the slow runs. Test runners that run new/changed tests first though are cool. Faster feedback on error but still comprehensive.
- sophiabits 3y ago
  Adding on to the other comment about CI setups, when I benchmarked removing eslint-plugin-prettier from my work’s ESLint configuration we saw a 37.5% speedup. On our fairly slow CI agents that ended up being pretty significant (~34s per run). I _wanted_ to drop the plugin and have Prettier automatically run on as a pre-commit hook (as you suggested) but in the end I lost :) We wound up keeping the plugin and eating the slower pipeline time. Of course there’s more overhead than just Prettier in this case—the ESLint plugin itself will be contributing towards the added runtime!—but depending on your codebase and CI machines, Prettier can definitely slow things down quite a bit. CI taking over a minute or two is a big drag on developer productivity imo. Ideal world is that everyone is on board with code formatting as a pre-commit task that only touches modified files so it doesn’t bloat CI unnecessarily, but it’s not always possible :(
cute_boi 3y ago
It is now 22.5k if it can be compiled upto wasm.
sophiabits 3y ago
There’s a project called “dprint” which you can use in the interim if Prettier is a bottleneck :) It’s a Rust-based code formatting platform with a Prettier plugin (among others). It’s a lot faster than running Prettier directly [1] The dprint plugin wraps Prettier under the hood so compatibility is good. The perf wins come from formatting files in parallel and incrementally. [1] https://david.deno.dev/posts/faster-prettier-with-dprint/ https://david.deno.dev/posts/faster-prettier-with-dprint/
yawboakye 3y ago
i’ve argued for writing tools in technology fit for purpose. a lot of web development tools being written in javascript (and then typescript) was because web developers had to build their own tools, and typically these folks were extremely comfortable with ruby and/or javascript, so naturally they reached for them. some of these important tools can have massive speedup gains just by switching internal data structures and algorithms. in my opinion, every rewrite implicitly makes this switch but the importance of these decisions are subsumed by the detail of the implementation language. prettier can be faster with a better algorithm. then maybe rust can help it take advantage of specific operating system improvements not naturally available to node. regardless, bravo and onwards with more rewrites!
- tubthumper8 3y ago
  > prettier can be faster with a better algorithm Do you have any examples of this or is there anything in particular that Prettier uses a suboptimal algorithm for?
cryptos 3y ago
> The main issue is that none of them match the long tail of formatting logic of prettier. I'm putting up a $10k bounty for any project written in Rust that passes > 95% of the prettier JavaScript tests. 95 % is not the "long tail" in my opinion. Maybe 100 % is too hard, but 95 % is rather low.
yipbub 3y ago
Is this going to be a race? Multiple people implementing it in parallel and one claims the prize?
danjc 3y ago
Isn't a bounty like the worst way to compensate someone for a defined piece of work?
- AlchemistCamp 3y ago
  It got the word out so well that it made the front page of HN, so I’d say it wasn’t a complete failure.
trilbyglens 3y ago
Next do rubocop
syrusakbary 3y ago
The people from Algora have created a great homepage for the bounty: https://console.algora.io/challenges/prettier https://console.algora.io/challenges/prettier Note: Guillermo Rauch and Wasmer also joined, adding $12,500 extra to the mix (total bounty is $22,500)

89 points by hardwaregeek 3y ago

Sick. Hope someone does it.

This would probably be a good thing on net, but how would plugins for Rust work? I use <https://github.com/NiklasPor/prettier-plugin-go-template https://github.com/NiklasPor/prettier-plugin-go-template> a bit and I'm not sure what one would have to do to make a plugin for a Rust binary.

veidr 3y ago
Well they could work any way the author wanted, but if it were me, I'd make the plugins WASM. That's how my favorite code formatter (dprint) does it. For that matter though, it's how I would implement just about any plugin, these days, except those that need to exchange large info rapidly (e.g. an editor plugin that needs to operate on every key press). It's dead easy to implement on the app side, and plugin authors can use an increasing number of higher-level languages to build plugins.
- mtsr 3y ago
  And on top of that it’s also easy to mostly guarantee the plug-in can only access the APIs you give it. Which is quite nice for third party code.
- CWIZO 3y ago
  Would love to learn more if you have any resources available?
- rob74 3y ago
  ...which would have the advantage that plugin authors could also use other languages that compile to WASM.
- crabmusket 3y ago
  > except those that need to exchange large info rapidly I've seen attempts at zero-copy memory sharing between JS and WASM, but I'm not sure if the same approach would allow different plugins to share memory and communicate via e.g. Cap'n Proto? https://observablehq.com/@kylebarron/zero-copy-apache-arrow-with-webassembly https://observablehq.com/@kylebarron/zero-copy-apache-arrow-...

lacker 3y ago

Looks like the bounty is at 20k now with contribution from @rauchg https://twitter.com/rauchg/status/1723400569392656771 https://twitter.com/rauchg/status/1723400569392656771

downvotetruth 3y ago

"I care that people can have a faster version of prettier, not how it's implemented." & "... for any project written in Rust" are contradictory.

topspin 3y ago
That was in the context of a question about whether external crates may be used. Rust is clearly implied.

maxloh 3y ago

Prettier runs on a single thread only. Running it on each thread could make it significantly faster [0]. [0]: https://github.com/microsoft/parallel-prettier https://github.com/microsoft/parallel-prettier

Alifatisk 3y ago
The performance gain is massive!

zImPatrick 3y ago

Why does everything need to be written in rust these days?

ramon156 3y ago
As a rust dev I'm just as perplexed. I've seen preprocessors and stuff like tailwind being rewritten in Rust aswell. Why though (
- mtsr 3y ago
  Some dev workflow tools really benefit from being very fast. And parsing and ast building and the like are actually things that rust is quite good for. That doesn’t mean it couldn’t be done in other languages, but it’s a weighing of trade-offs. And of course, groups of people, such as developers on socials, are quite sensitive to popularity.
  - vsnf 3y ago
    > And parsing and ast building and the like are actually things that rust is quite good for. Having built a compiler and typechecker in Rust, I don't know if I'd say that it was "quite good" for it. Going back, I'd still do it again, but it wasn't exactly a cakewalk. Rust's lifetime rules and visceral hatred of referential structs did not make my life easy at all. I definitely took the easy way out with a lot of string copies too.
  - dbrgn 3y ago
    Just as an example: Our eslint checks take 2 minutes to complete. Improving this would considerably improve developer productivity.
- fullspectrumdev 3y ago
  If it speeds up a build/CI pipeline it’s a worthwhile investment for a lot of folks
  - RamblingCTO 3y ago

deleted 3y ago

[deleted]

pyrolistical 3y ago

It would be problem just as fast if rewritten in any compiled language. Maybe I should rewrite all the rust implementation in zig and see if it’s just as fast

hu3 3y ago
I just imagined a transpiler from Rust to Zig.

__alexs 3y ago

Biome already gets pretty close.

lakpan 3y ago

I don’t really get it. Prettier isn’t something that you need to run that often on the entire repo; you’re probably just running it on save in a handful of milliseconds or on a pre-commit on hopefully less than 10 files. There are plenty other tools that often run on the entire repo, like tests, lint, build, type-checking, etc. focus on those. Bun showed that there’s a lot of space for improvement.

biorach 3y ago
Many organisations have a CI step that runs a linter or formatter on all the code and fails if the tool makes changes. The idea is to catch cases where contributors failed to run the tool on save. On large code bases this can be time consuming.
- zegl 3y ago
  The trick to make CI in large codebases fast, or how to make any system fast, is to only test what’s changed. In CI you only need to lint the files that have changed, or run the tests that depend on code that has changed etc. This way the time it takes to execute the tests scales with the amount of changes, and not with the total amount of code.
  - DougBTX 3y ago
    The difficulty with that is trust, if there’s any chance it will miss things by accident then I’d rather eat the slow runs. Test runners that run new/changed tests first though are cool. Faster feedback on error but still comprehensive.
sophiabits 3y ago
Adding on to the other comment about CI setups, when I benchmarked removing eslint-plugin-prettier from my work’s ESLint configuration we saw a 37.5% speedup. On our fairly slow CI agents that ended up being pretty significant (~34s per run). I _wanted_ to drop the plugin and have Prettier automatically run on as a pre-commit hook (as you suggested) but in the end I lost :) We wound up keeping the plugin and eating the slower pipeline time. Of course there’s more overhead than just Prettier in this case—the ESLint plugin itself will be contributing towards the added runtime!—but depending on your codebase and CI machines, Prettier can definitely slow things down quite a bit. CI taking over a minute or two is a big drag on developer productivity imo. Ideal world is that everyone is on board with code formatting as a pre-commit task that only touches modified files so it doesn’t bloat CI unnecessarily, but it’s not always possible :(

cute_boi 3y ago

It is now 22.5k if it can be compiled upto wasm.

sophiabits 3y ago

There’s a project called “dprint” which you can use in the interim if Prettier is a bottleneck :) It’s a Rust-based code formatting platform with a Prettier plugin (among others). It’s a lot faster than running Prettier directly [1] The dprint plugin wraps Prettier under the hood so compatibility is good. The perf wins come from formatting files in parallel and incrementally. [1] https://david.deno.dev/posts/faster-prettier-with-dprint/ https://david.deno.dev/posts/faster-prettier-with-dprint/

yawboakye 3y ago

i’ve argued for writing tools in technology fit for purpose. a lot of web development tools being written in javascript (and then typescript) was because web developers had to build their own tools, and typically these folks were extremely comfortable with ruby and/or javascript, so naturally they reached for them. some of these important tools can have massive speedup gains just by switching internal data structures and algorithms. in my opinion, every rewrite implicitly makes this switch but the importance of these decisions are subsumed by the detail of the implementation language. prettier can be faster with a better algorithm. then maybe rust can help it take advantage of specific operating system improvements not naturally available to node. regardless, bravo and onwards with more rewrites!

tubthumper8 3y ago
> prettier can be faster with a better algorithm Do you have any examples of this or is there anything in particular that Prettier uses a suboptimal algorithm for?

cryptos 3y ago

> The main issue is that none of them match the long tail of formatting logic of prettier. I'm putting up a $10k bounty for any project written in Rust that passes > 95% of the prettier JavaScript tests. 95 % is not the "long tail" in my opinion. Maybe 100 % is too hard, but 95 % is rather low.

yipbub 3y ago

Is this going to be a race? Multiple people implementing it in parallel and one claims the prize?

danjc 3y ago

Isn't a bounty like the worst way to compensate someone for a defined piece of work?

AlchemistCamp 3y ago
It got the word out so well that it made the front page of HN, so I’d say it wasn’t a complete failure.

trilbyglens 3y ago

Next do rubocop

syrusakbary 3y ago

The people from Algora have created a great homepage for the bounty: https://console.algora.io/challenges/prettier https://console.algora.io/challenges/prettier Note: Guillermo Rauch and Wasmer also joined, adding $12,500 extra to the mix (total bounty is $22,500)