8 ms·
Speeding up Prettier locally and on your CI with dprint
- alexjurkiewicz 4y agoThere's so much accidental complexity in the CI workflow purely to enable a tiny amount of caching. I hope the next generation of CI tools can improve this situation.
- dsherret 4y agoI know. I've been wondering if there's a way this complexity could be buried in the GitHub action for dprint (https://github.com/dprint/check https://github.com/dprint/check) so people would only have to specify that and everything would just happen, but I'm not sure it's possible at the moment. Caching seems to be limited to needing to use actions/cache (https://github.com/actions/cache https://github.com/actions/cache)
- dimgl 4y agoI've been trying Deno on a personal project and it keeps impressing me more and more. If Deno continues to improve like this, I can see it being the future of JavaScript development. This might go against the grain nowadays, but I'm really productive with JavaScript/TypeScript and any innovation in this space is appreciated. I've had to deal with so much ESM crap these last couple of months and using Deno has been a refreshing change of pace. Edit: I'm especially impressed with its dependency management. Import maps are a breeze and a welcome change to... whatever is going on in Nodeland. Like, I don't have to set up my imports both in tsconfig.json and package.json and walk this delicate tightrope to make sure that things don't just start breaking. With Deno's import maps it kind of just... works. I also love that I don't have to deal with a `node_modules` folder, and I can choose to vendor my dependencies if I'd like. It reminds me of the Go approach, which I really enjoy. There's other things I like but these are the big ones. The more I can get switched over to Deno the better.
- tmp_anon_22 4y agoI just spent the better part of an hour trawling through issues in the CRA, Webpack, and Axios github issues pages, pertaining to the webpack4 -> webpack5 (10/10/2020) release. I do not believe there is willpower for institutional changes to the javascript ecosystem that exists on timelines of less then 5 years.
- dimgl 4y agoYeah I recently went through something similar too. Spent a ton of time just digging through GitHub issues. Basically I've been dealing with the ESM changes. Add in the TypeScript complexity and all of the sudden I've now spent hours debugging some obscure issue rather than being productive... I've debated moving away from Node.js altogether because of it but there's no real realistic alternatives right now (for frontend development/bundling using React/Remix). I probably can't do any better, but the whole ESM migration has been completely mishandled. Even more infuriating is the response that things are working _as expected_... Hopefully Deno gains some traction and we can put this all behind us.
- antihero 4y agoHonestly I'd ditch CRA and hit up Vite nowadays. You will have to deal with a few poorly made packages (e.g. from Amazon) but it is a breath of fresh air.
- dimgl 4y ago+1, Vite is incredible
- iCarrot 4y agoI'm using Vite for a small greenfield project. Perhaps it's the selection of packages I use, but setting up was rather painful. Vite Vue + Typescript template run fine in dev mode but could not be built for production, thanks to a combination of an Vue issue and a tsconfig flag. Node polyfills work on `yarn dev`, but throw "kn.nextTick is not a function" for production build. Apparently I have to use package `rollup-plugin-polyfill-node` instead of `rollup-plugin-node-polyfills`. Now with all the setting up out of the way, it does feels good to have a dev server starting almost instantly, hot reload and full page reload stay constantly fast. Sometime though hot reload just fails and I had to manually reload the page, or manually remount the component I'm working on (by navigating back and forth) to see the changes. It also feels like I'm not even using typescript but plain javascript. Vite would happily serve broken code and some errors will only be shown on `yarn build`. I lost a watch compiler essentially.
- coolhoody 4y agoIt is worth having Deno even if you are not using it: just as a Js/Ts LSP server (it's inbuit). Smaller and better than tss etc. Also makes much more sense for local scripting. Give it a try even if in doubt for production.
- Klaster_1 4y agoCan you please elaborate on Deno language server being better over tss? Does it have smaller latency, does it start faster? Is it a drop-in replacement or some features are missing?
- coolhoody 4y agoIt is a drop-in replacement. Deno has a Node compatibility mode [0] which — besides the execution — also applies to LSP, so the Node-specific idioms are processed correctly. It is not cosmically faster than tsserver, but some — including myself — do prefer it. I wouldn't recommend to change anything if you are busy — but maybe take a glance when in the mood for checking out new things. [0] https://deno.land/manual@main/node/compatibility_mode https://deno.land/manual@main/node/compatibility_mode
- MapleWalnut 4y agoPrettier itself would feel much faster if it supported caching like other code formatters (Python's Black, Rust's rust-fmt)
- onion2k 4y agoPrettier does support that, in the sense that you can achieve it using an official solution. https://prettier.io/docs/en/precommit.html https://prettier.io/docs/en/precommit.html It's the Unix mentality of chaining things rather than building it in to the package.
- MapleWalnut 4y agoYeah, precommit isn’t my favorite thing out of personal preference. So it’s a shame prettier doesn’t have built in caching like other tools in the space.
- cas8 4y agoRegardless of Deno, I've been leaning more and more towards dprint over Prettier recently given how fast and configurable it is. I get Prettier's whole shtick about not being configurable so that your team can't fight over a style guide, but as a team that already had an internal style guide that predates Prettier, I just want a tool that can match what we have already decided on, which is where dprint has fit in nicely.
- benatkin 4y agoDeno is getting closer and closer to being able to compete with Node. A remaining performance issue was just partially fixed with this PR: https://github.com/denoland/deno/pull/14319 https://github.com/denoland/deno/pull/14319 Here's the issue: https://github.com/denoland/deno/issues/10157 https://github.com/denoland/deno/issues/10157
- rdsubhas 4y agoThe title is misleading. It's 40x faster only on the second run not because of any hotpath code code speedup, but because it has an option to use a cache to remember state of previous files. The full check durations are on the same ballpark. This has nothing to do with deno. Say prettier also added a cache to save the state of previous files, it would also be only 1s.
- dsherret 4y agoYes, the 40x faster is only for the runs after the first in this scenario. That said, if I disable the incremental cache locally (dprint check --incremental=false) and run this on the example code linked to in the article, on my machine the first run is 8.0s. If I run prettier's CLI it's 24.8s. dprint is faster because it formats files in parallel with Prettier and has Prettier snapshotted in a Deno runtime. There's some thread contention that's making it a little slower than it could be with v8 and I've been lazy to investigate it. Also, dprint uses very fast globbing with two threads working together in order to collect files. I would guess Prettier would still be slower if it did this because it's programmed in JS (though not by much), but it doesn't do it.
- onion2k 4y agoAdding caching to Prettier is reasonably easy, and most people will already have it through lint-staged or something similar. I'm much more impressed by "dprint is 3* faster than Prettier for the same task" than "dprint is 40* faster than Prettier because dprint caches results and Prettier doesn't". That makes me want to look at dprint. The "40* faster" figure makes it look like the benchmark is wrong.
- dsherret 4y agoThe title of the article was and is "Speeding up Prettier locally and on your CI with dprint". If you use the Rust-based plugins instead of Prettier it will be much faster as noted at the end of the article. To add another advantage of using dprint not mentioned, is that since it's pluggable you can use Prettier and other formatters all from the same formatting CLI. The main dprint repo does this in its dprint.json (https://github.com/dprint/dprint/blob/3d822a48133358ec4e2d5bfe051e9223616ad631/dprint.json https://github.com/dprint/dprint/blob/3d822a48133358ec4e2d5b...)
- oefrha 4y agoMy surprising takeaway is prettier is single-threaded when formatting multiple files. Seems like an embarrassingly parallel problem, even a default-off -j<N> option would go a long way.
- esprehn 4y agoIf you run prettier through jest (ex. By way of eslint) you get parallelism.
- esprehn 4y agoWe run prettier through eslint, and run eslint through jest which provides parallelism. eslint also has a cache... So I'm not sure what dprint gets you over that? https://github.com/jest-community/jest-runner-eslint https://github.com/jest-community/jest-runner-eslint https://github.com/prettier/eslint-plugin-prettier https://github.com/prettier/eslint-plugin-prettier
- syrusakbary 4y agoThe work that David has been doing with DPrint has been outstanding. They architected a whole plugin system is based on WebAssembly and Wasmer. Super impresive
- john567 4y agoWhy would you run code formatting tools in a CI pipeline? If the code had been committed in the wrong format the problem should be addressed before the code goes into version control. Stuff like this helps with maintaining a tidy history and consistent code base. Sounds like self inflicted wounds to me.
- 411111111111111 4y agoHow do you assure that the formatting as been done without running it in CI? A lot of steps in CI aren't there to do something, they're merely there for analysis reasons and conditionally marking builds as failed
- john567 4y agohttps://news.ycombinator.com/item?id=31176453 https://news.ycombinator.com/item?id=31176453
- onion2k 4y agoRunning a formatter in CI and failing the build if it finds anything is a good way to make sure people are running the right tools locally, which is in turn a good way to maintain quality. I've seen many devs work with broken environments and just ignore the problems because it doesn't actually stop them, or because they don't realise it's broken.
- john567 4y agohttps://news.ycombinator.com/item?id=31176453 https://news.ycombinator.com/item?id=31176453 > I've seen many devs work with broken environments and just ignore the problems because it doesn't actually stop them, or because they don't realise it's broken. That sounds like an educational issue. I work in a small team in a small organization.
- tqkxzugoaupvwqr 4y agoThe workflow is: You create a git branch, commit your changes to it and push them to the repo server. The CI then checks whether the code is formatted as expected. If the code is incorrectly formatted, the CI job fails and prints the format error(s). When the code is formatted properly (no CI jobs fail), you are allowed to merge into the master branch. Reading your comment I assume you want the repo server to reject incorrectly formatted code at push time. Rejecting commits/pushes has the disadvantage that you cannot share your code/branch with others (or yourself on other devices), and cannot back up your code to the repo server. You could argue formatting doesn’t take long. I’d argue, the repo server shouldn’t keep me from publishing code to a dev branch.
- iamjs 4y agoSerious question: Why not use source control to identify the files with changes and limit the scope of your formatter run to just those files. If the unchanged files met the formatter rules when they were committed, then repeated runs should be guaranteed to pass. Why introduce a cache when you could just do less work?
- lozenge 4y agoBecause on CI there's no guarantee that the parent commit has already passed. It might be unchecked, in progress, or CI failed despite being a parent commit. Now you could query the CI from the CI but that seems to be a dubious dependency to me, and user would need to set up an access token.
- maga 4y agoGreat to see dprint on the front page! While here it's speeding up prettier with caching/wasm, it can replace prettier altogether and be even faster (for example, with `dprint-plugin-typescript` for TS/JS dev). So, if you are doing web dev and not yet ready to switch to Deno, you can at least speed up formatting by using `dprint`. David, I already owe you guys more beer than I can buy for all the great work on the Deno toolchain, but if we could get this issue[1] with HTML template tags going, it would be easier for me to convince the Lit crowd to chip in. Tanks again for the great work! [1] https://github.com/dprint/dprint-plugin-typescript/issues/9 https://github.com/dprint/dprint-plugin-typescript/issues/9
- dsherret 4y agoIt's definitely something I would like to implement. I haven't found a parser that would be good for formatting yet so maybe I will have to write my own :/
- skybrian 4y agoI'm wondering if every language is a plugin with dprint, or does it have some built in? I don't see a list of supported languages.
- maccard 4y agoHow does it take 41 seconds to format 500 source code files? _how_ is that an acceptable amount of time?
- bloopernova 4y agoI wonder if anyone will take this work and make a visual studio code extension from it?
- dsherret 4y agoThere is one here: https://github.com/dprint/dprint-vscode https://github.com/dprint/dprint-vscode
- hispanic 4y agoFor a recent personal project, I wanted to format the generated HTML. My first stab at it used Prettier. But, I was very underwhelmed with the processing speed. I then gave js-beautify[1] a try, which I found to be much, much faster. [1] https://github.com/beautify-web/js-beautify https://github.com/beautify-web/js-beautify