8 ms·
> 50-100 Times Faster than ESLint > Our previous linting setup took 75 minutes to run, so we were fanning it out across 40+ workers in CI. By comparison, oxlin
by Aissen 3y ago
> 50-100 Times Faster than ESLint
> Our previous linting setup took 75 minutes to run, so we were fanning it out across 40+ workers in CI. By comparison, oxlint takes around 10 seconds to lint the same codebase on a single worker[…]
So it's in fact 18000 times faster on this embarrassingly parallel problem (but doing less for now).
- joeldo 3y ago75 / (1/6) = 450. Still very exciting!
- Aissen 3y agoYou forgot the 40 workers vs 1 worker.
- anamexis 3y agoThe way I read it, it was taking that amount of time before they split it into workers.
- rwilsonperkin 3y agoCorrect, it was 75 minutes total compute time. That was spread across workers to make the walltime more reasonable
- Aissen 3y agoIndeed, I really need to improve my reading comprehension.
- msoad 3y agoin a very large codebase, how common it is to run the linter for the entire repo? Is this an optimization worth spending time on?
- sapiogram 3y agoYes, because you lint everything in CI. Otherwise, linter warnings will start creeping into your codebase immediately, and the tool becomes much less useful.
- mathverse 3y agoWould not you lint only on files that changed?
- erikaww 3y agoI'm not sure if Eslint has this, but there could be cross-file lints (eg. unused variables). If some file changes, you may need to relint dependencies and dependent files. This could recursively trickle. I'm not sure if Eslint does this either, but indices or some incremental static analysis sounds like it could help the linter minimize rechecks or use previous state.
- msoad 3y agoif you have one file that every single file across the repo imports in a way and you make changes to that file, you might run the linter for the entire repo. But again, how likely is this scenario?
- erikaww 3y agoIf the index or incremental static analysis object was designed well enough, I don't think you would need to lint every file, you would just need to look at files that consume that variable. Maybe you would look at every index? I'm not sure how well this could scale across (600- 1000?) different lints though. I should look into static analysis a bit more.
- ehutch79 3y agoYou can tell eslint about globals in it's config. But if you're using variables that arn't declared in the file somehow, that might be an issue you want to look at in general. That's a potential foot gun a linter should be balking at.
- davedx 3y ago75 minutes to generate a bunch of mostly irrelevant nitpicks. What a colossal waste of compute resources. (1) IME if you’re using TypeScript then ESlint’s real value mostly approaches zero. For pure JS projects it’s useful for finding nullref type bugs. (1) > Our previous linting setup took 75 minutes to run, so we were fanning it out across 40+ workers in CI This. Is. Insane.
- ehutch79 3y agoDisagree on ESLint vs Typescript. ESLint and TyepScripts jobs should have minimal overlap. ESLint primary job is linting. It should be finding 'foot guns' and code style issues. Things that are absolutely valid in the language, but could lead to potential issues. Because of that, it's totally valid that you're not finding as much value in it. It depends on the rules you enable in it, etc. And yeah, it can feel super nitpicky when it's yelling at you for not having a radix in parseInt(). Typescript's 'compile' step or whatever, it doing type checking and making sure your code is valid. If you're using bare JS, your IDE should be doing this job, not eslint. (but yes, anything more than a few minutes to lint even a large code base is insane.)
- padjo 3y ago“rules of hooks” linting alone prevents a ton of bugs in your average React codebase and TS will provide no help there
- davedx 3y agoAh yes the “exhaustive dependencies” rule that can trigger huge unnecessary refactors for absolutely zero value. Linting has some value, it’s just that in my professional experience its costs outweigh its benefits
- padjo 3y agoIf you leave dependencies off a hook by mistake you’re creating bugs, unintended behaviour and making your code a nightmare to modify. That’s always going to be worth linting for. Can you over lint a codebase, sure I guess so, if your lint stage is taking hours it probably needs optimization, but your assertion that type checking is enough is incorrect.
- deleted 3y ago[deleted]
- JeanMeche 3y agoHow much of those 75min are due to @typescript-eslint ? Requiring the TS AST adds a massive overhead.