5 ms·

esbuild makes it pretty clear that writing the JS ecosystem in JS is a bad choice*. now we need to rewrite tsc/babel/npm/eslint/etc... in a language that compi

esbuild makes it pretty clear that writing the JS ecosystem in JS is a bad choice*. now we need to rewrite tsc/babel/npm/eslint/etc... in a language that compile down to native binary that is also cross platform (like go or rust), to gain the same performance improvements across the entire JS tool-chain. * based on the benchmark on their landing page.

deanclatworthy 6y ago
I think this comment undervalues the quality of engineering that Evan has put into esbuild. It’s not only quick because it’s written in golang, but also due to the decisions taken in architecture and an opinionated tool that does mainly one thing well.
- jakelazaroff 6y ago
  I don’t read OP as devaluing Evan’s work. Presumably the TypeScript/Webpack/etc teams are also doing great engineering work — but because they’re writing in JS, the performance ceiling is significantly lower.
afavour 6y ago
I don’t know about that. Just look at the performance difference between NPM and Yarn: both built with JS, but for me at least Yarn is way, way faster. I’m sure it would make sense to rewrite certain things in Rust or Go (then possibly pull them into Node as native modules) but they’re not magic. Good engineering will win out no matter what.
- earthboundkid 6y ago
  Good engineering is definitely key, but I think interpreted languages are a bad fit for developer tools (though they are extremely popular for writing developer tools). On the surface, writing JS tools in JS is good because hey, you definitely know JS, right? But in practice it is bad because I don't want the JS I am writing in any way constrained by the JS you wrote. The problem with any interpreted language is that to run the tool you need all the source and all the dependencies plus the interpreter in something like their intended versions to be present in the right locations on disk. This is a huge PITA, and if you're say writing JavaScript targeting ES5 but your tools are targeting ES2020 or vice versa, there can be painful yak shaving sessions getting everything on the same page. Essentially all developer tools should be static binaries.
  - brundolf 6y ago
    Would I be right in guessing that most of your experience with dynamic languages comes from Python? 1) Modern JITed JS is fast. It's closer to Java than it is to what we typically think of as "interpreted languages". 2) "something like their intended versions to be present in the right locations on disk"; Node packages really just depend on the user having a new-enough install of Node and NPM, somewhere in the path. Any dependencies that live on npm will install to the local directory to ensure against conflicts or permissions issues. The only exception I can think of is that occasionally (looking at you, node-sass) npm dependencies will have to do a native build of something and require a local C/++ compiler somewhere in the path. This is pretty rare in my experience, and isn't a super brittle dependency anyway. 3) "if you're say writing JavaScript targeting ES5 but your tools are targeting ES2020 or vice versa, there can be painful yak shaving sessions getting everything on the same page"; this really doesn't happen in any distributed NPM dependencies I've worked with. The great majority are written for the least-common-denominator of environments; if they use ES2020 internally, they'll do their own transpilation step to make sure you can use them in older environments. Also, "or vice versa" really doesn't apply because all ES versions are 100% backwards-compatible. You just need a Node version new enough to cover the newest features that any of your code or dependencies is using. Dependencies will be as compatible as they can be; your own code is under your control. Virtually never an issue.
    - earthboundkid 6y ago
      1. My complaint is not about speed. 2. Node is much better than Python because the correct location on disk is pretty much project/node_modules. However, it is still that case that e.g. Babel was broken by Node 12.17.0 [1] last year, which necessitates something like NVM. I don't want to deal with that for my build tools. I just want a binary that is known to work to continue to work until it is replaced. 3. Yes, conventionally everything in Nodeland was written out to dist as ES5. This also sucks ass because now you end up shipping a bunch of ES5 crap to your end users even though you aren't targeting ES5. It is very hard to build an actual ES2020 module unless you're willing to not use external dependencies at all. Snowpack et al are shifting this slowly, but it has barely even begun as a trend. [0] https://nodejs.org/en/blog/release/v12.17.0/ https://nodejs.org/en/blog/release/v12.17.0/ https://www.google.com/search?q=babel+compat+data+corejs3+shipped+proposals https://www.google.com/search?q=babel+compat+data+corejs3+sh...
- arcatek 6y ago
  Speaking of Yarn in particular since I maintain it, I strongly believe that more than speed we (package managers) need a good polish. And to do that, we need a community that feels empowered to make changes when they feel frustrated about something - which makes using JS / TS critical. By contrast, the UX of a bundler / linter is fairly straightforward (compared to Yarn's 46 builtin commands), so making external contributions easier isn't really necessary.
- FlashBlaze 6y ago
  Agreed regarding yarn and npm comparison. During my development of React Native, I was using Yarn and I was blown away at the rate of installing the packages. Since then, I've pretty much used yarn over npm for all my React and React Native projects.
brundolf 6y ago
One of the main benefits of Esbuild (if I recall correctly) is that all passes share a single AST. Whereas in a traditional setup your linter, transpiler, formatter, and sometimes even bundler and minifier all do their own totally independent passes on the source which means parsing it from scratch each time. So this would be a classic case of integrating everything into a monolith for a boost in efficiency.
christophilus 6y ago
I don’t think so. I’d bet that a JS solution that followed esbuild’s architecture would be in the same order of magnitude of performance. Now, you could argue that JS encourages complex, inefficient abstractions and over engineering and Go encourages the opposite. That, I’d believe.

deanclatworthy 6y ago

I think this comment undervalues the quality of engineering that Evan has put into esbuild. It’s not only quick because it’s written in golang, but also due to the decisions taken in architecture and an opinionated tool that does mainly one thing well.

jakelazaroff 6y ago
I don’t read OP as devaluing Evan’s work. Presumably the TypeScript/Webpack/etc teams are also doing great engineering work — but because they’re writing in JS, the performance ceiling is significantly lower.

afavour 6y ago

I don’t know about that. Just look at the performance difference between NPM and Yarn: both built with JS, but for me at least Yarn is way, way faster. I’m sure it would make sense to rewrite certain things in Rust or Go (then possibly pull them into Node as native modules) but they’re not magic. Good engineering will win out no matter what.

earthboundkid 6y ago
Good engineering is definitely key, but I think interpreted languages are a bad fit for developer tools (though they are extremely popular for writing developer tools). On the surface, writing JS tools in JS is good because hey, you definitely know JS, right? But in practice it is bad because I don't want the JS I am writing in any way constrained by the JS you wrote. The problem with any interpreted language is that to run the tool you need all the source and all the dependencies plus the interpreter in something like their intended versions to be present in the right locations on disk. This is a huge PITA, and if you're say writing JavaScript targeting ES5 but your tools are targeting ES2020 or vice versa, there can be painful yak shaving sessions getting everything on the same page. Essentially all developer tools should be static binaries.
- brundolf 6y ago
  Would I be right in guessing that most of your experience with dynamic languages comes from Python? 1) Modern JITed JS is fast. It's closer to Java than it is to what we typically think of as "interpreted languages". 2) "something like their intended versions to be present in the right locations on disk"; Node packages really just depend on the user having a new-enough install of Node and NPM, somewhere in the path. Any dependencies that live on npm will install to the local directory to ensure against conflicts or permissions issues. The only exception I can think of is that occasionally (looking at you, node-sass) npm dependencies will have to do a native build of something and require a local C/++ compiler somewhere in the path. This is pretty rare in my experience, and isn't a super brittle dependency anyway. 3) "if you're say writing JavaScript targeting ES5 but your tools are targeting ES2020 or vice versa, there can be painful yak shaving sessions getting everything on the same page"; this really doesn't happen in any distributed NPM dependencies I've worked with. The great majority are written for the least-common-denominator of environments; if they use ES2020 internally, they'll do their own transpilation step to make sure you can use them in older environments. Also, "or vice versa" really doesn't apply because all ES versions are 100% backwards-compatible. You just need a Node version new enough to cover the newest features that any of your code or dependencies is using. Dependencies will be as compatible as they can be; your own code is under your control. Virtually never an issue.
  - earthboundkid 6y ago
    1. My complaint is not about speed. 2. Node is much better than Python because the correct location on disk is pretty much project/node_modules. However, it is still that case that e.g. Babel was broken by Node 12.17.0 [1] last year, which necessitates something like NVM. I don't want to deal with that for my build tools. I just want a binary that is known to work to continue to work until it is replaced. 3. Yes, conventionally everything in Nodeland was written out to dist as ES5. This also sucks ass because now you end up shipping a bunch of ES5 crap to your end users even though you aren't targeting ES5. It is very hard to build an actual ES2020 module unless you're willing to not use external dependencies at all. Snowpack et al are shifting this slowly, but it has barely even begun as a trend. [0] https://nodejs.org/en/blog/release/v12.17.0/ https://nodejs.org/en/blog/release/v12.17.0/ https://www.google.com/search?q=babel+compat+data+corejs3+shipped+proposals https://www.google.com/search?q=babel+compat+data+corejs3+sh...
arcatek 6y ago
Speaking of Yarn in particular since I maintain it, I strongly believe that more than speed we (package managers) need a good polish. And to do that, we need a community that feels empowered to make changes when they feel frustrated about something - which makes using JS / TS critical. By contrast, the UX of a bundler / linter is fairly straightforward (compared to Yarn's 46 builtin commands), so making external contributions easier isn't really necessary.
FlashBlaze 6y ago
Agreed regarding yarn and npm comparison. During my development of React Native, I was using Yarn and I was blown away at the rate of installing the packages. Since then, I've pretty much used yarn over npm for all my React and React Native projects.

brundolf 6y ago

One of the main benefits of Esbuild (if I recall correctly) is that all passes share a single AST. Whereas in a traditional setup your linter, transpiler, formatter, and sometimes even bundler and minifier all do their own totally independent passes on the source which means parsing it from scratch each time. So this would be a classic case of integrating everything into a monolith for a boost in efficiency.

christophilus 6y ago

I don’t think so. I’d bet that a JS solution that followed esbuild’s architecture would be in the same order of magnitude of performance. Now, you could argue that JS encourages complex, inefficient abstractions and over engineering and Go encourages the opposite. That, I’d believe.