14 ms·
ESbuild – A fast JavaScript bundler and minifier in Go
- q3k 7y agoFinally. The extreme slugishness of webpack&co has always bugged me any time I ventured into JS development, to the point of me getting immediately discouraged.
- tmpz22 7y agoIt’s a natural cycle though. New bundler created that is fast and simple to use -> New features are requested and added to the bundler which makes it more sluggish -> New bundler created that is fast and simple. I wonder how this compares to ParcelJS. I’m a big fan of Go but there are usually fringe benefits in having the bundler written in the language that it is bundling I.e. JavaScript.
- mtmail 7y agoBenchmark against parcel is in the documentation. 200x faster.
- tmpz22 7y agoThat’s exceptional, if the benchmarks test identical workloads and features I would definitely consider switching to this. FWIW it may sound like I’m hostile to this framework when I’m just trying to be critical in a constructive way. I use golang as my main driver at work and find most projects in Go to be better then its common counterparts, however I have been through so many JS bundlers that I scratch my head when anyone claims superior results on (Self-declared) hobby ones. OP should know this seems really solid and like great work. I just don’t want junior devs running head first putting this in production.
- _bxg1 7y agoAlas, there's such a massive ecosystem at this point around Webpack and Babel and family (extra features like tree-shaking, loaders support for different things, project configurations, etc.), that I think you'd be hard-pressed to make an outright replacement take off. What you could do (and I'm surprised this hasn't been done yet) is lower some of the hot paths of Webpack itself into a lower-level language. Possibly even moreso than Webpack, Babel seems like a natural target since it has much simpler input/output. Although one barrier I can see with that (unless it's C/C++) is requiring people to install a whole other build system just for it to be used in an npm install. C/C++ compilers generally come with the system, but Go/Rust ones do not. Edit: further info https://stackoverflow.com/questions/20728255/could-one-write-a-native-node-js-extension-in-go-as-opposed-to-c#39262638 https://stackoverflow.com/questions/20728255/could-one-write...
- wildpeaks 7y agoI suspect the supposed sluggishness attributed to Webpack actually comes from Babel instead because it's been fast ever since I removed Babel from my pipelines (because Typescript already covers what I was using Babel for).
- nitsky 7y agoCheck out this project that is an attempt to rewrite Babel in Rust. It includes a Webpack plugin and their benchmarks show a major speedup. https://github.com/swc-project/swc https://github.com/swc-project/swc
- _bxg1 7y agoVery cool. Gotta say though, that logo is impossible to read...
- catlifeonmars 7y agoI think that’s a good observation. Babel is probably a perfect place to start. Two things: (1) In Go you can trivially distribute pre built binaries for <num arch> * <num os> targets. Just building with GOOS set to windows, darwin, and linux (with GOARCH set to amd64 and arm64) would cover 99% of all users. (2) Depending on where bottlenecks are, RPC/HTTP/WS can be just as effective as dynamic linking. So maybe even the need for a C/C++ toolchain could be avoided.
- _bxg1 7y agoI didn't know that about Go, that's pretty cool! I assumed it was like Rust, where the compiler/toolchain makes builds on arbitrary specific platforms really easy to do, but where you still have to actually do those very narrow builds.
- pcwalton 7y ago> I assumed it was like Rust, where the compiler/toolchain makes builds on arbitrary specific platforms really easy to do, but where you still have to actually do those very narrow builds. What's the difference?
- wadkar 7y agoX in Go seems to be the new Y in Rust. I am curious to know what is the benefit of doing this in Go? I thought that the reason why webpack is complex is because the whole JavaScript ecosystem is complex with respect to the ES20XX standards. I fail to understand how rewriting the logic in another language is going to reduce the complexity. Speed is important, sure, but certainly not at the cost of correctness!
- _bxg1 7y agoNot sure anyone said anything about reducing complexity. Webpack and Babel in tandem are pretty much "the compiler of modern JavaScript". They do a lot of crunch work and can take a long time to run. Makes perfect sense to me to try and do that in a faster language.
- tmpz22 7y agoI think X in Go preceded Y in Rust by several years. In short Go is a simple language with minimal syntax and explicit design patterns that make it very easy to read and understand once you’ve learned the ropes. It’s a typed and compiled language with all the benefits that entails, with great concurrency support. It’s standard library is fantastic, compile times are great (one of the original goals for Go was to reduce c++ compile times at Google), with out of the box support for networked/web applications on the standard library. Finally the community for Go is very strong with things like the gophers slack channel which is insanely popular and friendly.
- wadkar 7y agoHaha, yes. Thanks, I’ll certainly check out the gophers community. As the sibling points out babel/webpack are like compilers of JavaScript world so reducing compile times have a huge impact on developer experience. I am just wondering where does the complexity of juggling various ES* standards go? Also, the webpack plugins and loads of config examples might require this project to support additional complexity. Will the compiled/typed nature of Golang make it difficult to correctly maintain parity with babel/webpack? I am really looking forward to learning golang this year. I am almost done ~~suffering~~ learning JavaScript/React + CSS3/media-queries this winter.
- vijaybritto 7y agoI was thinking that a parser and tokeniser distributed as wasm could increase the build speed while maintaining the js part for customisation. But this is a full fledged native bundler. Though less flexible it's incredibly fast!
- alipang 7y agoI believe something like this project is inevitable. You're probably thinking "Where's the Rust version of this?". I'll save you a roundtrip to Google that'll find you SWC. [1] SWC is already more mature, at least more so than this project. Using rust does also seem to have some advantages compared to Go. Still, nothing wrong with some competition, just think it'll be pretty hard to replace the entire ecosystem around Typescript/Babel/Webpack in one go. Not going to work for existing project - probably way too high of a risk for new ones, sadly. [1] https://github.com/swc-project/swc https://github.com/swc-project/swc
- _bxg1 7y agoNote that what Babel does and what Webpack does are orthogonal. They're frequently used together and the challenges of converting them from JS to native are similar, but the OP isn't really in "competition" with SWC.
- alipang 7y agoSure, well aware that Babel and Webpack are not the same thing. SWC has a Webpack plugin, whereas ESBuild seems to aim to replace both Webpack and Babel. Correct me if I'm wrong. This is a big reason why I think SWC is more likely to succeed, and why I say they're in "competition" since they overlap partially, even if it's not 1-1.
- _bxg1 7y agoHmm, I got a little thrown off by the words "bundler and minifier", which are both Webpack tasks. Scrolling further down I do see "JSX transformation" thrown in, which is definitely normally a Babel thing. If this project is trying to replace everything in a single pass then yeah, it's probably doomed to failure. Not only from the ambition of the task, but from the lack of the modular structure that allows community support to thrive: anybody can write a Webpack loader or a Babel plugin without getting their hands dirty in those codebases. That doesn't seem to be the case here.
- 7y ago
- shirshak55 7y agohow does this compare with SWC made with rust?
- constexpr 7y agoIt's cool to wake up in the morning and see my project on HN! For the record, I'm not trying to compete with the entire JavaScript ecosystem and create an extremely flexible build system that can build anything. I'm trying to create a build tool that a) works well for a given sweet spot of use cases (bundling JavaScript, TypeScript, and maybe CSS) and b) resets the expectations of the community for what it means for a JavaScript build tool to be fast. Our current tools are way to slow in my opinion. Let's build faster tools!
- deleted 7y ago[deleted]
- pythonist 7y agoThank you for making it!
- abetlen 7y agoVery cool project, do you know how this compares with Hugo's javascript minifier / bundler?
- gcatalfamo 7y agoI’d like to know this as well.
- earthboundkid 7y agoHugo uses https://github.com/tdewolff/minify https://github.com/tdewolff/minify internally, so that’s the relevant comparison.
- pierrec 7y agoThis supports ES6 modules, which I believe are totally unsupported by Hugo Pipes. I remember being annoyed at having to throw node and webpack into the pipeline just because of that (which slowed everything to a crawl).
- eberkund 7y agoHugo doesn't have a JavaScript bundler, it just lets you concatenate JavaScript files which is different from a JS bundler which needs to support JS modules.
- tzury 7y agoreading the go code is a true pleasure and great resource to study the language further.
- tyfon 7y agoI made my last webpage in 1999 using HTML tables and cut-up "design" jpgs so I am a bit out of date in this area. I see all these preprocessors and crap for "web apps" and I wonder why nobody just writes HTML/CSS/Javascript instead of all of these other "languages" and abstractions or is the base web language now so bad that nobody touches it manually?
- Eric_WVGG 7y agoWeb apps, basically. You couldn’t make Google Sheets that way, for example.
- tyfon 7y agoSo it would be impossible to write google sheets in pure html/javascript (except for the server bits that is)?
- adventured 7y agoThe matter is that it wouldn't make sense to do it. It's certainly possible. If you try building a large scale application (not merely a website or simple CRUD-style service), something as complex and demanding as a Google Sheets in pure html/javascript, you're going to reinvent a lot of wheels in the process and waste an enormous amount of time, for reasons (hopeful performance gains, and minimizing external requirements) that will not be strong enough to justify the time cost. More likely in the process of reinventing wheels, you'll lose a lot of your hoped for performance and simplification gains, as you create your own monsters and bloat.
- BlueTemplar 7y agoWell, the issue these days is that JavaScript is becoming a serious security problem. Hopefully more and more people will just start blocking JavaScript outright, and this will just have been a misguided attempt to try and turn the web browser into a full blown virtual machine...
- 7y ago
- mariopt 7y agoIf these numbers are true, it would be nice to have more documentation and tutorials. This would help the javascript community a lot. I built a gaming computer to work with javascript codebases, true story.
- sunflowerdeath 7y agoActually, webpack does a lot of additional stuff that might be needed for some complex configurations, so it can't be easily replaced with this tool. But also almost 75% of the time (in webpack) is spent inside the minimizing plugin, and that is very simple task. So it would be great to have just the minimize plugin based on this tool. Then maybe investigate what other performance critical parts of the webpack can be replaced without requiring to change the ecosystem.
- throwawaygo 7y agoYass. This is a really well written go codebase. Go read it and see what life was like before the framework kids showed up.
- christophilus 7y agoEh. I’ve been working in the industry since before the framework kids showed up. I’ve seen plenty of truly awful C++, C#, and COBOL codebases.
- drej 7y agoCan we drop the "in Go" in links like these? I don't really see how that adds any value to the end user.
- tylerchr 7y agoThe thing that caught my attention about this post was specifically that it‘s a JavaScript bundler not written in JavaScript. The “in Go” in this case made the difference between my noticing and skipping a very interesting submission, so I think it’s relevant.
- profmonocle 7y agoAlmost all JavaScript tooling is itself written in JavaScript, so the fact that it's written in Go is actually notable to me.
- thepaulstella 7y agoThat’s a no go for me.
- rienbdj 7y agoGo is such a strange choice of language for this kind of thing compared to something in the ML family.
- earthboundkid 7y agoYes, think of all the great tools written in Haskell, like pandoc and uh… a second example someone will reply to me with that no one has ever fucking heard of.
- throwaway8941 7y agoNah, the second one is also pretty popular. https://www.shellcheck.net https://www.shellcheck.net
- earthboundkid 7y agoWell, I can say I have heard of it. It still strikes me as a pretty much trivial tool compared to the bazillion things written in Rust and Go in the last five years, but it’s better than nothing.
- kroltan 7y agoWay to get overly salty. ML is not just Haskell, there is OCaml and F# as other notable dialects. (and to a much more limited extent, Rust, which borrows a few features but is otherwise more traditionally C-like) The language family lends itself very well for writing data transformation, especially tree processing (of which most parsers require), which is a very common kind of operation when writing minifiers and syntax transformers.
- earthboundkid 7y agoI agree that syntax transformations are in the wheelhouse of ML-family languages, but this just makes the absence of any popular tools besides pandoc more striking.
- Gehinnn 7y agoI think if somebody replaces webpack with something that is 100x faster has 80% of webpacks most important features, many companys would pay real money for it.
- deleted 7y ago[deleted]
- globular-toast 7y agoI've been using parcel because it's the only thing I know and I always thought it was slow but it claims to be fast and I'm new to js so I took it's word for it. According to this it's the slowest of the lot, though. Why does it claim to be "blazing fast"?
- constexpr 7y agoYes this was a surprise for me too. The benchmark I used for esbuild deliberately tests full build time without a cache, since that is an important metric to me. I assume Parcel is talking about incremental build times when the cache has already been populated. In that case I'm sure Parcel is not as slow.
- sod 7y agoDepends on what you are measuring. If 95% of what you care for is development mode, then it may not even be faster. Individual files only have to be compiled once in watch mode. For some bundlers even between builds. If you measure production builds, it's a different story. As every change has to most likely redo tree shaking and minifying over all the code. If you want source maps, it gets even worse.
- mekster 7y agoFor my project with maybe 30 dependencies in package.json, the incremental build finishes within a second on watch mode when I edit a file, so it's quite fast in that case.
- shrumm 7y agoThis is really cool - our dashboard uses Vue for the frontend and a Go backend. Everytime I deploy, I think about how insanely slow the frontend 'compilation' + unit tests step is compared to testing and building the Go code. It's one of the things that makes Go a joy to code in for me. On a similar note, we have some services written in Python and I was doing some research for some static type checking tools. At least in my testing, Pyright (written in Node) outperforms the pants out of similar tools written in Python. Using Go to bundle + minify Javascript sounds perfectly reasonable to me.
- dcre 7y agoNice work! Building from scratch without any caching is not the build time devs will be seeing 99% of the time, so it’s probably not the best benchmark.
- boulos 7y agoHow fast is an incremental rebuild with webpack though? That esbuild finishes from scratch in .5s (though probably with the code previously loaded from disk) means it’ll still likely be the fastest. Posters up thread claim that minifying is the slowest part of webpack, so presumably that’s pretty consistently slow relative to input size.
- dcre 7y agoYou don’t need to minify during development, so like building from scratch I’m not sure how much it matters for a practical benchmark. I don’t at all doubt esbuild is faster than the others, just in practice I doubt it’s as much faster as the benchmark suggests.
- deleted 7y ago[deleted]
- truth_seeker 7y ago> Parsing, printing, and source map generation are all fully parallelized I think this the major cause of speedup. Now that NodeJS supports WorkerThreads, it can be done in parallel. Along with this if right choice of data structures used there is no need to depend upon writing a build manager in another language instead of host language. Also, as of today JS(V8) performance is getting closer to Go runtime. https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/go-node.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- mappu 7y ago> I think this the major cause of speedup. This simply can't be true. On a 4 core laptop you might expect parallelism to provide up to 4x performance. But this has 75x / 200x performance, that must come partly from other reasons.
- truth_seeker 7y agoFirst of all you didn't even care to read I also mentioned that choosing the right data structures also. ;) You are deluded by the benchmark posted too. I terms of different build process steps that library isn't doing as much as other js build frameworks.
- mappu 7y agoI understood your phrase "the major cause" to mean majority reason, that couldn't be true since the remaining difference is greater. If you meant something different than that, then I misunderstood you. We both agree that the remainder of the performance (the bulk of it) must come from other reasons than parallelism. As well as your suggestions (doing less work, choosing the right data structures) - I would also propose (A) no V8 JIT warmup time (B) no need for fstat on many node_modules files (C) fewer AST passes (D) possibility of value struct types => less boxing => lower GC pressure (E) less dynamism/no plugin architecture.
- saddam96 7y agoSuperb project. Did you try comparing against Packem. We leverage Rust as a backend for the bundling and resolving phases: https://packem.github.io https://packem.github.io