14 ms·
Tiny Treeshaker: JavaScript tree shaking in 200 lines of code
- Waterluvian 5y agoSomething that drives me bonkers about current tree shaking is that there’s no good way to see what got shaken. I want a graph that shows me all the branches that got culled. Even better would be a dependency graph that shows me what is causing things not to be culled. I had an issue where Antd was pulling in 200KB of icons and I couldn’t figure out why until I spent hours bisecting it by commenting out entire sections of my application.
- cookiengineer 5y agoIt would actually be quite amazing if dead code analysis of branches are integrated into eslint or other linters. But yeah, I guess that would also require a VM like understanding of timeouts, intervals and the Browser's triggered events so it's kinda hard to do.
- mirekrusin 5y agoIt doesn't have to be perfect, from static analsis you can get what people want. It won't cover dynamically created keys but people don't use it anyway. You'd just have to provide list of entrypoints and ie. vscode plugin could gray out or put horizontal grey bar for code that's shaken. It would be great plugin.
- kohlerm 5y agoA solution would need to be fast an incremental. Also we have got much faster bundlers with tree-shaking support recently (for example esbuild and parcel adopting swc), fast incremental correct builds seems to be still not completely there. E.g. cache invalidation would need to be correct in all possible cases, which does not seem to be the case yet, according to some Github issues I read recently. Otherwise I agree such a tool in VS would be great. Not that if you generate sourcemaps you kind of have a similar view in Chrome. Shouldn't therefore not be too difficult to implement in principle for VS.
- mirekrusin 5y agoInteresting thing to notice is that for vscode plugin it probably doesn't have to be fast/incremental – the reason is that you want to skip node_modules completely (minus symlinked) - because it's for code you're developing, not full blown tree shaking on final bundle that has to visit everything.
- onion2k 5y agoSomething that drives me bonkers about current tree shaking is that there’s no good way to see what got shaken. On a super basic level you can just build with and without tree-shaking and diff the results. If you turn off transpiling and minification, and move your bundled NPM modules to their own separate file, you can see what parts of your own code got shaken out pretty easily. I'm tempted to have a go at writing a tool to do that actually. It would be useful.
- jgalt212 5y agodiffs (and network graphs) seem like a great idea, but they quickly get so big that one can't make heads or tails of the visualization.
- barbazoo 5y agoTIL about tree shaking and that that's a thing in JS. Is that applicable to other languages? Why not Python?
- andrewstuart2 5y agoIt's common in javascript at least because you're delivering the code across the network, frequently, and at dev time, you're probably pulling in large libraries of components and frameworks that you may not use in their entirety. Since the load time is one of the core metrics used by SEO, and has shown to have a significant impact on conversion rates, it's worth the investment to remove code that's not being used. I'm sure there are other places where it's applicable, but modern javascript and webapps have to be the quintessential use case.
- avbor 5y agoMost other compiled languages do dead code elimination, which sounds similar but is a little different. Think of dead code elimination as removing code that doesn't change the output, while tree shaking instead includes code that could run. To apply this to python is interesting - if you were creating a packaged version, I could see "compiling" the code to a separate package with only the required imports.
- IshKebab 5y ago> Think of dead code elimination as removing code that doesn't change the output, while tree shaking instead includes code that could run. Those are both dead code elimination. Webpack even says: > Tree shaking is a term commonly used in the JavaScript context for dead-code elimination.
- littlecranky67 5y agoI know that Microsoft is doing some kind of treeshaking in Blazor (basically a .NET runtime in WASM), since they do not want to ship a full .NET base library to the browser. Not JS related, but the problem is the same - you want to ship as little data as needed on websites that are dynamically loaded.
- 5y ago
- geokon 5y agoAnyone have any experience using a Java treeshaker? I have a Clojure GUI/JFX App that's bloated and I've only managed to slim it down with manual package pruning. I couldn't get Proguard to work with Clojure. Granted proguard is quite baroque so I probably did something wrong. But maybe there is some simpler solution out there for the JVM? Or would this simply be impossible b/c Clojure uses reflection? Though so does Javascript from what I understand..
- setr 5y agoShouldn’t the JVM already be doing this, as part of dead code analysis during the compile step? I believe tree shaking in JavaScript also only handles static analysis — eg imports and function calls. I imagine though usage of something like exec throws DCE right out the window, poisoning the whole program
- geokon 5y agoNo, the Java compiler does not do this. You can try it yourself and see. All dependencies are bundled into the JAR. If you have some large library as a dependency (like BoofCV or JavaFX for instance) then all of the classes and dependencies are bundled. Even if you don't use 99% of them. That's why a lot of Java libraries are broken up into many smaller sub-libraries which you can then manually pick and choose from. It's also why you sometimes see people re-implement smaller sections of larger libraries - b/c you don't want to be forced to drag in the whole mess of code As far as I understand, and I'm murky on the details, but if you're language supports reflection then you can call any dependency during runtime. And so this is not amenable to static analysis. So the Java compiler doesn't have the same guarantees as say a C++ compiler. Most people are running JVM on the server so they just don't care about executable size (and on Android people use proguard). But reflection in general is a source of headaches.. If you use something like GraalVM then it will try to prune dead code but it will not work well with reflections. That all being said, on the specifics, I'm actually not entirely sure how you'd use reflection to arbitrarily call library code at run time. If anyone knows, I'd be curious to see an example
- 5y ago
- esperent 5y agoI suspect (although I haven't checked the code) that this is actually a "dead code elimination" algorithm which is easier to write than a tree shaking algorithm. Tree-shaking is actually "live code inclusion", the opposite side of dead code elimination. https://en.m.wikipedia.org/wiki/Tree_shaking https://en.m.wikipedia.org/wiki/Tree_shaking
- markl42 5y agoMight be worth checking the code but > tree shaking eliminates unused functions from across the bundle by starting at the entry point and only including functions that may be executed Is what this code (tries to!) do
- shwestrick 5y ago"opposite" is not the right term... tree-shaking appears to be a particular technique for dead code elimination. I.e., there are many algorithms you could use to eliminate dead code, and tree-shaking seems to just be a simple one that works well for JS.
- somehnacct3757 5y agoWill the need for bundling and tree-shaking still be so great with HTTP/2 and js module support in browsers?
- danielEM 5y ago3 x Yes: 1. Because of lower overall bandwidth (in many cases it is a SIGNIFICANT difference between code size before and after treeshaking) 2. Because of better startup performance - less code to process, less pings to the server for new modules, less load on servers too (streaming one file vs streaming hundreds of them separately) 3. Because of better performance (eg. removing conditional branches that will never execute) ;-)
- somehnacct3757 5y agoI don't think you understood the question. HTTP/2 allows for one connection to the server to stream multiple files. There's no more network setup penalty for making a request. So all that remains is downloading the files. Now it doesn't matter if they're in one bundle or 20 files. You also don't have to tree shake up front because the execution will do it at runtime. If you deploy some unused modules, the browser will never request them and the server will never send them. For the remaining benefits I can think of - more bundle bytes means more optimal compression; or shaking out functions within a module - I'm not so sure the benefits outweigh the complexity of maintaining and using bundlers. We could be splatting the src directory straight to the server and letting HTTP/2 do the rest.
- somehnacct3757 5y ago;-)
- danielEM 5y agoI think you didn't understood my answer ;-) It still matters if it is one file or 20 (for example for file server loads). Then requesting 20 files separately (even on established connection) doesn't bring you a profit, as even HTTP/1.x browsers try to keep connection "alive". What HTTP/2 brings here is a possibility to push to client files in advance so you can do a bit of "prebundling" on server side. Heard of couple different techniques of "prebundling" on server side BUT none of them will be as simple and performant as bundling it yourself :-) (and want to make sure that you realize that: there is no profit if modules will be requested from server at runtime) And I guess you never tired things like terser (https://terser.org/docs/api-reference https://terser.org/docs/api-reference) so you don't know how much profit it brings not only in a bundle size, but also a startup performance and even runtime performance. Try it, play with different hoisting and mangling options (especially test mangling properties) and check the memory consumption, startup times and performance of your code. Not sure what you're talking about and if you know what you're talking about here: "more bundle bytes means more optimal compression; or shaking out functions within a module" But to give you a hint will tell you to: compress src directory of your project and compress bundled and minified source of your project. See yourself what performs better ;-) Hope helped you! Cheers!
- Tade0 5y agoI love the function and variable names - straight to the point, without trying hard to be too precise.
- tonyedgecombe 5y agoThe interesting stuff seems to be going on in https://github.com/facebook/jscodeshift https://github.com/facebook/jscodeshift
- tyingq 5y agoHow does treeshaking deal with error paths (or other uncommonly used code paths) that aren't often used, but are important not to throw out?
- ygra 5y agoThe same as it deals with common code paths: What can't be guaranteed to never be called, it left in. E.g. you can remove functions in a module that are never imported anywhere else and never called, rather easily. You can also remove branches that are unreachable, e.g. inside an if (false). But for things like function f(s) { if (s === '') throw new Error() ... } there probably are very few tools that will statically be able to tell that f is never called with an empty string and thus the validation logic is not needed. It's JS after all and tools that mangle it all fall on a spectrum between doing very little, but very safely and trying to do too much and breaking code.