4 ms·
> Memory usage building Firefox with debug enabled was reduced from 15GB to 3.5GB; link time from 1700 seconds to 350 seconds. This is huge- you can now do thi
by krakensden 13y ago
> Memory usage building Firefox with debug enabled was reduced from 15GB to 3.5GB; link time from 1700 seconds to 350 seconds.
This is huge- you can now do this on a laptop
- supergauntlet 13y agowell, you could do it on a laptop before, you just needed a much beefier laptop. But that's really impressive, how did they cut the resource usage to a quarter of previous usage?
- alok-g 13y agoI am very curious too. Such drastic improvements often indicates something wrong before (either in design or implementation) that got fixed.
- Denvercoder9 13y agoI bet it is caused by these two optimizations: > Early removal of virtual methods reduces the size of object files and improves link-time memory usage and compile time. > Function bodies are now loaded on-demand and released early improving overall memory usage at link time.
- deleted 13y ago[deleted]
- nly 13y ago15 GB was presumably with LTO enabled, you could always build it on a fairly average laptop with it disabled. This is great news though, as LTO is pretty awesome.
- hf 13y agoFor the uninitiated: LTO stands for Link-time optimization and happens when the compiler merges/links all separately-compiled object files into one (executable or library). Although it seems obvious that this might be a good idea, why would it 1) use exorbitant amounts of memory; and 2) be "pretty awesome" instead of, say, mildly useful? edit: And both questions satisfactorily answered in the time it took me to peruse the preamble of https://en.wikipedia.org/wiki/Interprocedural_optimization https://en.wikipedia.org/wiki/Interprocedural_optimization Thank you!
- mattgrice 13y agoLink-time optimization is not a very descriptive term. It's usually called whole-program or interprocedural optimization. Link-time optimization is just how it has been implemented. It uses a lot of memory because it requires keeping a representation of more or less the entire program in memory at one time, in a format which is amenable to analysis. It is not out of the ordinary for an optimizer's internal representation to be on the order of 1000x the size of the source code.
- anaphor 13y agoFor an example of a compiler other than GCC that does whole-program optimization see the Stalin Scheme compiler: https://github.com/barak/stalin https://github.com/barak/stalin
- dietrichepp 13y agoLTO, as the name implies, means that you do another optimization pass at link time. This enables lots of optimizations that don't work when you look at one module at a time. Functions can be inlined across module boundaries, even when they're not declared inline. You can turn virtual functions into regular functions, if you know that the virtual function is never overridden, or if you can derive the exact type. You can change calling conventions for functions. You can do better escape and aliasing analysis. If a function is only called once, then you can probably optimize it a lot better because you know exactly how it will be called. As with all optimizations, not all programs will see any significant benefit. Programs with heavy inner loops like physics simulators and graphics processors will not see much benefit, since the local optimizer works well enough. Programs like compilers, interpreters, and web browsers will see larger benefits. However, the benefits can be high—30% improvements in running time are not unheard of. In short, LTO is a high-cost, high-benefit optimization.
- frozenport 13y agoAs a Gentoo user, we always did it on our laptop. Build time was about 30 minutes.
- bgirard 13y agoFirefox recently (Trunk as of December) started using Unified Sources so builds are now typically 7-9 minutes on laptops now for non Windows platforms.