7 ms·

At one point I got Chrome (a genuinely massive C++ project) to compile+link in under six seconds: http://neugierig.org/software/chromium/notes/2011/02/ninja.ht

by evmar 6y ago

At one point I got Chrome (a genuinely massive C++ project) to compile+link in under six seconds: http://neugierig.org/software/chromium/notes/2011/02/ninja.html http://neugierig.org/software/chromium/notes/2011/02/ninja.h... One of the big tricks not mentioned there is to decompose the app into multiple .so files (only when debugging -- release is still a single binary), so that you don't need to rebuild files unaffected by your incremental changes. I wrote more about that here: http://neugierig.org/software/blog/2012/07/gyp-toc.html http://neugierig.org/software/blog/2012/07/gyp-toc.html

benesch 6y ago
Oh, wow, I didn't realize that incremental compiles for Chromium were what had inspired you to write Ninja! I'm a huge fan of Ninja. Composing a project of multiple shared libraries is a trick I've wanted to employ many times. Unfortunately neither Rust nor Go—the languages in which the two largest software projects I've worked on professionally are written—supports that model. (At least, not without throwing away the language-default build system, which has a lot of other costs.)
mwcampbell 6y ago
On the topic of monolithic versus component builds, how much does Chromium's decision to use a large statically-linked module for release builds actually impact performance, both startup and steady-state? Has this been measured? For me, this characteristic of Chromium was always one of the most striking differences between it and IE, and even though Firefox has had one monster DLL for a while, Chromium takes this further. I just wonder how big a deal this obvious difference actually is for performance.
- evmar 6y ago
  I don't recall measuring it in particular. Some thoughts on it anyway: 1. Even if you take a single binary and split it into multiple objects, you still must do the same "linking" work of resolving where functions point to. It's just that work is done as part of startup, at runtime, by the dynamic loader. I have seen (in other contexts) splitting up a big C++ binary into multiple pieces dramatically impact startup time. 2. You can't inline across shared objects. Inlining is the "mother of all optimizations" -- it's the biggest benefit, and most other optimizations interact with it. (For the simplest case, imagine a getter on an object. Inlined, it compiles to a pointer+offset dereference, while across modules it compiles to a lookup in a pointer table plus a function call.) This also means you can't delete unused functions and so on. 3. There's no real memory savings from shared objects in this manner, even when different subprocesses of Chrome only use different subsets of the binary, because the OS only pages in the part of the binary it uses. In general, given there's no user benefit for multiple shared objects, there's little reason to not have a single binary, except for build time. But for releases Chrome does a lot of other work (I think even profile-guided optimization these days, though that is after my time) for performance anyway. In the old days we were making changes to save as little as 40ms of startup time (http://neugierig.org/software/chromium/notes/2009/01/startup.html http://neugierig.org/software/chromium/notes/2009/01/startup... and http://neugierig.org/software/chromium/notes/2009/04/perf.html http://neugierig.org/software/chromium/notes/2009/04/perf.ht...). Looking back at the post now, I see that -fPIC cost 12% of the WebKit JS engine performance, which is maybe another strike against relocatable objects (though maybe these days we use PIE anyway? I am pretty rusty on all this). If I had an app that was multiple shared objects I might be more interested in measuring the benefit before switching it to a single binary, since while I know the benefit is nonzero you have to balance it against the cost of implementation. But in Chrome's case we started with the single binary build, so there was little reason to measure. I would guess in IE's case they were more interested in providing reusable APIs used by multiple other systems. In Chrome's case that was never a design goal.
phendrenad2 6y ago
Why is release still monolithic? Startup speed?
- evmar 6y ago
  I responded to this in https://news.ycombinator.com/item?id=24619216 https://news.ycombinator.com/item?id=24619216
exikyut 6y ago
Ninja is neat. I wonder if a) this is still possible with Chromium today, and b) if/how it might be possible to incrementally build changed functions into new .so files, rewrite the symbol tables in the old .so files to garbage (if needed?), then relink (either incrementally/in-place or from-scratch) the executable. Dependency analysis and garbage collection could delete fully-superceded .so files.
- evmar 6y ago
  I haven't touched it in many years, but: I think (a) it's still possible, per https://chromium.googlesource.com/chromium/src/+/master/docs/component_build.md https://chromium.googlesource.com/chromium/src/+/master/docs... . (b) Creating new .so files that shadow functions in others is an interesting idea! I think you're implying that the executable wouldn't pick them up for free, which I think is true, but I'm not actually sure. I forget the details of how this works but it might be that you could just play tricks with the order of the files to make them shadow. In practice with a fast linker (I used gold at the time, but I think lld is even faster than gold now) it doesn't matter so much as long as you keep the component libraries small enough. But my info is also nearly a decade out of date now...

by evmar 6y ago

benesch 6y ago

Oh, wow, I didn't realize that incremental compiles for Chromium were what had inspired you to write Ninja! I'm a huge fan of Ninja. Composing a project of multiple shared libraries is a trick I've wanted to employ many times. Unfortunately neither Rust nor Go—the languages in which the two largest software projects I've worked on professionally are written—supports that model. (At least, not without throwing away the language-default build system, which has a lot of other costs.)

mwcampbell 6y ago

On the topic of monolithic versus component builds, how much does Chromium's decision to use a large statically-linked module for release builds actually impact performance, both startup and steady-state? Has this been measured? For me, this characteristic of Chromium was always one of the most striking differences between it and IE, and even though Firefox has had one monster DLL for a while, Chromium takes this further. I just wonder how big a deal this obvious difference actually is for performance.

evmar 6y ago
I don't recall measuring it in particular. Some thoughts on it anyway: 1. Even if you take a single binary and split it into multiple objects, you still must do the same "linking" work of resolving where functions point to. It's just that work is done as part of startup, at runtime, by the dynamic loader. I have seen (in other contexts) splitting up a big C++ binary into multiple pieces dramatically impact startup time. 2. You can't inline across shared objects. Inlining is the "mother of all optimizations" -- it's the biggest benefit, and most other optimizations interact with it. (For the simplest case, imagine a getter on an object. Inlined, it compiles to a pointer+offset dereference, while across modules it compiles to a lookup in a pointer table plus a function call.) This also means you can't delete unused functions and so on. 3. There's no real memory savings from shared objects in this manner, even when different subprocesses of Chrome only use different subsets of the binary, because the OS only pages in the part of the binary it uses. In general, given there's no user benefit for multiple shared objects, there's little reason to not have a single binary, except for build time. But for releases Chrome does a lot of other work (I think even profile-guided optimization these days, though that is after my time) for performance anyway. In the old days we were making changes to save as little as 40ms of startup time (http://neugierig.org/software/chromium/notes/2009/01/startup.html http://neugierig.org/software/chromium/notes/2009/01/startup... and http://neugierig.org/software/chromium/notes/2009/04/perf.html http://neugierig.org/software/chromium/notes/2009/04/perf.ht...). Looking back at the post now, I see that -fPIC cost 12% of the WebKit JS engine performance, which is maybe another strike against relocatable objects (though maybe these days we use PIE anyway? I am pretty rusty on all this). If I had an app that was multiple shared objects I might be more interested in measuring the benefit before switching it to a single binary, since while I know the benefit is nonzero you have to balance it against the cost of implementation. But in Chrome's case we started with the single binary build, so there was little reason to measure. I would guess in IE's case they were more interested in providing reusable APIs used by multiple other systems. In Chrome's case that was never a design goal.

phendrenad2 6y ago

Why is release still monolithic? Startup speed?

evmar 6y ago
I responded to this in https://news.ycombinator.com/item?id=24619216 https://news.ycombinator.com/item?id=24619216

exikyut 6y ago

Ninja is neat. I wonder if a) this is still possible with Chromium today, and b) if/how it might be possible to incrementally build changed functions into new .so files, rewrite the symbol tables in the old .so files to garbage (if needed?), then relink (either incrementally/in-place or from-scratch) the executable. Dependency analysis and garbage collection could delete fully-superceded .so files.

evmar 6y ago
I haven't touched it in many years, but: I think (a) it's still possible, per https://chromium.googlesource.com/chromium/src/+/master/docs/component_build.md https://chromium.googlesource.com/chromium/src/+/master/docs... . (b) Creating new .so files that shadow functions in others is an interesting idea! I think you're implying that the executable wouldn't pick them up for free, which I think is true, but I'm not actually sure. I forget the details of how this works but it might be that you could just play tricks with the order of the files to make them shadow. In practice with a fast linker (I used gold at the time, but I think lld is even faster than gold now) it doesn't matter so much as long as you keep the component libraries small enough. But my info is also nearly a decade out of date now...