3 ms·
There are a million responses to your comment, but unless I missed something, no one seems to be redirecting you back to the actual problem at hand and are reco
by saurik 9y ago
There are a million responses to your comment, but unless I missed something, no one seems to be redirecting you back to the actual problem at hand and are recommending all sorts of generic ways to reduce the time required to compile a large C++ project... but the issue here was not the generic one: the entire thing that made this article interesting is that his CPU was not being used at all, so this has nothing to do with "a Chrome build is truly a computational load to be reckoned with". He even goes out of his way to show his CPU load graphs so we can see multiple seconds in a row where his computer is 98% idle and yet he still can't even move his mouse.
Your comment, and essentially every other single one on this entire thread thereby makes me wonder if anyone on this subthread read the article :/.
OK, I decided to search for the word "process", and found one person responding to you who did read the article, and depressingly only a handful of people even responding to the top-level article who apparently read the article. This entire post is such a great example of "the problem with this kind of discussion forum" :/.
https://news.ycombinator.com/item?id=14735977 https://news.ycombinator.com/item?id=14735977
REGARDLESS...
What was described in this article wasn't "Chrome's build is too slow", it was "there is a weird issue in Windows 10 (which apparently wasn't even a problem with Windows 7, and so we could easily argue is a regression) where process destruction takes a global lock on something that is seemingly shared with basic things like UI message passing". The fact that he was running a Chrome build to demonstrate how this manages to occasionally more than decimate the processing power of his computer was just a random example that this user ran into: it could have been any task doing anything that involved spawning a lot of processes, and the story would have been exactly the same.
Now, that said, if you want to redirect this to "what can the Chrome team do to mitigate this issue", and you want the answer to not be "please please lean on Microsoft to do something about this weird lock regression in Windows 10 so as to improve the process spawn parallelism for every project, not just compiling Chrome"... well, "sure", we can say you are "doing this wrong", and it is arguably even a "trivial fix"!
Right now, the C++ compiler pipeline tends to spawn at least one (if not more than one) process per translation unit. If gcc or clang (I'm not sure which one would be easier for this; I'm going to be honest and say "probably clang" even though it feels like a punch in the gut) were to be refactored into a build server and then the build manager (make or cmake or ninja or whatever it is Google decided to write this week) connected to a pool of those to queue builds, you would work around this issue and apparently get a noticeable speed up in Chrome compiles on Windows 10, due to the existence of this process destruction lock.
One could even imagine ninja just building clang into itself and then running clang directly in-process on a large number of threads (rather than a large number of processes), and so there would only be a single giant process that did the entire build, end-to-end. That would probably bring a bunch of other performance advantages to bear as well, and is probably a weekend-long project to get to a proof-of-concept stage for an intern, come to think of it... you should get on it! ;P
- Dylan16807 9y ago> Your comment, and essentially every other single one on this entire thread thereby makes me wonder if anyone on this subthread read the article :/. Just let people talk about what they want to talk about, maybe? The main problem in the article is interesting but far less actionable than the overall situation of slow compilation. Do you want things like rampant speculation and insulting windows 10? Do you expect everyone to pull out kernel debuggers to be able to make directly relevant comments? It's okay to talk about a related issue. Concluding that they didn't read the article is kind of insulting.
- titzer 9y agoYou're right. I didn't read the whole article. I skimmed it. But I did actually pick up on the locking problem and see the graphs with huge amounts of idle time. I knew this comment was only marginally related to the actual article. So, :-/ However I suspect that as soon as that lock regression in Windows is fixed, that monster CPU load is coming back home to roost, and the workstation is going to be just as dead as my 64 core Linux workstation has been when I've actually '-j 500' without gomacc up and running correctly. So, by all means, Microsoft should fix this lock regression. But there's this...elephant...in...this room here.