4 ms·
I strongly disagree with the assertion that C compilation is I/O bound. If you have optimizations turned off, this is probably true, but if you are using a dec
by aidenn0 17y ago
I strongly disagree with the assertion that C compilation is I/O bound. If you have optimizations turned off, this is probably true, but if you are using a decent optimizing compiler, this is not the case.
We ran some tests where I work on a pretty good sized code-base and found that we were CPU bound, even on an 8-core system.
[edit]
There is an exception, if you are using networked file systems of some type (especially clearcase dynamic views) then you are almost certainly I/O bound.
- __david__ 17y agoIf C compilation were I/O bound then ccache would not speed things up. It does. I've used it on my NFS mounted filesystems and it speeds things up there too. There is clearly a point where if your filesystem is slow enough that the process does become I/O bound. I have no experience with ClearCase but from what I've heard it's molasses slow and might be to that point.
- JoeAltmaier 17y agoYears ago I implemented a network file cache using a file-modification-count returned in every open handle. At that point it became an order of magnitude faster to build over the network than local, because the local file system had no such cache. Anyway this race between speed, space and network performance has been around a while.
- leif 17y agoI wish I could upvote this more. There is no way compilation is I/O bound. Even over the network this is true. My old lab had homedirs all on NFS (gigabit I think, but that's not too important) and all our builds were CPU-bound, even without raising the -j argument from cores+1. The same lab (before I was there) even wrote a paper on why the "compile" test is a horrible one for measuring filesystem performance; it just doesn't do enough I/O. I'd be interested to see what happens if you were to compile something large written in assembly (or close to it), but I'd put money that just the sequential optimizer would be slower than I/O. EDIT: found the paper and an article about it http://www.linux-mag.com/cache/7464/1.html http://www.linux-mag.com/cache/7464/1.html http://www.fsl.cs.sunysb.edu/project-fsbench.html http://www.fsl.cs.sunysb.edu/project-fsbench.html
- chadaustin 17y agoThat was kind of tangential point to the article, and it was merely inspiration for the name, but I'll ask Dusty for the citation. This was years ago, so who knows. [Update: he says he thinks it was on the D mailing list years ago, and was probably related to the fact that every source file in a nontrivial C program includes dozens or hundreds of headers, especially with a naive compiler.] [Update 2: TinyCC http://bellard.org/tcc/ http://bellard.org/tcc/ compiles at 30 MB/s. It's not hard to imagine a dual-core CPU on a conventional drive failing to feed the compiler enough source to be CPU-bound.]
- blasdel 17y agoHave you tested the difference between cold and warm FS caches on a project with a shitload of files? There's a reason why the core Linux devs love Intel SSDs! The reason that parallel make helps so much (especially with 2x more jobs than processors) is that you aren't idling while blocked for I/O, and are piling up the I/O queue higher so the scheduler can de-randomize some I/O (elevator algorithm FTW).