4 ms·

I would be shocked if there are improvements over clang (and if there were, clang would fix them) in the general case. clang has a pretty optimized preprocesso

I would be shocked if there are improvements over clang (and if there were, clang would fix them) in the general case. clang has a pretty optimized preprocessor (It even has things like using SSE2 to do some neat things with character processing). GCC has a fairly modern one, but it's still beatable (and clang beats it handily).

pjmlp 13y ago
Well, as I mentioned on a parallel thread, I know of a project taking around 3 hours to compile, that doesn't look very optimized to me.
- DannyBee 13y ago
  You know of a project that takes 3 hours of time to preprocess with clang? I have serious doubts. Overall compilation time is kind of irrelevant to this discussion, because Warp is just a preprocessor. While you can get some X speedup to gcc by replacing the preprocessor, X, as a factor of overall compilation time, is usually 0.2-0.5 in most cases, depending on size of file. I expect the gains warp gets over gcc overall from preprocessing to be similar to those clang gets over gcc overall from preprocessing. (Though it depends on the size of files being compiled, etc). Most companies that want actual fast overall compilation and have the resources, build caching distributed compilation infrastructure (Google, Facebook). As mentioned, if warp is really that much faster than clang's preprocessor that it mattered, clang would be fixed :)
  - pjmlp 13y ago
    Faire enough, I meant full build times.
- bch 13y ago
  What's stated doesn't say anything about optimization. It could be that it takes a long time to run processes that take a long time.
- reubenmorais 13y ago
  What is this even supposed to mean? Is the majority of that time spent preprocessing? How do other compilers perform? Your anecdote alone is absolutely useless.
WalterBright 13y ago
Feel free to try it and see for yourself! Post results here.
- DannyBee 13y ago
  Don't take this the wrong way, but you guys are the ones making performance claims that it's faster! I would expect you guys to post the numbers comparing it to the two most used things out there :)
  - WalterBright 13y ago
    I don't post numbers anymore because I'd always wind up in arguments with people who simply didn't believe them, or thought I'd unfairly manipulated them, or cherry-picked the test cases, whatever. Hence I encourage you to run your own numbers.
    - DannyBee 13y ago
      Fair enough. I had some trouble using the ubuntu 13 packages for gdc, so i downloaded it from the gdc project binaries as of the latest available there, as recommended by the readme. Using that to compile warp with gdc with the flags it suggests (-release is not recognized by gdc, -O3 is), i get a warp that works. For including every file in /usr/include/boost/*.hpp in one .cc file (which produces roughly 16 megabytes of C++ code), we get: [dannyb@mainserver 12:40:56] ~ :) $ time gcc -E e.cc >f In file included from e.cc:101:0: /usr/include/boost/spirit.hpp:18:4: warning: #warning "This header is deprecated. Please use: boost/spirit/include/classic.hpp" [-Wcpp] # warning "This header is deprecated. Please use: boost/spirit/include/classic.hpp" ^ gcc -E e.cc > f 3.18s user 0.25s system 97% cpu 3.528 total [dannyb@mainserver 12:40:51] ~ :) $ time clang -E e.cc >f In file included from e.cc:101: /usr/include/boost/spirit.hpp:18:4: warning: "This header is deprecated. Please use: boost/spirit/include/classic.hpp" [-W#warnings] # warning "This header is deprecated. Please use: boost/spirit/include/classic.hpp" ^ 1 warning generated. clang -E e.cc > f 1.42s user 0.14s system 93% cpu 1.657 total [dannyb@mainserver 12:40:33] ~ :( $ time ./warp/fwarpdrive_gcc4_8_1 -I/usr/include -I/usr/include/c++/4.8 -I/usr/include/x86_64-linux-gnu/c++/4.8 -I/usr/include/x86_64-linux-gnu -I/usr/lib/gcc/x86_64-linux-gnu/4.8/include/ -I/usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed/ e.cc >f cla/usr/include/boost/spirit.hpp(18) : warning: "This header is deprecated. Please use: boost/spirit/include/classic.hpp" ./warp/fwarpdrive_gcc4_8_1 -I/usr/include -I/usr/include/c++/4.8 e.cc 2.88s user 0.06s system 95% cpu 3.080 total I've repeated these timings 10 times, and they are within 0.5% of these numbers each time. I've also tried this on a large C++ project i have, that generates about 200 meg of preprocessed source (that i can't share, sadly) and got similar relative timings. I also tried it on some smaller projects. Based on data i have so far, clang blows warp out of the water by a factor of 2 in most cases i've tried it. The above tests include stdout IO, but the relative numbers are the same without it: [dannyb@mainserver 12:48:24] ~ :( $ time gcc -E e.cc -o f In file included from e.cc:101:0: /usr/include/boost/spirit.hpp:18:4: warning: #warning "This header is deprecated. Please use: boost/spirit/include/classic.hpp" [-Wcpp] # warning "This header is deprecated. Please use: boost/spirit/include/classic.hpp" ^ gcc -E e.cc -o f 3.14s user 0.27s system 99% cpu 3.418 total [dannyb@mainserver 12:48:33] ~ :) $ time clang -E e.cc -o f In file included from e.cc:101: /usr/include/boost/spirit.hpp:18:4: warning: "This header is deprecated. Please use: boost/spirit/include/classic.hpp" [-W#warnings] # warning "This header is deprecated. Please use: boost/spirit/include/classic.hpp" ^ 1 warning generated. clang -E e.cc -o f 1.41s user 0.13s system 94% cpu 1.631 total [dannyb@mainserver 12:48:40] ~ :) $ (I reordered this one to make the timings in the same order as they were before) [dannyb@mainserver 12:47:38] ~ :( $ time ./warp/fwarpdrive_gcc4_8_1 -o f -I/usr/include -I/usr/include/c++/4.8 -I/usr/include/x86_64-linux-gnu/c++/4.8 -I/usr/include/x86_64-linux-gnu -I/usr/lib/gcc/x86_64-linux-gnu/4.8/include/ -I/usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed/ e.cc /usr/include/boost/spirit.hpp(18) : warning: "This header is deprecated. Please use: boost/spirit/include/classic.hpp" ./warp/fwarpdrive_gcc4_8_1 -o f -I/usr/include -I/usr/include/c++/4.8 e.c 2.93s user 0.02s system 99% cpu 2.953 total Warp is definitely faster than GCC, though.
andralex 13y ago
I'd be curious to see how clang does, too. What matters to us is that warp is easy to get into so we can easily adapt it to our build system (in particular multithreaded preprocessing that saves on opening same included files multiple times).

pjmlp 13y ago

Well, as I mentioned on a parallel thread, I know of a project taking around 3 hours to compile, that doesn't look very optimized to me.

DannyBee 13y ago
You know of a project that takes 3 hours of time to preprocess with clang? I have serious doubts. Overall compilation time is kind of irrelevant to this discussion, because Warp is just a preprocessor. While you can get some X speedup to gcc by replacing the preprocessor, X, as a factor of overall compilation time, is usually 0.2-0.5 in most cases, depending on size of file. I expect the gains warp gets over gcc overall from preprocessing to be similar to those clang gets over gcc overall from preprocessing. (Though it depends on the size of files being compiled, etc). Most companies that want actual fast overall compilation and have the resources, build caching distributed compilation infrastructure (Google, Facebook). As mentioned, if warp is really that much faster than clang's preprocessor that it mattered, clang would be fixed :)
- pjmlp 13y ago
  Faire enough, I meant full build times.
bch 13y ago
What's stated doesn't say anything about optimization. It could be that it takes a long time to run processes that take a long time.
reubenmorais 13y ago
What is this even supposed to mean? Is the majority of that time spent preprocessing? How do other compilers perform? Your anecdote alone is absolutely useless.

WalterBright 13y ago

Feel free to try it and see for yourself! Post results here.

DannyBee 13y ago
Don't take this the wrong way, but you guys are the ones making performance claims that it's faster! I would expect you guys to post the numbers comparing it to the two most used things out there :)
- WalterBright 13y ago
  I don't post numbers anymore because I'd always wind up in arguments with people who simply didn't believe them, or thought I'd unfairly manipulated them, or cherry-picked the test cases, whatever. Hence I encourage you to run your own numbers.
  - DannyBee 13y ago
    Fair enough. I had some trouble using the ubuntu 13 packages for gdc, so i downloaded it from the gdc project binaries as of the latest available there, as recommended by the readme. Using that to compile warp with gdc with the flags it suggests (-release is not recognized by gdc, -O3 is), i get a warp that works. For including every file in /usr/include/boost/*.hpp in one .cc file (which produces roughly 16 megabytes of C++ code), we get: [dannyb@mainserver 12:40:56] ~ :) $ time gcc -E e.cc >f In file included from e.cc:101:0: /usr/include/boost/spirit.hpp:18:4: warning: #warning "This header is deprecated. Please use: boost/spirit/include/classic.hpp" [-Wcpp] # warning "This header is deprecated. Please use: boost/spirit/include/classic.hpp" ^ gcc -E e.cc > f 3.18s user 0.25s system 97% cpu 3.528 total [dannyb@mainserver 12:40:51] ~ :) $ time clang -E e.cc >f In file included from e.cc:101: /usr/include/boost/spirit.hpp:18:4: warning: "This header is deprecated. Please use: boost/spirit/include/classic.hpp" [-W#warnings] # warning "This header is deprecated. Please use: boost/spirit/include/classic.hpp" ^ 1 warning generated. clang -E e.cc > f 1.42s user 0.14s system 93% cpu 1.657 total [dannyb@mainserver 12:40:33] ~ :( $ time ./warp/fwarpdrive_gcc4_8_1 -I/usr/include -I/usr/include/c++/4.8 -I/usr/include/x86_64-linux-gnu/c++/4.8 -I/usr/include/x86_64-linux-gnu -I/usr/lib/gcc/x86_64-linux-gnu/4.8/include/ -I/usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed/ e.cc >f cla/usr/include/boost/spirit.hpp(18) : warning: "This header is deprecated. Please use: boost/spirit/include/classic.hpp" ./warp/fwarpdrive_gcc4_8_1 -I/usr/include -I/usr/include/c++/4.8 e.cc 2.88s user 0.06s system 95% cpu 3.080 total I've repeated these timings 10 times, and they are within 0.5% of these numbers each time. I've also tried this on a large C++ project i have, that generates about 200 meg of preprocessed source (that i can't share, sadly) and got similar relative timings. I also tried it on some smaller projects. Based on data i have so far, clang blows warp out of the water by a factor of 2 in most cases i've tried it. The above tests include stdout IO, but the relative numbers are the same without it: [dannyb@mainserver 12:48:24] ~ :( $ time gcc -E e.cc -o f In file included from e.cc:101:0: /usr/include/boost/spirit.hpp:18:4: warning: #warning "This header is deprecated. Please use: boost/spirit/include/classic.hpp" [-Wcpp] # warning "This header is deprecated. Please use: boost/spirit/include/classic.hpp" ^ gcc -E e.cc -o f 3.14s user 0.27s system 99% cpu 3.418 total [dannyb@mainserver 12:48:33] ~ :) $ time clang -E e.cc -o f In file included from e.cc:101: /usr/include/boost/spirit.hpp:18:4: warning: "This header is deprecated. Please use: boost/spirit/include/classic.hpp" [-W#warnings] # warning "This header is deprecated. Please use: boost/spirit/include/classic.hpp" ^ 1 warning generated. clang -E e.cc -o f 1.41s user 0.13s system 94% cpu 1.631 total [dannyb@mainserver 12:48:40] ~ :) $ (I reordered this one to make the timings in the same order as they were before) [dannyb@mainserver 12:47:38] ~ :( $ time ./warp/fwarpdrive_gcc4_8_1 -o f -I/usr/include -I/usr/include/c++/4.8 -I/usr/include/x86_64-linux-gnu/c++/4.8 -I/usr/include/x86_64-linux-gnu -I/usr/lib/gcc/x86_64-linux-gnu/4.8/include/ -I/usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed/ e.cc /usr/include/boost/spirit.hpp(18) : warning: "This header is deprecated. Please use: boost/spirit/include/classic.hpp" ./warp/fwarpdrive_gcc4_8_1 -o f -I/usr/include -I/usr/include/c++/4.8 e.c 2.93s user 0.02s system 99% cpu 2.953 total Warp is definitely faster than GCC, though.

andralex 13y ago

I'd be curious to see how clang does, too. What matters to us is that warp is easy to get into so we can easily adapt it to our build system (in particular multithreaded preprocessing that saves on opening same included files multiple times).