5 ms·

It is interesting to read how the compiler behaves when using instructions but the number of generated instructions is only one metric. As always, there is noth

It is interesting to read how the compiler behaves when using instructions but the number of generated instructions is only one metric. As always, there is nothing better than measuring the runtime of your actual implementation. I'm writing this because I had a tight-running loop that returned a bool and in 1 out of 100k or so cases, it would return false. It turns out that instead of checking that bool in each loop iteration, it was actually significantly faster to check nothing and throw an exception in that very rare case. Something like this: try { while ( true ) { foo.read(); } } catch ( const std::exception& ) { // foo.read throws on eof } instead of this: while ( !foo.eof() ) { foo.read(); } This might not be the most C++-like code but it saved me like 50% of runtime in one case. Afaik as long as exceptions are not thrown, there is zero overhead, which would explain this. And, the call to eof might have been too complex and long-running, maybe I could have optimized that one instead. But there are abstractions that might hinder such optimizations.

throwaway9870 4y ago
You are making two functions calls in the second loop, vs one in the first. If the result of read() could be checked for eof, then you would remove one call. Also, since the branch is so easily predicted, I would be surprised if there is much of a performance hit once the second call is removed. If you see a 50% speed-up in the above code, I question what read is doing and the implementation of eof(). I would expect read() to dominate the if statement unless eof is extremely expensive. I have written dozens of read()/eof() functions over the years and read() always dominates because the read involves copying memory and/or making OS calls, etc., while eof() is a simple comparison. So if read() dominates runtime, example 1 and 2 above should not be anywhere close to 50% different.
- mxmlnkn 4y ago
  That particular code came from a bit reader class. So, if you are reading only a few bits per call, then the read call becomes very cheap. It might be as "simple" as this pseudocode if ( offset + requestedBitCount < 64 ) { offset += requestedBitCount; return ( bitBuffer >> offset ) & nLowestBitsSet( requestedBitCount ); } And yes, I think the eof call might have been too complex especially in contrast to the short read call. It probably should only check a flag that might be set by the read call automatically.
  - throwaway9870 4y ago
    I still don't get it. You need to put an if statement in that function to raise the exception. So I still don't see where 50% comes from.
    - mxmlnkn 4y ago
      At the very least you are redundantly executing logic without the exception. The check for eof has to be done implicitly anyway inside read because it has to fill the bit buffer with data from the byte buffer or the byte buffer with data from the file. And if both fail, then we already know the result of eof, so no need to duplicate checking for eof in the outer read calling loop. Here is the full commit with ad-hoc benchmark results in the commit message: https://github.com/mxmlnkn/pragzip/commit/0b1af498377838c30fea504191e127027802d2d2 https://github.com/mxmlnkn/pragzip/commit/0b1af498377838c30f... and here the benchmarks I ran at that time: https://github.com/mxmlnkn/pragzip/blob/0b1af498377838c30fea504191e127027802d2d2/tests/benchmarkBitReader.cpp https://github.com/mxmlnkn/pragzip/blob/0b1af498377838c30fea... In this commit, I even replace std::optional with an exception. But, I didn't document benchmark results and the commit message begins with "try" so they probably weren't significantly better but also not worse or else I wouldn't commit it: https://github.com/mxmlnkn/pragzip/commit/0f1babaf3b9cbe291269a43078c814ea738c0d35 https://github.com/mxmlnkn/pragzip/commit/0f1babaf3b9cbe2912... At the end of that file are infrequently updated results of the benchmarks. As you can see, it's part of my random-seekable multi-threaded gzip and bzip2 parallel decompression libraries.
      - jnordwick 4y ago
        Wihtout seeing the assembly it is difficult to say but for such a large difference, the culprit often seems to the loop iop cache. If you check the alignment of the top of the loop instructions (and it inlined away the calls) sometimes you can see an alignment change for some trivial bullshit that has vasat performance impacts because it causes a series of other missed optmizations. That damn loop cache is great for performance when you hit it,but sucks horrible when you don't even think out it and spend days trying to figure out some performance quandry.
rightbyte 4y ago
In the exception code a foo.eof() check need to be in foo.read(). So I guess the runtime in theory should be the same and the implementation happened to be slow. Maybe eol() did some syscall each time instead of caching a EOF int like in C.

throwaway9870 4y ago

You are making two functions calls in the second loop, vs one in the first. If the result of read() could be checked for eof, then you would remove one call. Also, since the branch is so easily predicted, I would be surprised if there is much of a performance hit once the second call is removed. If you see a 50% speed-up in the above code, I question what read is doing and the implementation of eof(). I would expect read() to dominate the if statement unless eof is extremely expensive. I have written dozens of read()/eof() functions over the years and read() always dominates because the read involves copying memory and/or making OS calls, etc., while eof() is a simple comparison. So if read() dominates runtime, example 1 and 2 above should not be anywhere close to 50% different.

mxmlnkn 4y ago
That particular code came from a bit reader class. So, if you are reading only a few bits per call, then the read call becomes very cheap. It might be as "simple" as this pseudocode if ( offset + requestedBitCount < 64 ) { offset += requestedBitCount; return ( bitBuffer >> offset ) & nLowestBitsSet( requestedBitCount ); } And yes, I think the eof call might have been too complex especially in contrast to the short read call. It probably should only check a flag that might be set by the read call automatically.
- throwaway9870 4y ago
  I still don't get it. You need to put an if statement in that function to raise the exception. So I still don't see where 50% comes from.
  - mxmlnkn 4y ago
    At the very least you are redundantly executing logic without the exception. The check for eof has to be done implicitly anyway inside read because it has to fill the bit buffer with data from the byte buffer or the byte buffer with data from the file. And if both fail, then we already know the result of eof, so no need to duplicate checking for eof in the outer read calling loop. Here is the full commit with ad-hoc benchmark results in the commit message: https://github.com/mxmlnkn/pragzip/commit/0b1af498377838c30fea504191e127027802d2d2 https://github.com/mxmlnkn/pragzip/commit/0b1af498377838c30f... and here the benchmarks I ran at that time: https://github.com/mxmlnkn/pragzip/blob/0b1af498377838c30fea504191e127027802d2d2/tests/benchmarkBitReader.cpp https://github.com/mxmlnkn/pragzip/blob/0b1af498377838c30fea... In this commit, I even replace std::optional with an exception. But, I didn't document benchmark results and the commit message begins with "try" so they probably weren't significantly better but also not worse or else I wouldn't commit it: https://github.com/mxmlnkn/pragzip/commit/0f1babaf3b9cbe291269a43078c814ea738c0d35 https://github.com/mxmlnkn/pragzip/commit/0f1babaf3b9cbe2912... At the end of that file are infrequently updated results of the benchmarks. As you can see, it's part of my random-seekable multi-threaded gzip and bzip2 parallel decompression libraries.
    - jnordwick 4y ago
      Wihtout seeing the assembly it is difficult to say but for such a large difference, the culprit often seems to the loop iop cache. If you check the alignment of the top of the loop instructions (and it inlined away the calls) sometimes you can see an alignment change for some trivial bullshit that has vasat performance impacts because it causes a series of other missed optmizations. That damn loop cache is great for performance when you hit it,but sucks horrible when you don't even think out it and spend days trying to figure out some performance quandry.

rightbyte 4y ago

In the exception code a foo.eof() check need to be in foo.read(). So I guess the runtime in theory should be the same and the implementation happened to be slow. Maybe eol() did some syscall each time instead of caching a EOF int like in C.