4 ms·

Well I tried it on an IvyBridge Xeon E5-2680-V2, gcc version 4.4.3 with -O3, table size = 32. Single table at a time ends up being faster: 155 M searches

Well I tried it on an IvyBridge Xeon E5-2680-V2, gcc version 4.4.3 with -O3, table size = 32. Single table at a time ends up being faster: 155 M searches / sec for one table at a time. 108 M searches / sec for two tables at a time. 88 M searches / sec for four tables at a time. The interleaved version of the code looks like this (for two at a time): #define LOGSIZE 5 unsigned dsearch (unsigned key, unsigned * vector) { unsigned size = (1 << LOGSIZE) / 2; unsigned * low1 = vector; unsigned * low2 = vector + ((1 << LOGSIZE) / 2); unsigned i; for (i = LOGSIZE - 1; i != 0; i--) { size /= 2; unsigned mid1 = low1[size]; unsigned mid2 = low2[size]; if (mid1 <= key) low1 += size; if (mid2 <= key) low2 += size; } if (*low2 == key) low1 = low2; return low1 - vector; } The compiled code ends up branch free (gcc does a good job), but is 30 instructions instead of 18. Oh well, was worth a shot..

nkurz 12y ago
Excellent for doing the the actual test! My interpretation might be the opposite of yours: it's not that the interleaved version fails to overcome the latency, rather the apparently one-at-a-time version is already able to issue all the loads speculatively, and thus isn't being hurt (much?) by latency. If the addresses are known ahead of time, processors are great about issuing loads as soon as possible. They 'retire' them in order, but if the situation is right they can 'execute' them dozens of cycles ahead of time, even across loop iterations. My guess would be that the slowdown from the interleaved searches is because spreading the loads across multiple cachelines actually works against you, since stalls become more rather than less frequent. Ivy Bridge can have (about?) 10 outstanding L1 requests, which it can issue two per cycle. With a 5 cycle (I think?) L1 latency, this is enough to cover up the latency. Sustaining the two loads per cycle is hard, though, since only 4 ops per cycle can retired. But there are not many other operations here, so the unrolled loop may come close.
tjpoutanen 12y ago
I think the reason it's slower for you is that the CPU pipeline is starved for data... even when it is in the same cache line. Since we are geeking out on binary search on a Saturday night, here's an approach I developed for traversing multiple binary trees in parallel with SIMD instructions, no branches and data prefetching. It's used at Bing to calculate the document score that ranks Web search results, and traverses thousands of Machine Learning derived decision trees. It resulted in a 4x speedup over the naive approach. http://web.cse.ohio-state.edu/~ren/papers/CGO13_BinRen.pdf http://web.cse.ohio-state.edu/~ren/papers/CGO13_BinRen.pdf
- vtuulos 12y ago
  This is very cool. Thanks for sharing!

nkurz 12y ago

Excellent for doing the the actual test! My interpretation might be the opposite of yours: it's not that the interleaved version fails to overcome the latency, rather the apparently one-at-a-time version is already able to issue all the loads speculatively, and thus isn't being hurt (much?) by latency. If the addresses are known ahead of time, processors are great about issuing loads as soon as possible. They 'retire' them in order, but if the situation is right they can 'execute' them dozens of cycles ahead of time, even across loop iterations. My guess would be that the slowdown from the interleaved searches is because spreading the loads across multiple cachelines actually works against you, since stalls become more rather than less frequent. Ivy Bridge can have (about?) 10 outstanding L1 requests, which it can issue two per cycle. With a 5 cycle (I think?) L1 latency, this is enough to cover up the latency. Sustaining the two loads per cycle is hard, though, since only 4 ops per cycle can retired. But there are not many other operations here, so the unrolled loop may come close.

tjpoutanen 12y ago

I think the reason it's slower for you is that the CPU pipeline is starved for data... even when it is in the same cache line. Since we are geeking out on binary search on a Saturday night, here's an approach I developed for traversing multiple binary trees in parallel with SIMD instructions, no branches and data prefetching. It's used at Bing to calculate the document score that ranks Web search results, and traverses thousands of Machine Learning derived decision trees. It resulted in a 4x speedup over the naive approach. http://web.cse.ohio-state.edu/~ren/papers/CGO13_BinRen.pdf http://web.cse.ohio-state.edu/~ren/papers/CGO13_BinRen.pdf

vtuulos 12y ago
This is very cool. Thanks for sharing!