He's talking about branch prediction because it's intimately tied with speculative execution. He's trying to explain why the standard 'branchy' version of bina

He's talking about branch prediction because it's intimately tied with speculative execution. He's trying to explain why the standard 'branchy' version of binary search has faster performance than one might naively expect, thus why the speedup from increasing MLP (memory level parallism) is less than one might expect. When the processor reaches a conditional branch (the 'if' statement comparing the test value), rather than waiting for the actual results of the comparison to be known (and thus waiting for the test value to be loaded), it blindly guesses one of the branches and continues executing instructions as fast as it can. It doesn't commit the results of these instructions ('retire' the instructions) until the real answer is known, but crucially, it executes the loads sooner than they would otherwise be executed. Most of these early loads are along the wrong path and the load is wasted, but half the time, the first guess is right, and half the time, the branch after that, etc. The end result is that branchy binary search ends up roughly twice as fast as a branchless version that waits for the loads to complete and efficiently makes only the loads that need to be made.