Those hundreds of cycles of speculative execution can't include more than a handful of cache modifications though, because a change to the caching state implies

Those hundreds of cycles of speculative execution can't include more than a handful of cache modifications though, because a change to the caching state implies a miss in the speculated execution itself. So you can't have more than a small number of those before the original stall is over and the misprediction resolved.