4 ms·

Only if your speculative reads do cause irreversible side-effects on those caches. You could implement them in a way that doesn't modify the caches... but that

Only if your speculative reads do cause irreversible side-effects on those caches. You could implement them in a way that doesn't modify the caches... but that would be complicated and probably use more power and have lower performance.

webaholic 9y ago
One of the main reasons for speculative execution is to fetch data into the caches ahead of them being needed. If you don't modify the cache, then you throw that away. May be one way would be to use a smaller, separate cache for speculative execution and then copy that value to the regular cache once speculation is confirmed? This would add a one cycle latency for cache-to-cache transfer but there might be better ways.
- caf 9y ago
  This might actually improve performance because it would prevent actually-hot data being evicted from the cache in favour of cold data that was loaded in a not-taken speculated branch.
- rasz 9y ago
  no need for cache-to-cache transfer, just like with invisible registers we could have cache line renaming
- gpderetta 9y ago
  It is not enough. The cache line still need be evicted (or at least marked as no longer exclusive) from other cpus, so the side effect is still visible.
gmueckl 9y ago
There does not need to be a performance hit, but cache complexity must rise: speculative execution must use a separate cache for any data that was fetched speculatively. Only when that branch is truly accepted must that data enter the "real" cache. As long as speculative execution does not go on for too long, these secondary caches can stay really tiny (a handful of cache lines maybe).
- londons_explore 9y ago
  The "speculation time" can be hundreds of cycles if you have a branch or memory read that takes a long time to resolve. This problem is already solved with speculative writes to main memory - a speculative store buffer keeps a sequence of memory operations which need to be done when the operation retires. These buffers are very power hungry, because every future speculative read must check every entry in the speculative store buffer to see if it is re-reading a previously written address. That many to many mapping leads to an exponential amount of checking logic. The same could be done for cache reads/writes, but I have a feeling it would quickly get very complex, large, and power hungry.
  - caf 9y ago
    Those hundreds of cycles of speculative execution can't include more than a handful of cache modifications though, because a change to the caching state implies a miss in the speculated execution itself. So you can't have more than a small number of those before the original stall is over and the misprediction resolved.
  - gmueckl 9y ago
    What you are describing is sinply plain associative memory. If I remember correctly, this is complex in its imolementation, but does not grow exponentially. Plesse correct me if I am wrong.
    - londons_explore 9y ago
      fully associative memory is generally very power hungry. Thats why in CPU's caches are usually "2 way associative" or "4 way associative". That means the data you're looking for might be in one of 2 (or 4) places. Fully associative means the data you're looking for might be in any memory slot, and you're gonna have to check them all. Checking them all in parallel is possible, so it isn't a speed issue, but it is a massive power issue. Average power use is the main limiting factor in CPU's today. In general in a CPU, transistors which stay in the same state don't use much power. Transistors changing state use power. In a fully associative memory, the transistors doing the comparing change state with every comparison. Whereas with a regular memory only the transistors for the individual bit of the memory being read or written change state and use power. (the above is a simplification, but contains the key elements).

webaholic 9y ago

One of the main reasons for speculative execution is to fetch data into the caches ahead of them being needed. If you don't modify the cache, then you throw that away. May be one way would be to use a smaller, separate cache for speculative execution and then copy that value to the regular cache once speculation is confirmed? This would add a one cycle latency for cache-to-cache transfer but there might be better ways.

caf 9y ago
This might actually improve performance because it would prevent actually-hot data being evicted from the cache in favour of cold data that was loaded in a not-taken speculated branch.
rasz 9y ago
no need for cache-to-cache transfer, just like with invisible registers we could have cache line renaming
gpderetta 9y ago
It is not enough. The cache line still need be evicted (or at least marked as no longer exclusive) from other cpus, so the side effect is still visible.

gmueckl 9y ago

There does not need to be a performance hit, but cache complexity must rise: speculative execution must use a separate cache for any data that was fetched speculatively. Only when that branch is truly accepted must that data enter the "real" cache. As long as speculative execution does not go on for too long, these secondary caches can stay really tiny (a handful of cache lines maybe).

londons_explore 9y ago
The "speculation time" can be hundreds of cycles if you have a branch or memory read that takes a long time to resolve. This problem is already solved with speculative writes to main memory - a speculative store buffer keeps a sequence of memory operations which need to be done when the operation retires. These buffers are very power hungry, because every future speculative read must check every entry in the speculative store buffer to see if it is re-reading a previously written address. That many to many mapping leads to an exponential amount of checking logic. The same could be done for cache reads/writes, but I have a feeling it would quickly get very complex, large, and power hungry.
- caf 9y ago
  Those hundreds of cycles of speculative execution can't include more than a handful of cache modifications though, because a change to the caching state implies a miss in the speculated execution itself. So you can't have more than a small number of those before the original stall is over and the misprediction resolved.
- gmueckl 9y ago
  What you are describing is sinply plain associative memory. If I remember correctly, this is complex in its imolementation, but does not grow exponentially. Plesse correct me if I am wrong.
  - londons_explore 9y ago
    fully associative memory is generally very power hungry. Thats why in CPU's caches are usually "2 way associative" or "4 way associative". That means the data you're looking for might be in one of 2 (or 4) places. Fully associative means the data you're looking for might be in any memory slot, and you're gonna have to check them all. Checking them all in parallel is possible, so it isn't a speed issue, but it is a massive power issue. Average power use is the main limiting factor in CPU's today. In general in a CPU, transistors which stay in the same state don't use much power. Transistors changing state use power. In a fully associative memory, the transistors doing the comparing change state with every comparison. Whereas with a regular memory only the transistors for the individual bit of the memory being read or written change state and use power. (the above is a simplification, but contains the key elements).