4 ms·

Prefetchers don't do anything do reduce the power consumption involved in streaming data from memory, they only reduce the latency seen by the CPU. In fact, si

Prefetchers don't do anything do reduce the power consumption involved in streaming data from memory, they only reduce the latency seen by the CPU. In fact, since the prefetcher sometimes makes mispredictions, they tend to increase power.

baybal2 7y ago
But they do reduce cache misses and unneeded cache evictions dramatically, and that allows CPUs to use external memory less. This reduces the power consumption.
- mlyle 7y ago
  A prefetcher reduces cache misses by retrieving the data first; the number of memory accesses to get the data remains the same. [Though it may be through a more efficient pipelined operation]. How does a prefetcher reduce unneeded cache evictions? It fetches data that may not be needed, increasing cache evictions. The only thing a prefetcher can do to improve power efficiency, IMO, is by increasing utilization-- if the processor stays 99% busy instead of 97% busy, the fixed portions of the power budget are amortized over more operations.
  - tempguy9999 7y ago
    > A prefetcher reduces cache misses by retrieving the data first I'm struggling with that. To avoid a cache miss the prefetcher woul have to prefetch the data from memory so early, and memory is so slow, that an entire train of in-flight instructions could complete, and then likely have the CPU pause, before the fetch-from-ram completed. I may be misunderstanding you though.
    - mlyle 7y ago
      The whole point of a prefetcher is to avoid cache misses. What do you think a prefetcher is for? Memory is very latent, and moderately slow. Hardware prefetching tends to mostly make memory accesses longer, so you don't pay the RAS/CAS penalties and instead stream just more memory words back to fill an additional cache line beyond the one demanded. e.g. One key part of Intel L2 hardware prefetching is working upon cache line pairs. If you miss on an even numbered cache line, and counters support it, it decides to also retrieve the next odd-numbered cache line at the same time. The downside of a hardware prefetcher is three things, both stemming from it perhaps retrieving memory that isn't needed. 1) it can cause useful information to be evicted from cache if a prefetch is errant. 2) it can tie up the memory bus with an unnecessary access and make a necessary one more latent. 3) it can consume power for a prefetch that isn't needed.
      - tempguy9999 7y ago
        > What do you think a prefetcher is for? We may be talking past each other. My understanding of a prefetcher is to get the data ASAP. It typically prefetches from cache, not from memory (if it prefetches from mem it's so slow it's useless). > Memory is very latent, and moderately slow Those 2 terms seem synonymous, but memory access is slooow - from doc in front of me, for haswell: L1 access hit, 4 or 5 cycles latency; L2 hit 12 cycles latency; L3 hit 36 to 66 cycles latency; ram latency 36 cycles + 57 NS / 62 cycles + 100NS depending (found ref, it's https://www.7-cpu.com/cpu/Haswell.html https://www.7-cpu.com/cpu/Haswell.html) for the ram latencies that's for different setups (single/dual cpu , I'm not sure) running ~3.5GHz. So for 36C + 57NS that 36 + (3.5 * 57) = 235 cycles latency. And it can get worse. Much worse. I'm sorry i can't find it now but in hennesy & patterson I recall it being said that prefetchers can hide most of the latency of a L1 or L2 miss if it hits L3, but if it needs main mem then it's stuffed. Sorry, that is from memory and may be wrong! But I'm pretty sure a prefetcher is wasted if it needs to hit ram. > If you miss on an even numbered cache line, and counters support it, it decides to also retrieve the next odd-numbered cache line at the same time. I see, but I think it does better these days, it'll remember a stride and prefetch by the next stride. Even striding backwards through mem (cache)! But it won't cross a page boundary. Really need an expert to chime in here, anyone?
        mlyle 7y ago
        > My understanding of a prefetcher is to get the data ASAP. It typically prefetches from cache, not from memory (if it prefetches from mem it's so slow it's useless). The prefetcher runs at whatever level it's at (L1/L2) and fetches from whatever level has the memory. So the L2 prefetcher may be grabbing from L3, and may be grabbing from SDRAM. > ram latency 36 cycles + 57 NS / 62 cycles + 100NS depending That's the ram LATENCY ON RANDOM ACCESS. If you extend an access to fetch the next sequential line because you think it will be used, you don't pay any of the latency penalty-- you might need to strobe a column access but words keep streaming out. For this reason sequential prefetch is particularly powerful. Even if we're just retrieving from a single SDRAM channel, it's just another 3ns to continue on to retrieve the next line. (DDR4-2400 is 2400MT/s, 8 transactions to fill a line, 8/2400000000=3.3n) > I see, but I think it does better these days, it'll remember a stride and prefetch by the next stride. Even striding backwards through mem (cache)! But it won't cross a page boundary. Sure, the even/odd access extender is just one very simple prefetcher that is a part of modern intel processors that I included for illustration. And we're completely ignoring software prefetch. Go ahead, do the experiment. Run a memory-heavy workload and look at cache miss rates. Then turn off prefetch and see what you get. Most workloads, you'll get a lot more misses. ;) https://github.com/deater/uarch-configure/blob/master/intel-prefetch/intel-prefetch-disable.c https://github.com/deater/uarch-configure/blob/master/intel-...
  - baybal2 7y ago
    > How does a prefetcher reduce unneeded cache evictions? It fetches data that may not be needed, increasing cache evictions. I wanted to say that a smarter prefetcher, or better to say a whole entirety of on-chip logic that works to minimise cache misses, will lower the rate of unneeded evictions.

baybal2 7y ago

But they do reduce cache misses and unneeded cache evictions dramatically, and that allows CPUs to use external memory less. This reduces the power consumption.

mlyle 7y ago
A prefetcher reduces cache misses by retrieving the data first; the number of memory accesses to get the data remains the same. [Though it may be through a more efficient pipelined operation]. How does a prefetcher reduce unneeded cache evictions? It fetches data that may not be needed, increasing cache evictions. The only thing a prefetcher can do to improve power efficiency, IMO, is by increasing utilization-- if the processor stays 99% busy instead of 97% busy, the fixed portions of the power budget are amortized over more operations.
- tempguy9999 7y ago
  > A prefetcher reduces cache misses by retrieving the data first I'm struggling with that. To avoid a cache miss the prefetcher woul have to prefetch the data from memory so early, and memory is so slow, that an entire train of in-flight instructions could complete, and then likely have the CPU pause, before the fetch-from-ram completed. I may be misunderstanding you though.
  - mlyle 7y ago
    The whole point of a prefetcher is to avoid cache misses. What do you think a prefetcher is for? Memory is very latent, and moderately slow. Hardware prefetching tends to mostly make memory accesses longer, so you don't pay the RAS/CAS penalties and instead stream just more memory words back to fill an additional cache line beyond the one demanded. e.g. One key part of Intel L2 hardware prefetching is working upon cache line pairs. If you miss on an even numbered cache line, and counters support it, it decides to also retrieve the next odd-numbered cache line at the same time. The downside of a hardware prefetcher is three things, both stemming from it perhaps retrieving memory that isn't needed. 1) it can cause useful information to be evicted from cache if a prefetch is errant. 2) it can tie up the memory bus with an unnecessary access and make a necessary one more latent. 3) it can consume power for a prefetch that isn't needed.
    - tempguy9999 7y ago
      > What do you think a prefetcher is for? We may be talking past each other. My understanding of a prefetcher is to get the data ASAP. It typically prefetches from cache, not from memory (if it prefetches from mem it's so slow it's useless). > Memory is very latent, and moderately slow Those 2 terms seem synonymous, but memory access is slooow - from doc in front of me, for haswell: L1 access hit, 4 or 5 cycles latency; L2 hit 12 cycles latency; L3 hit 36 to 66 cycles latency; ram latency 36 cycles + 57 NS / 62 cycles + 100NS depending (found ref, it's https://www.7-cpu.com/cpu/Haswell.html https://www.7-cpu.com/cpu/Haswell.html) for the ram latencies that's for different setups (single/dual cpu , I'm not sure) running ~3.5GHz. So for 36C + 57NS that 36 + (3.5 * 57) = 235 cycles latency. And it can get worse. Much worse. I'm sorry i can't find it now but in hennesy & patterson I recall it being said that prefetchers can hide most of the latency of a L1 or L2 miss if it hits L3, but if it needs main mem then it's stuffed. Sorry, that is from memory and may be wrong! But I'm pretty sure a prefetcher is wasted if it needs to hit ram. > If you miss on an even numbered cache line, and counters support it, it decides to also retrieve the next odd-numbered cache line at the same time. I see, but I think it does better these days, it'll remember a stride and prefetch by the next stride. Even striding backwards through mem (cache)! But it won't cross a page boundary. Really need an expert to chime in here, anyone?
      - mlyle 7y ago
        > My understanding of a prefetcher is to get the data ASAP. It typically prefetches from cache, not from memory (if it prefetches from mem it's so slow it's useless). The prefetcher runs at whatever level it's at (L1/L2) and fetches from whatever level has the memory. So the L2 prefetcher may be grabbing from L3, and may be grabbing from SDRAM. > ram latency 36 cycles + 57 NS / 62 cycles + 100NS depending That's the ram LATENCY ON RANDOM ACCESS. If you extend an access to fetch the next sequential line because you think it will be used, you don't pay any of the latency penalty-- you might need to strobe a column access but words keep streaming out. For this reason sequential prefetch is particularly powerful. Even if we're just retrieving from a single SDRAM channel, it's just another 3ns to continue on to retrieve the next line. (DDR4-2400 is 2400MT/s, 8 transactions to fill a line, 8/2400000000=3.3n) > I see, but I think it does better these days, it'll remember a stride and prefetch by the next stride. Even striding backwards through mem (cache)! But it won't cross a page boundary. Sure, the even/odd access extender is just one very simple prefetcher that is a part of modern intel processors that I included for illustration. And we're completely ignoring software prefetch. Go ahead, do the experiment. Run a memory-heavy workload and look at cache miss rates. Then turn off prefetch and see what you get. Most workloads, you'll get a lot more misses. ;) https://github.com/deater/uarch-configure/blob/master/intel-prefetch/intel-prefetch-disable.c https://github.com/deater/uarch-configure/blob/master/intel-...
- baybal2 7y ago
  > How does a prefetcher reduce unneeded cache evictions? It fetches data that may not be needed, increasing cache evictions. I wanted to say that a smarter prefetcher, or better to say a whole entirety of on-chip logic that works to minimise cache misses, will lower the rate of unneeded evictions.