4 ms·
While it is very true that the memory is much bigger bottleneck than the CPU in computing today, it is a quite a bit disingenuous for the "in-memory" computing
by baybal2 7y ago
While it is very true that the memory is much bigger bottleneck than the CPU in computing today, it is a quite a bit disingenuous for the "in-memory" computing crowd to keep pointing to dramatic difference in energy needed for a memory transaction in "their vs ours" solution.
Yes, onchip SRAM can take single picojoules vs hundred picojules per bit on external DRAM, but using external DRAM is a perfectly valid design decision that does not dramatically increase the power consumption because CPUs have large caches and very smart prefetch logic. Some algorithms may seem to demand tens of gigabits of memory bandwidth on paper, but in reality smart prefetchers can reduce that to hundreds megabits.
Where things like this matters is scalability. Parallel computing performance scales not much with memory bandwidths per CPU, but with the number of individual CPUs, threads and their prefetch units. Here, the decision to have hundreds of wimpy CPUs, but all with individual memory access will be valid.
Edit: typo fix
- tl 7y ago> Some algorithms may seem to demand tenths of gigabits of memory bandwidth on paper, but in reality smart prefetchers can reduce that to hundred megabits. Are tenths of a gigabit and hundreds of megabits not the same thing?
- natpalmer1776 7y ago1/10th Gigabit = 100 Megabits, so I assume its a typo(?)
- dr_zoidberg 7y agoI understood it as: Tenths of gigabits -> 10, 20, 30 gigabits Hundred megabits -> 0.2, 0.3, 0.4 gigabits So "tenths" as in "multiples of ten" instead of "fractions of ten". So probably a typo of "tens" being mistaken for "tenths" (maybe autocorrect?).
- colejohnson66 7y agoProbably meant tens of gigabits
- Symmetry 7y agoPrefetchers don't do anything do reduce the power consumption involved in streaming data from memory, they only reduce the latency seen by the CPU. In fact, since the prefetcher sometimes makes mispredictions, they tend to increase power.
- baybal2 7y agoBut they do reduce cache misses and unneeded cache evictions dramatically, and that allows CPUs to use external memory less. This reduces the power consumption.
- mlyle 7y agoA prefetcher reduces cache misses by retrieving the data first; the number of memory accesses to get the data remains the same. [Though it may be through a more efficient pipelined operation]. How does a prefetcher reduce unneeded cache evictions? It fetches data that may not be needed, increasing cache evictions. The only thing a prefetcher can do to improve power efficiency, IMO, is by increasing utilization-- if the processor stays 99% busy instead of 97% busy, the fixed portions of the power budget are amortized over more operations.
- tempguy9999 7y ago> A prefetcher reduces cache misses by retrieving the data first I'm struggling with that. To avoid a cache miss the prefetcher woul have to prefetch the data from memory so early, and memory is so slow, that an entire train of in-flight instructions could complete, and then likely have the CPU pause, before the fetch-from-ram completed. I may be misunderstanding you though.
- mlyle 7y agoThe whole point of a prefetcher is to avoid cache misses. What do you think a prefetcher is for? Memory is very latent, and moderately slow. Hardware prefetching tends to mostly make memory accesses longer, so you don't pay the RAS/CAS penalties and instead stream just more memory words back to fill an additional cache line beyond the one demanded. e.g. One key part of Intel L2 hardware prefetching is working upon cache line pairs. If you miss on an even numbered cache line, and counters support it, it decides to also retrieve the next odd-numbered cache line at the same time. The downside of a hardware prefetcher is three things, both stemming from it perhaps retrieving memory that isn't needed. 1) it can cause useful information to be evicted from cache if a prefetch is errant. 2) it can tie up the memory bus with an unnecessary access and make a necessary one more latent. 3) it can consume power for a prefetch that isn't needed.