5 ms·
This is really interesting, but note the power increase seen inside the DRAM modules. Anandtech have an excellent analysis: https://www.anandtech.com/show/1475
by bem94 7y ago
This is really interesting, but note the power increase seen inside the DRAM modules.
Anandtech have an excellent analysis: https://www.anandtech.com/show/14750/hot-chips-31-analysis-inmemory-processing-by-upmem https://www.anandtech.com/show/14750/hot-chips-31-analysis-i...
> The operation itself [inside the dram module] actually requires 2x the power (20 pJ compared to 10 pJ [inside the main CPU]), but the overall gain in power efficiency is 170 pJ vs 3010 pJ, or just under 20x
> One thing that this slide states that might be confusing is the server power consumption – the regular server is listed as only 300W, but the PIM solution is up to 700W. This is because the power-per-DRAM module would increase under UPMEM’s solution.
I'm assuming that those numbers represent "peak" power, and that when idle the compute parts of the DRAM can be power/clock gated. The implication being that if you are doing lots of in-memory analytics, you'll get a power saving, but if you switch to "something else" and don't get good utilisation of the in-memory compute, you'll probably ruin your power/energy efficiency.
I guess that means the future improvements will involve bringing the power consumption of the modified DRAM modules back in line with their "normal" cousins.
- baybal2 7y agoWhile it is very true that the memory is much bigger bottleneck than the CPU in computing today, it is a quite a bit disingenuous for the "in-memory" computing crowd to keep pointing to dramatic difference in energy needed for a memory transaction in "their vs ours" solution. Yes, onchip SRAM can take single picojoules vs hundred picojules per bit on external DRAM, but using external DRAM is a perfectly valid design decision that does not dramatically increase the power consumption because CPUs have large caches and very smart prefetch logic. Some algorithms may seem to demand tens of gigabits of memory bandwidth on paper, but in reality smart prefetchers can reduce that to hundreds megabits. Where things like this matters is scalability. Parallel computing performance scales not much with memory bandwidths per CPU, but with the number of individual CPUs, threads and their prefetch units. Here, the decision to have hundreds of wimpy CPUs, but all with individual memory access will be valid. Edit: typo fix
- tl 7y ago> Some algorithms may seem to demand tenths of gigabits of memory bandwidth on paper, but in reality smart prefetchers can reduce that to hundred megabits. Are tenths of a gigabit and hundreds of megabits not the same thing?
- natpalmer1776 7y ago1/10th Gigabit = 100 Megabits, so I assume its a typo(?)
- dr_zoidberg 7y agoI understood it as: Tenths of gigabits -> 10, 20, 30 gigabits Hundred megabits -> 0.2, 0.3, 0.4 gigabits So "tenths" as in "multiples of ten" instead of "fractions of ten". So probably a typo of "tens" being mistaken for "tenths" (maybe autocorrect?).
- colejohnson66 7y agoProbably meant tens of gigabits
- Symmetry 7y agoPrefetchers don't do anything do reduce the power consumption involved in streaming data from memory, they only reduce the latency seen by the CPU. In fact, since the prefetcher sometimes makes mispredictions, they tend to increase power.
- baybal2 7y agoBut they do reduce cache misses and unneeded cache evictions dramatically, and that allows CPUs to use external memory less. This reduces the power consumption.
- mlyle 7y agoA prefetcher reduces cache misses by retrieving the data first; the number of memory accesses to get the data remains the same. [Though it may be through a more efficient pipelined operation]. How does a prefetcher reduce unneeded cache evictions? It fetches data that may not be needed, increasing cache evictions. The only thing a prefetcher can do to improve power efficiency, IMO, is by increasing utilization-- if the processor stays 99% busy instead of 97% busy, the fixed portions of the power budget are amortized over more operations.
- sp332 7y agoWatts are Joules / second. If you're doing operations that take 1/20 as much power each but you're doing them 20x faster, it takes the same amount of power in Watts. For a better comparison you'd need to measure Watts * seconds for a given task.