3 ms·

There are some memories supporting basic in-memory operations. For example: https://mosys.com/products/blazar-family/be3rmw-bandwidth-engine-3-rmw/ https://mos

There are some memories supporting basic in-memory operations. For example: https://mosys.com/products/blazar-family/be3rmw-bandwidth-engine-3-rmw/ https://mosys.com/products/blazar-family/be3rmw-bandwidth-en.... This supports operations like read-modify-write within the memory device itself. (I have no affiliation with this company.) The barrier to adoption of this is not technical, it's economic. Memory industry has focused on making the highest capacity and lowest cost/bit products. This drives high manufacturing volume which drives economies of scale. Memory products with integrated functions are inherently niche, and therefore do not have anywhere near the market size and economy of scale. Designers have decided (historically) that it is cheaper at the system level to keep the logic operations within the CPU and use a "dumb" commodity memory, even though this necessitates more bandwidth usage. (It's a complex engineering trade-off.) With logic performance continuing to scale faster than memory bandwidth, at some point an architecture that reduces the required memory bandwidth (such as computing in-memory) might start to make sense economically.

vlovich123 6y ago
I think there are niche use-cases that would warrant the cost/complexity trade-off. Namely cloud infrastructure for databases where you might be processing large amounts and transferring all that data first to the CPU is pretty ridiculous. I think the complexity problem is solvable. If you can build such a thing for memory, you can reuse the same general concept for storage like NVME/SSD/spinning disk. It’s entirely possible this is warranted even in consumer devices as quite a bit of OS operations deal with modifying/querying memory whereas doing offload can win you some serious wins (making the machine feel way more snappy/interactive + more powerful to execute things locally). I don’t have hope for traditional CPU designers so the question is whether someone can both design a memory system with this power AND a computational model that makes this easy to adopt in major languages while offering a perf win (given that a memory fetch costs ~100ns for ~256 bytes). It’s challenging and unlikely to come from x86 land where back Hw compatibility is extremely important. The innovation may eventually come from mobile land (which is where Apple is coming from) where memory controllers are custom designed and part of the SoC anyway, changing with each revision, so the hard part remains again how you make this fast, efficient, able to handle multiple concurrent programs (or at least have the OS control coarsest which program’s memory accesses were being prioritized), and language integration (so you can hand off an algorithm and it would be executed without upending existing software design knowledge). The OS integration could be even more intelligent - if the process/thread is spending its time processing memory without returning results to the CPU, just put it to sleep until the result is available (separating memory access and CPU utilization, resulting in drastically better utilization of cycles rather than naively waiting for each memory stall one at a time). So tldr I agree with you totally. This is not going to happen if memory manufacturers continue to go for the “cheap and large” memory route. I do see hope that such concepts may be explored if we get more innovation in the CPU space.
LargoLasskhyfv 6y ago
Sounds like http://iram.cs.berkeley.edu/ http://iram.cs.berkeley.edu/ and the meanwhile flopped Micron Automata Processor (Micron AP).

vlovich123 6y ago

I think there are niche use-cases that would warrant the cost/complexity trade-off. Namely cloud infrastructure for databases where you might be processing large amounts and transferring all that data first to the CPU is pretty ridiculous. I think the complexity problem is solvable. If you can build such a thing for memory, you can reuse the same general concept for storage like NVME/SSD/spinning disk. It’s entirely possible this is warranted even in consumer devices as quite a bit of OS operations deal with modifying/querying memory whereas doing offload can win you some serious wins (making the machine feel way more snappy/interactive + more powerful to execute things locally). I don’t have hope for traditional CPU designers so the question is whether someone can both design a memory system with this power AND a computational model that makes this easy to adopt in major languages while offering a perf win (given that a memory fetch costs ~100ns for ~256 bytes). It’s challenging and unlikely to come from x86 land where back Hw compatibility is extremely important. The innovation may eventually come from mobile land (which is where Apple is coming from) where memory controllers are custom designed and part of the SoC anyway, changing with each revision, so the hard part remains again how you make this fast, efficient, able to handle multiple concurrent programs (or at least have the OS control coarsest which program’s memory accesses were being prioritized), and language integration (so you can hand off an algorithm and it would be executed without upending existing software design knowledge). The OS integration could be even more intelligent - if the process/thread is spending its time processing memory without returning results to the CPU, just put it to sleep until the result is available (separating memory access and CPU utilization, resulting in drastically better utilization of cycles rather than naively waiting for each memory stall one at a time). So tldr I agree with you totally. This is not going to happen if memory manufacturers continue to go for the “cheap and large” memory route. I do see hope that such concepts may be explored if we get more innovation in the CPU space.

LargoLasskhyfv 6y ago

Sounds like http://iram.cs.berkeley.edu/ http://iram.cs.berkeley.edu/ and the meanwhile flopped Micron Automata Processor (Micron AP).