4 ms·

Maybe we should attack the memory wall problem first, since it promises a big acceleration in some areas, and it's not strictly related to limits of physics, li

by petra 10y ago

Maybe we should attack the memory wall problem first, since it promises a big acceleration in some areas, and it's not strictly related to limits of physics, like lithography ? One way to do so is silicon photonics: it's possible to achieve 280 tbps bandwidth for 168W @ chip area of 4mm2 for 168W. Or you could triple that but you'll need ~1.5KW . This of course will require better cooling, but DARPA has a project, using microfluidics channels, that can cool 1KW/cm2 and 30KW/cm2. And as for power delivery - with TSV's you can deliver 150W/cm2. And sure, such chips may be more expensive both in cost and power than today's are. But infinite memory bandwidth and photonics can make programmer's lives so much easier, and offer great acceleration in memory constrained problems, and make new architectures feasible(network computers with infinite bandwidth?). So maybe there's an initial market ? And once we're there, and money starts flowing, maybe we'll see another type of "moore's law" focused on reducing the costs of cooling/photonics/power-delivery? And BTW, some good news: ST(i think) is opening a photonics fab. links: https://www.extremetech.com/extreme/224516-microfluidics-darpa-is-betting-embedded-water-droplets-could-cool-next-gen-chips https://www.extremetech.com/extreme/224516-microfluidics-dar... https://pdfs.semanticscholar.org/4f9b/d24ea869f64c4457f69ad6d9dc2b3ddb93ac.pdf https://pdfs.semanticscholar.org/4f9b/d24ea869f64c4457f69ad6... http://drum.lib.umd.edu/bitstream/handle/1903/17153/TR%20DRUM%20revised%2023%20June%202016.pdf?sequence=5&isAllowed=y http://drum.lib.umd.edu/bitstream/handle/1903/17153/TR%20DRU...

adrianN 10y ago
Latency is more important than just bandwidth, and since RAM is pretty far away from the ALU you'll quickly hit limits there.
- petra 10y ago
  You could get 10x better perf and easier programmability , for starters.
- barrkel 10y ago
  What if instead of passing a fetch across the bus, you passed a tiny little program that included conditional indirections - if you want to get x.y.z, you'd have a list of two indirections and an offset in the little program. Push the latency problem of pointerful code closer to the data.
  - jackyinger 10y ago
    That would be nifty. Though a radical departure from the status quo. Not to say it couldn't be done. Currently DRAM is designed to fetch large pages at a time which are cached in the CPU. Cache is highly beneficial to latency when there is good memory locality in the program being run.
  - hinkley 10y ago
    Here's an even simpler one: double lookup. Fetch a block of memory at the address stored in this memory address (which I may not have fetched already). Our data hierarchies create graphs that have lots of internal pointers in them, and we are constantly warned about the cost of pointer indirections. Memory already knows how to deal with addresses. Teaching it to chase pointers should be easier than teaching it Boolean math. [edit: it appears the real limitation to both our ideas is that virtual memory prevents you from making any decisions on the wrong side of the MMU. You could only ever make conditional fetches from the same page, which could be slightly useful but would be so hard to use I don't know who would bother]
    - dom0 10y ago
      This does not work, because the latency of accessing memory is inherent to it. It's simply the time required to open a DRAM row (or charge bit lines), which stays more or less the same independent of process scaling. Moving the pointer chasing closer to the memory does not change this larger delay.
- Dylan16807 10y ago
  The distance isn't a big issue. It's trivial to put the RAM close enough that the signals spend less than a nanosecond in transit on a round trip. It's all about the speed of the memory itself.
  - adrianN 10y ago
    1ns is already two cycles at 2GHz and that doesn't count the switching delays of all the transistors between the memory cell and the register.
    - sqeaky 10y ago
      If RAM lookups only cost 2 cycles they would beat L2 and L3 cache. The real problem is the speed of the RAM, not the distance, for now. When RAM gets a request it can take many cycles to respond, sometimes as many as 50 or 60 before the first byte is returned. If you are smart you can request large sequential chunks and after the first bytes things come down to the the CPU very fast, but that initial hit is still something a lot of people optimizing algorithms are wrestling with.
- deepnotderp 10y ago
  Actually, the latency of DRAM isn't necessarily caused by physical transport, but the speed of the memory itself.
- CalChris 10y ago
  Latency is more important than bandwidth but it's also a lot harder. You're running into c and c is a constant.
dom0 10y ago
The memory wall is not a bandwidth issue, but a problem of (concurrent - many more strands of execution than memory channels) latency and there are as far as I know pretty much no ideas how to fundamentally address that (DRAM alternatives might, but so far were all rather disappointing even compared to much older DRAM). You can add more channels and more ranks into the system, which gives some small linear increase w.r.t. to concurrency, but brings a a big (channels) or minor (ranks) power increase. This doesn't change non-concurrent latency, so does not help sequential workloads.
cowardlydragon 10y ago
Why have the wall anyway? I'm a bit naive at the transistor level, but those L2/L3 caches are pretty big in historical terms. If Intel does eventually get that memory/SSD hybrid up to snuff, and they are looking to do things with extra transistor/silicon real estate... just put in a whole lot of memory on-chip. The entire software industry could use a diet. If Moore's Law ended practically, then the "wait for the faster chip" mentality dies out and we start actually optimizing code, size and performance wise.
deepnotderp 10y ago
What? In my experience SiP tends to be more energy efficient than CMOS interconnect transport.

by petra 10y ago

adrianN 10y ago

Latency is more important than just bandwidth, and since RAM is pretty far away from the ALU you'll quickly hit limits there.

petra 10y ago
You could get 10x better perf and easier programmability , for starters.
barrkel 10y ago
What if instead of passing a fetch across the bus, you passed a tiny little program that included conditional indirections - if you want to get x.y.z, you'd have a list of two indirections and an offset in the little program. Push the latency problem of pointerful code closer to the data.
- jackyinger 10y ago
  That would be nifty. Though a radical departure from the status quo. Not to say it couldn't be done. Currently DRAM is designed to fetch large pages at a time which are cached in the CPU. Cache is highly beneficial to latency when there is good memory locality in the program being run.
- hinkley 10y ago
  Here's an even simpler one: double lookup. Fetch a block of memory at the address stored in this memory address (which I may not have fetched already). Our data hierarchies create graphs that have lots of internal pointers in them, and we are constantly warned about the cost of pointer indirections. Memory already knows how to deal with addresses. Teaching it to chase pointers should be easier than teaching it Boolean math. [edit: it appears the real limitation to both our ideas is that virtual memory prevents you from making any decisions on the wrong side of the MMU. You could only ever make conditional fetches from the same page, which could be slightly useful but would be so hard to use I don't know who would bother]
  - dom0 10y ago
    This does not work, because the latency of accessing memory is inherent to it. It's simply the time required to open a DRAM row (or charge bit lines), which stays more or less the same independent of process scaling. Moving the pointer chasing closer to the memory does not change this larger delay.
Dylan16807 10y ago
The distance isn't a big issue. It's trivial to put the RAM close enough that the signals spend less than a nanosecond in transit on a round trip. It's all about the speed of the memory itself.
- adrianN 10y ago
  1ns is already two cycles at 2GHz and that doesn't count the switching delays of all the transistors between the memory cell and the register.
  - sqeaky 10y ago
    If RAM lookups only cost 2 cycles they would beat L2 and L3 cache. The real problem is the speed of the RAM, not the distance, for now. When RAM gets a request it can take many cycles to respond, sometimes as many as 50 or 60 before the first byte is returned. If you are smart you can request large sequential chunks and after the first bytes things come down to the the CPU very fast, but that initial hit is still something a lot of people optimizing algorithms are wrestling with.
deepnotderp 10y ago
Actually, the latency of DRAM isn't necessarily caused by physical transport, but the speed of the memory itself.
CalChris 10y ago
Latency is more important than bandwidth but it's also a lot harder. You're running into c and c is a constant.

dom0 10y ago

The memory wall is not a bandwidth issue, but a problem of (concurrent - many more strands of execution than memory channels) latency and there are as far as I know pretty much no ideas how to fundamentally address that (DRAM alternatives might, but so far were all rather disappointing even compared to much older DRAM). You can add more channels and more ranks into the system, which gives some small linear increase w.r.t. to concurrency, but brings a a big (channels) or minor (ranks) power increase. This doesn't change non-concurrent latency, so does not help sequential workloads.

cowardlydragon 10y ago

Why have the wall anyway? I'm a bit naive at the transistor level, but those L2/L3 caches are pretty big in historical terms. If Intel does eventually get that memory/SSD hybrid up to snuff, and they are looking to do things with extra transistor/silicon real estate... just put in a whole lot of memory on-chip. The entire software industry could use a diet. If Moore's Law ended practically, then the "wait for the faster chip" mentality dies out and we start actually optimizing code, size and performance wise.

deepnotderp 10y ago

What? In my experience SiP tends to be more energy efficient than CMOS interconnect transport.