5 ms·

Memristors promise to flatten part of the memory hierarchy, merging non-volatile storage and RAM. But the interesting question to me as a systems programmer is

Memristors promise to flatten part of the memory hierarchy, merging non-volatile storage and RAM. But the interesting question to me as a systems programmer is whether they will ever merge cache with everything else too. This question is really important because if there were no cache there would no longer be any benefit to compactness and locality in memory -- data structures distributed across large areas of the address space would be just as efficient as ones that are well-localized. To be honest this idea worries me a little. The performance advantage of small and local data structures is one of the main natural forces that encourages software to be small and simple. Software tends to grow over time, both in size and complexity, especially when more people are involved. Unchecked I fear that there would be no natural counterbalance to this tendency. Put another way, I'm actually a bit glad that Eclipse is horribly slow, because it's an easy-to-observe symptom of the fact that it's a horribly complex stack of software. If software like Eclipse could get by with acceptable performance because it had fast access to vast swaths of memory, it would be that much harder for an upstart competitor in this space (like Light Table) to be disruptive, because Eclipise would perform well enough that competing on speed doesn't impress. I'm probably oversimplifying a bit, but I do think it generally good for programmers when simpler software also has better performance naturally. But even if Memristors could be as fast as SRAM, could you out enough of it close enough to the CPU that it truly flattens the memory hierarchy completely? This is where I hit the limits of my knowledge of computer architecture.

glhaynes 14y ago
I don't quite understand the argument - if there's no problem with increased size, why should we worry about size? People from the 1960s would be shocked to hear how large our software is today already, but since we have the capacity to run it successfully, I'm glad that we've freed up programmers to work with more productivity. As far as complexity: there will always be costs to complexity in terms of agility and reliability. And using less compact data structures often leads to less complex code, anyway.
- haberman 14y ago
  > As far as complexity: there will always be costs to complexity in terms of agility and reliability. Yes, but I worry about a world where only the programmers bear the cost of the complexity. If reduced complexity does not benefit the user, it becomes harder to justify time spent on reducing complexity. > And using less compact data structures often leads to less complex code, anyway. I disagree, unless you're talking about extreme cases like fancy structure packing. More compact data structures mean less program state. Less program state means less complicated state changes and less redundancy. Litmus test: if your program has entered a bad state, how much data do you have to inspect to discover what the inconsistency is? And how much code do you have to inspect to figure out how it happened?
  - geon 14y ago
    > a world where only the programmers bear the cost of the complexity. It is the users who pay the ultimate price of complexity. Unreliable, expensive software with long release cycles cost them money, time and happiness. A company that does not realize this is doomed. It doesn't matter if the unreliable software runs fast. As a counter argument, when you are not restricted by performance, you can utilize your memory more effectively to remove complexity from your software.
- sliverstorm 14y ago
  using less compact data structures often leads to less complex code Well, there's also plenty of times where a more compact data structure can lead to less complex code.
seles 14y ago
The memory hierarchy can never completely flatten. Information takes physical space to store and the speed of light is finite. So the more information, the further things are apart, and the longer access times will take. That being said this theoretical limit might not be the main bottleneck right now. Parallelism can also bypass this limit since not all information needs to be processed at the same location.
- adaszko 14y ago
  > The memory hierarchy can never completely flatten. Information takes physical space to store and the speed of light is finite. So the more information, the further things are apart, and the longer access times will take. Not necessarily. Memristors were shown to be capable of performing logical operations [1]. Given that, one can imagine computations migrating through the circuit to stay physically as close as possible to the data thereby eliminating the need for caches. [1] http://www.nature.com/nature/journal/v464/n7290/full/nature08940.html http://www.nature.com/nature/journal/v464/n7290/full/nature0...
  - Symmetry 14y ago
    Since we can use the same sort of MOSFETs found in SRAM to perform logical operations you'd could equally well imagine computations migrating out into memory even without memristors. The problem is more one of computational model than physical process. People are certainly looking at things part-way there like grid-computing or whatever, but learning how to use these things efficiently is a hard computer science problem.
    - ChuckMcM 14y ago
      Micron built DIMM modules with computation on them as an experiment in this. They of course simply put a CPU inside the DIMM controller (imagine each of your DIMMs was a GUMSTIX rather than a regular DIMM) but the concept they were shooting for was the same. I did some preliminary VHDL work on a 'memory' controller which was basically a DIMM/CPU bus for a psuedo cluster of these things. The bottom line was that it was buildable but there were a lot of challenges with the way compilers and what not generated code (or not) for these sorts of systems. I think a couple of projects at Umich and UCB got funded but I don't know if they produced any results.
Symmetry 14y ago
Basically, we're never getting rid of the need for caches. You can't put enough of it close enough to the CPU because the muxes needed to select the data you need have too many FO4s[1] of fanning, and because you're limited by the speed of light, and the arrays you need are too big to get all of them close enough to the processor. Another problem is write endurance. RRAM is expected to have be able to take many orders of magnitude more writes before failing than Flash, but its still limited. The fact that you have layers of SRAM cache between it and the processor buffering against repeated writes to the same location is why I'd still be comfortable using it as main memory. Otherwise you could get an infinite loop that could actually damage the memory. [1]http://en.wikipedia.org/wiki/FO4 http://en.wikipedia.org/wiki/FO4
- bradleyland 14y ago
  "You can't put enough of it close enough to the CPU because the muxes needed to select the data you need have too many FO4s[1] of fanning" So it's like in Minecraft when you can't make a redstone design any more compact because of limitations in the rules of redstone adjacency.
  - Symmetry 14y ago
    Wow, sorry, that sentence I constructed there was really confusing. There are really two separate effects. On the one hand, you can only make a cell of this memory so small, so you can't pack it as densely as you'd like. The other is that you have one cell of memory you'd like to access, and an output line you'd like the data from it to go out on. And you don't want the data from any of the other N billion cells of memory to go out on that line. There are actually some structures you use to retrieve that particular bit, but the more bits you're selecting between the more delay is introduced. In most current CPU designs this is a actually a bigger deal than speed of light, at least for the various levels of on-chip cache.
- juiceandjuice 14y ago
  You're neglecting the possibility that we abandon the Von Neumann architecture altogether.
IsTom 14y ago
Existence of cache comes from trade-off between speed and cost, so it will quite likely never disappear. Notice how there are many layers of cache in current systems: HDD <-> HDD Cache <-> RAM <-> L3 ... <-> Processor registers. If you get your mass storage as fast as registers then perhaps .

glhaynes 14y ago

I don't quite understand the argument - if there's no problem with increased size, why should we worry about size? People from the 1960s would be shocked to hear how large our software is today already, but since we have the capacity to run it successfully, I'm glad that we've freed up programmers to work with more productivity. As far as complexity: there will always be costs to complexity in terms of agility and reliability. And using less compact data structures often leads to less complex code, anyway.

haberman 14y ago
> As far as complexity: there will always be costs to complexity in terms of agility and reliability. Yes, but I worry about a world where only the programmers bear the cost of the complexity. If reduced complexity does not benefit the user, it becomes harder to justify time spent on reducing complexity. > And using less compact data structures often leads to less complex code, anyway. I disagree, unless you're talking about extreme cases like fancy structure packing. More compact data structures mean less program state. Less program state means less complicated state changes and less redundancy. Litmus test: if your program has entered a bad state, how much data do you have to inspect to discover what the inconsistency is? And how much code do you have to inspect to figure out how it happened?
- geon 14y ago
  > a world where only the programmers bear the cost of the complexity. It is the users who pay the ultimate price of complexity. Unreliable, expensive software with long release cycles cost them money, time and happiness. A company that does not realize this is doomed. It doesn't matter if the unreliable software runs fast. As a counter argument, when you are not restricted by performance, you can utilize your memory more effectively to remove complexity from your software.
sliverstorm 14y ago
using less compact data structures often leads to less complex code Well, there's also plenty of times where a more compact data structure can lead to less complex code.

seles 14y ago

The memory hierarchy can never completely flatten. Information takes physical space to store and the speed of light is finite. So the more information, the further things are apart, and the longer access times will take. That being said this theoretical limit might not be the main bottleneck right now. Parallelism can also bypass this limit since not all information needs to be processed at the same location.

adaszko 14y ago
> The memory hierarchy can never completely flatten. Information takes physical space to store and the speed of light is finite. So the more information, the further things are apart, and the longer access times will take. Not necessarily. Memristors were shown to be capable of performing logical operations [1]. Given that, one can imagine computations migrating through the circuit to stay physically as close as possible to the data thereby eliminating the need for caches. [1] http://www.nature.com/nature/journal/v464/n7290/full/nature08940.html http://www.nature.com/nature/journal/v464/n7290/full/nature0...
- Symmetry 14y ago
  Since we can use the same sort of MOSFETs found in SRAM to perform logical operations you'd could equally well imagine computations migrating out into memory even without memristors. The problem is more one of computational model than physical process. People are certainly looking at things part-way there like grid-computing or whatever, but learning how to use these things efficiently is a hard computer science problem.
  - ChuckMcM 14y ago
    Micron built DIMM modules with computation on them as an experiment in this. They of course simply put a CPU inside the DIMM controller (imagine each of your DIMMs was a GUMSTIX rather than a regular DIMM) but the concept they were shooting for was the same. I did some preliminary VHDL work on a 'memory' controller which was basically a DIMM/CPU bus for a psuedo cluster of these things. The bottom line was that it was buildable but there were a lot of challenges with the way compilers and what not generated code (or not) for these sorts of systems. I think a couple of projects at Umich and UCB got funded but I don't know if they produced any results.

Symmetry 14y ago

Basically, we're never getting rid of the need for caches. You can't put enough of it close enough to the CPU because the muxes needed to select the data you need have too many FO4s[1] of fanning, and because you're limited by the speed of light, and the arrays you need are too big to get all of them close enough to the processor. Another problem is write endurance. RRAM is expected to have be able to take many orders of magnitude more writes before failing than Flash, but its still limited. The fact that you have layers of SRAM cache between it and the processor buffering against repeated writes to the same location is why I'd still be comfortable using it as main memory. Otherwise you could get an infinite loop that could actually damage the memory. [1]http://en.wikipedia.org/wiki/FO4 http://en.wikipedia.org/wiki/FO4

bradleyland 14y ago
"You can't put enough of it close enough to the CPU because the muxes needed to select the data you need have too many FO4s[1] of fanning" So it's like in Minecraft when you can't make a redstone design any more compact because of limitations in the rules of redstone adjacency.
- Symmetry 14y ago
  Wow, sorry, that sentence I constructed there was really confusing. There are really two separate effects. On the one hand, you can only make a cell of this memory so small, so you can't pack it as densely as you'd like. The other is that you have one cell of memory you'd like to access, and an output line you'd like the data from it to go out on. And you don't want the data from any of the other N billion cells of memory to go out on that line. There are actually some structures you use to retrieve that particular bit, but the more bits you're selecting between the more delay is introduced. In most current CPU designs this is a actually a bigger deal than speed of light, at least for the various levels of on-chip cache.
juiceandjuice 14y ago
You're neglecting the possibility that we abandon the Von Neumann architecture altogether.

IsTom 14y ago

Existence of cache comes from trade-off between speed and cost, so it will quite likely never disappear. Notice how there are many layers of cache in current systems: HDD <-> HDD Cache <-> RAM <-> L3 ... <-> Processor registers. If you get your mass storage as fast as registers then perhaps .