6 ms·

> The advent of nonvolatile memory is changing the equation, though, and swapping is starting to look interesting again It's not clear to me why the nonvolati

by mmt 8y ago

> The advent of nonvolatile memory is changing the equation, though, and swapping is starting to look interesting again It's not clear to me why the nonvolatile part makes any difference to swap. I'd expect it would make more sense to attach a volatile, RAM-based SSD [1], maybe via PCIe or even once removed via storage fabric. Perhaps it's just incidental, and it's merely price and/or capacity that's important, considering that it's attached via the memory bus. [1] Not that I've ever seen such a product with a high enough capacity for a low enough price. It seems like it could be a way to recycle old and/or slow RAM, maybe a startup idea, but I'm no hardware guy.

shawn 8y ago
In the game and simulation industry, texture resolution is a problem. If you run the math on how big textures need to be to match pixel resolution 1:1 in 3D space, it quickly becomes infeasable for large scenes. One solution is to stream the mip levels, and to create virtualized mipmaps. This basically lets you create a 128k by 128k texture, which works fine because only small blocks are loaded. It’s literally the same thing as virtual memory, but for texture memory. One downside is that when you shut down the game / simulation, all of that data no longer exists. It was virtual. It either streamed from the sever or you packed it to disk somewhere. Either way you have to set it up again the next time the program starts. I don’t know whether the present topic would let you bypass this limitation, but if there was some way to start a program with a huge amount of memory allocations preloaded, that would be very attractive. And if you can reboot without losing that, then suddenly you can start crafting persistent worlds that load instantly with ~infinite resolution. This is the dream that voxel tech was supposed to make into reality, and it will only become more of a concern over time as mixed reality tech matures.
- sterlind 8y ago
  the feature you're looking for is called opening a memory mapped file. I've seen tremendous performance improvements, but it's a really underused feature for some reason. Memory map Performance is incredible.
  - wtallis 8y ago
    Does Windows have any analog to madvise()? It seems like that's probably necessary to making things work well when you're mapping huge data sets, unless you're sure they reside on really fast storage.
    - PeCaN 8y ago
      Windows 8 and later has PrefetchVirtualMemory[1]. Before that your best bet is to get a little hacky and try reading the file asynchronously to get it in the cache (Windows caches files pretty aggressively, so this more or less works). [1] https://msdn.microsoft.com/en-us/library/windows/desktop/hh780543(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/hh7...
      - senozhatsky 8y ago
        Seems that VirtualAlloc() [1] is a little bit closer to madvise(), but still is pretty far. madvise() is quite powerful. For instance, take a look at MADV_FREE. [1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa366887(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/aa3... -ss
        PeCaN 8y ago
        I figured the use-case here was MADV_WILLNEED, which is what PrefetchVirtualMemory is for. VirtualAlloc flags would be for things like reserving virtual pages without committing them. There's no single API on Windows that does the myriad of things that madvise does.
  - ddorian43 8y ago
    Yep, see lmdb read performance. Or many search engines doing mmap.
  - bluejekyll 8y ago
    > it's a really underused feature for some reason My suspicion is that it’s mainly because memory mapping files, while safe for reading, have a whole bunch of edge cases where the file can accidentally be corrupted.
- mmt 8y ago
  > I don’t know whether the present topic would let you bypass this limitation I can't imagine that it would, since it's just swapping. That's why I asked the question. How to take advantage of the nonvolatile aspect of that memory as you describe is a much broader, more complicated topic, involving bootloaders and kernels.
benwills 8y ago
My interpretation wasn't that it was specifically that it's non-volatile. Rather, that it's the performance:price of NVME that's "changing the equation." The combination of NVME's IOPS and throughput, particularly for sequential reads/writes like you'd have with something like huge pages, begins making it a reasonable consideration for caching/secondary memory in certain situations. edit: s/NVME/non-volatile/
- mmt 8y ago
  That's credible given the announced capacities and expecte performance, but I'll be interested to see how the real prices/availability compare with RAM, especially if Intel is the sole supplier.
  - wtallis 8y ago
    Non-volatile memories like NAND flash and Intel's 3D XPoint have fundamental advantages over DRAM that guarantee they'll be cheaper than DRAM. DRAM requires a transistor for every memory cell, and stores one bit per cell. Non-volatile memories like NAND flash and 3D XPoint don't need an extra transistor to isolate each memory cell, which frees them up to be built with multiple layers per die, and the non-volatility is key to their ability to store more than one bit per memory cell (though the currently-shipping 3D XPoint is only storing one bit per cell). So the non-volatility is indirectly related to why those memories are more cost-effective than DRAM, but they still have the array structure that allows for much lower access times than a hard drive.
    - mmt 8y ago
      > like NAND flash This has been in SSDs for long enough without being hailed as a game changer, which is why I question anything common between the two that appears to be called out as the cause. > Intel's 3D XPoint have fundamental advantages over DRAM that guarantee they'll be cheaper This may be true for the producer, but in a monopoly situation, one could reasonably expect any savings will be collected as profit rather than passed on to the consumer.
shawnz 8y ago
> It's not clear to me why the nonvolatile part makes any difference to swap. I'd expect it would make more sense to attach a volatile, RAM-based SSD Why not just attach that RAM as system memory in such a situation? The point of using nonvolatile is that it's cheaper and consumes less energy
- mmt 8y ago
  > Why not just attach that RAM as system memory in such a situation? Because the number of available DIMM slots, and therefore total capacity, is scarce. Increases in capacity require increases in density, which carry a cost (and presumably power density and/or speed) premium. System memory scarcity is why swap is a thing in the first place. Having that extra RAM on a slower, farther bus, as swap, means it doesn't have quite as stringent performance requirements (or abilities) with the benefit of increased overall capacity. This tradeoff sounds familiar. > The point of using nonvolatile is that it's cheaper That has yet to be seen. It stands to reason that it'll be cheaper than the RAM that would otherwise go in the same DIMM slot, but it won't perform as well, either. We can be confident it will perform much better than even PCIe/NVMe SSDs. Is it cheaper than a generation or two ago's RAM in a big array (which, again, I know doesn't actually exist)? The main question is, given Intel's monopoly, what the real world price per performance is going to be like. > consumes less energy I hadn't really thought about this as a significant consideration, although at very large memory capacities a few watts here and there do add up, and it's certainly important when comparing anything new against anything old. Still, it seems like it would just be another cost factor.
bhouston 8y ago
Old ram tends to be at small sizes. Thus it makes it not that economical to act as backup storage for the modern era.
- mmt 8y ago
  When I say "old RAM" I mean from the same, current, modern era, though, yes, it is often smaller or lower-density due to being pulls from upgrades, for example. The modern era often also includes previous-generation technology, such as DDR3 in use today even though DDR4 is available (with non-intuitive pricing relationships at any given time). That said, why would size (density) have to do with being economical? Are the required memory controllers to expensive, because there simply aren't any other than inside CPUs? The train of logic needs explaining.
  - bhouston 8y ago
    It is hard to recycle used equipment at scale generally and economically. Old ram if it is small will take up a ton of space as well. It is the same reason why I don't reuse old hds, it does make economic sense. I would rather overpay for a 12 tb drive than get 12 1tb drives for free. The trouble, increased failure rates, the space usage etc is just not worth it. A few ssds in a high speed non redundant raid would be much better. Or a new class of volitile slow ram that has tons of space would be interesting. So more space at lower cost than ram but slower. That would be interesting. Give me a few 128GB sticks please.
    - mmt 8y ago
      > It is hard to recycle used equipment at scale generally and economically. Old ram if it is small will take up a ton of space as well. You've said this already, but both of these statements need unpacking and quantifying, especially "a ton of space". There's been a persistent myth that, in datacenters, it's space that's the expensive and/or scarce resource. It isn't. It's power and cooling. Only in the last 5-6 years have we seen the construction of datacenter facilities that could handle half-U densities, let alone what some "blade" systems were trying to do. > It is the same reason why I don't reuse old hds, it does make economic sense. I would rather overpay for a 12 tb drive than get 12 1tb drives for free. The trouble, increased failure rates, This is a false equivalence, since RAM doesn't have moving parts. A better comparison would be if you discovered a box of brand new drives, still sealed in their bags with dessicant. Even ignoring the fact that 12 drives will have much better performance than a single one, would you still just blindly throw them away to buy the single one, without considering if it would be cheaper to buy 12 disk slots instead? If the answer is "yes", this is inconsistent with "economic". > the space usage etc is just not worth it. I've already addressed the space usage as a red herring. What's the "etc"? > A few ssds in a high speed non redundant raid would be much better. Why a few? Isn't a single high-end one enough to saturate an NVMe or PCIe slot these days? > Or a new class of volitile slow ram that has tons of space would be interesting. So more space at lower cost than ram but slower. That would be interesting. Give me a few 128GB sticks please. What if this "new" class is actually DDR3 and the "sticks" are actually PCIe cards? That's what I've been suggesting.

by mmt 8y ago

shawn 8y ago

In the game and simulation industry, texture resolution is a problem. If you run the math on how big textures need to be to match pixel resolution 1:1 in 3D space, it quickly becomes infeasable for large scenes. One solution is to stream the mip levels, and to create virtualized mipmaps. This basically lets you create a 128k by 128k texture, which works fine because only small blocks are loaded. It’s literally the same thing as virtual memory, but for texture memory. One downside is that when you shut down the game / simulation, all of that data no longer exists. It was virtual. It either streamed from the sever or you packed it to disk somewhere. Either way you have to set it up again the next time the program starts. I don’t know whether the present topic would let you bypass this limitation, but if there was some way to start a program with a huge amount of memory allocations preloaded, that would be very attractive. And if you can reboot without losing that, then suddenly you can start crafting persistent worlds that load instantly with ~infinite resolution. This is the dream that voxel tech was supposed to make into reality, and it will only become more of a concern over time as mixed reality tech matures.

sterlind 8y ago
the feature you're looking for is called opening a memory mapped file. I've seen tremendous performance improvements, but it's a really underused feature for some reason. Memory map Performance is incredible.
- wtallis 8y ago
  Does Windows have any analog to madvise()? It seems like that's probably necessary to making things work well when you're mapping huge data sets, unless you're sure they reside on really fast storage.
  - PeCaN 8y ago
    Windows 8 and later has PrefetchVirtualMemory[1]. Before that your best bet is to get a little hacky and try reading the file asynchronously to get it in the cache (Windows caches files pretty aggressively, so this more or less works). [1] https://msdn.microsoft.com/en-us/library/windows/desktop/hh780543(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/hh7...
    - senozhatsky 8y ago
      Seems that VirtualAlloc() [1] is a little bit closer to madvise(), but still is pretty far. madvise() is quite powerful. For instance, take a look at MADV_FREE. [1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa366887(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/aa3... -ss
      - PeCaN 8y ago
        I figured the use-case here was MADV_WILLNEED, which is what PrefetchVirtualMemory is for. VirtualAlloc flags would be for things like reserving virtual pages without committing them. There's no single API on Windows that does the myriad of things that madvise does.
- ddorian43 8y ago
  Yep, see lmdb read performance. Or many search engines doing mmap.
- bluejekyll 8y ago
  > it's a really underused feature for some reason My suspicion is that it’s mainly because memory mapping files, while safe for reading, have a whole bunch of edge cases where the file can accidentally be corrupted.
mmt 8y ago
> I don’t know whether the present topic would let you bypass this limitation I can't imagine that it would, since it's just swapping. That's why I asked the question. How to take advantage of the nonvolatile aspect of that memory as you describe is a much broader, more complicated topic, involving bootloaders and kernels.

benwills 8y ago

My interpretation wasn't that it was specifically that it's non-volatile. Rather, that it's the performance:price of NVME that's "changing the equation." The combination of NVME's IOPS and throughput, particularly for sequential reads/writes like you'd have with something like huge pages, begins making it a reasonable consideration for caching/secondary memory in certain situations. edit: s/NVME/non-volatile/

mmt 8y ago
That's credible given the announced capacities and expecte performance, but I'll be interested to see how the real prices/availability compare with RAM, especially if Intel is the sole supplier.
- wtallis 8y ago
  Non-volatile memories like NAND flash and Intel's 3D XPoint have fundamental advantages over DRAM that guarantee they'll be cheaper than DRAM. DRAM requires a transistor for every memory cell, and stores one bit per cell. Non-volatile memories like NAND flash and 3D XPoint don't need an extra transistor to isolate each memory cell, which frees them up to be built with multiple layers per die, and the non-volatility is key to their ability to store more than one bit per memory cell (though the currently-shipping 3D XPoint is only storing one bit per cell). So the non-volatility is indirectly related to why those memories are more cost-effective than DRAM, but they still have the array structure that allows for much lower access times than a hard drive.
  - mmt 8y ago
    > like NAND flash This has been in SSDs for long enough without being hailed as a game changer, which is why I question anything common between the two that appears to be called out as the cause. > Intel's 3D XPoint have fundamental advantages over DRAM that guarantee they'll be cheaper This may be true for the producer, but in a monopoly situation, one could reasonably expect any savings will be collected as profit rather than passed on to the consumer.

shawnz 8y ago

> It's not clear to me why the nonvolatile part makes any difference to swap. I'd expect it would make more sense to attach a volatile, RAM-based SSD Why not just attach that RAM as system memory in such a situation? The point of using nonvolatile is that it's cheaper and consumes less energy

mmt 8y ago
> Why not just attach that RAM as system memory in such a situation? Because the number of available DIMM slots, and therefore total capacity, is scarce. Increases in capacity require increases in density, which carry a cost (and presumably power density and/or speed) premium. System memory scarcity is why swap is a thing in the first place. Having that extra RAM on a slower, farther bus, as swap, means it doesn't have quite as stringent performance requirements (or abilities) with the benefit of increased overall capacity. This tradeoff sounds familiar. > The point of using nonvolatile is that it's cheaper That has yet to be seen. It stands to reason that it'll be cheaper than the RAM that would otherwise go in the same DIMM slot, but it won't perform as well, either. We can be confident it will perform much better than even PCIe/NVMe SSDs. Is it cheaper than a generation or two ago's RAM in a big array (which, again, I know doesn't actually exist)? The main question is, given Intel's monopoly, what the real world price per performance is going to be like. > consumes less energy I hadn't really thought about this as a significant consideration, although at very large memory capacities a few watts here and there do add up, and it's certainly important when comparing anything new against anything old. Still, it seems like it would just be another cost factor.

bhouston 8y ago

Old ram tends to be at small sizes. Thus it makes it not that economical to act as backup storage for the modern era.

mmt 8y ago
When I say "old RAM" I mean from the same, current, modern era, though, yes, it is often smaller or lower-density due to being pulls from upgrades, for example. The modern era often also includes previous-generation technology, such as DDR3 in use today even though DDR4 is available (with non-intuitive pricing relationships at any given time). That said, why would size (density) have to do with being economical? Are the required memory controllers to expensive, because there simply aren't any other than inside CPUs? The train of logic needs explaining.
- bhouston 8y ago
  It is hard to recycle used equipment at scale generally and economically. Old ram if it is small will take up a ton of space as well. It is the same reason why I don't reuse old hds, it does make economic sense. I would rather overpay for a 12 tb drive than get 12 1tb drives for free. The trouble, increased failure rates, the space usage etc is just not worth it. A few ssds in a high speed non redundant raid would be much better. Or a new class of volitile slow ram that has tons of space would be interesting. So more space at lower cost than ram but slower. That would be interesting. Give me a few 128GB sticks please.
  - mmt 8y ago
    > It is hard to recycle used equipment at scale generally and economically. Old ram if it is small will take up a ton of space as well. You've said this already, but both of these statements need unpacking and quantifying, especially "a ton of space". There's been a persistent myth that, in datacenters, it's space that's the expensive and/or scarce resource. It isn't. It's power and cooling. Only in the last 5-6 years have we seen the construction of datacenter facilities that could handle half-U densities, let alone what some "blade" systems were trying to do. > It is the same reason why I don't reuse old hds, it does make economic sense. I would rather overpay for a 12 tb drive than get 12 1tb drives for free. The trouble, increased failure rates, This is a false equivalence, since RAM doesn't have moving parts. A better comparison would be if you discovered a box of brand new drives, still sealed in their bags with dessicant. Even ignoring the fact that 12 drives will have much better performance than a single one, would you still just blindly throw them away to buy the single one, without considering if it would be cheaper to buy 12 disk slots instead? If the answer is "yes", this is inconsistent with "economic". > the space usage etc is just not worth it. I've already addressed the space usage as a red herring. What's the "etc"? > A few ssds in a high speed non redundant raid would be much better. Why a few? Isn't a single high-end one enough to saturate an NVMe or PCIe slot these days? > Or a new class of volitile slow ram that has tons of space would be interesting. So more space at lower cost than ram but slower. That would be interesting. Give me a few 128GB sticks please. What if this "new" class is actually DDR3 and the "sticks" are actually PCIe cards? That's what I've been suggesting.