4 ms·

With very minimal knowledge of hardware, I wonder if we'll eventually have storage cheap enough and fast enough so we could just remove RAM from a computer all

With very minimal knowledge of hardware, I wonder if we'll eventually have storage cheap enough and fast enough so we could just remove RAM from a computer all together and we'd have HDD and RAM in the same physical component. We spend so much time optimizing communication between different technologies, removing "middle-men" components but still requiring them as part of a whole that it's surprising (to me) that I haven't seen anything like that in the news/media/etc. I assume it's from an obvious technical/price limitation, but again IDK a lot about hardware so...

ajflores1604 7y ago
That's the dream of Intel's "Optane DC Persistent Memory". As with most new technology though, it's still very expensive and currently targeted exclusively at enterprise solutions. Edit: 'exclusively' is a bit heavy handed, but for the sizes that a typical consumer would think of (250gb-2tb), it still isn't available outside of a simple cache drive. This link has a good breakdown: https://www.pcworld.com/article/3388135/everything-you-need-to-know-about-optane-right-now.html https://www.pcworld.com/article/3388135/everything-you-need-...
- rl3 7y ago
  >... but for the sizes that a typical consumer would think of (250gb-2tb), it still isn't available outside of a simple cache drive. The 900P and 905P series are Optane devices targeted towards prosumers presumably; they offer 280GB-1.5TB of storage. That said, they're still very expensive. I really wish Optane would get cheaper, or at least refresh faster. It's so utterly superior to NAND devices in terms of random access latency and endurance. All of my systems are Optane and it's amazing, but I feel the capacity squeeze since I can't afford the higher capacity drives. Intel's decision to offer small capacity cache devices was a mistake in my opinion. It was too confusing for the average consumer, and too complicated on a vendor support level. Their prosumer 900P series was initially PCI-E x4 AIC form factor, which was also a mistake in my opinion since they later followed up with proper U.2 and M.2 form factors. Likewise their enterprise offerings aren't really great in terms of availability. It's too many SKUs that represent poor vision and execution. In hindsight they should've just directly competed with Samsung NAND in M.2 form factor somehow. Sell at a loss for all I care; the tech is so superior to NAND that it deserves a much larger foothold than it currently has.
  - ajflores1604 7y ago
    I haven't had the benefit of experiencing optane first hand, but the specs look insane to me. And I completely agree, Intel bumbled the whole strategy for it. And on top of the strategy, the marketing is abysmal, not just for optane but even their processor lines. This video laid out their new 10nm processor naming scheme (https://youtu.be/hzXKDQR_1d4 https://youtu.be/hzXKDQR_1d4) and at this point it seems malicious. Like they're trying to make it so consumers accidentally get the highest margin chip on the stack by getting a low power 'i7'. Or maybe hope consumers just stop trying to compare between processor skus, or head to head benchmarks against AMD, and instead focus on the brand that is 'intel inside'.
proverbialbunny 7y ago
Latency is more important than bandwidth when it comes to reading and writing data, except for specialized work loads. I don't know the delay for current gen graphics cards, but when CUDA first came out in '07 vram had a roughly a 10x delay to system ram. vram would work great as virtual memory, where chunks of ram can be loaded and unloaded when the systems main memory runs out, but using vram as system ram today isn't ideal. Likewise running an ssd as system ram wouldn't be ideal either. This is one of the reasons why cache sizes have increased so much throughout the years. It's not just more cores. The cpu is branching out further pulling in data that will be hypothetically necessary in a few ms. Using vram you'd need the cpu to prefetch hundreds of ms ahead of time. This would add a lot of complexity to the cpu. AMD announced a cpu with a 256MB L3 cache. Imagine, in 10 years a cpu with 1GB of cache.
- kbumsik 7y ago
  > Latency is more important than bandwidth when it comes to reading and writing data, except for specialized work loads. It is also worth noting that there is pretty big delay between CPU and PCIe.
shaklee3 7y ago
Ideally we'd get rid of the discrete CPU altogether for systems like this. Mellanox attempted to do this with their bluefield box, which essentially put arm cores on the nics, and let them control the transfers to storage. There's really no reason to have a dual Xeon system when you're just rdma'ing to storage. Now, if we can only have that for GPU, but CUDA no longer supports arm outside of tegra.
- rrss 7y ago
  > CUDA no longer supports arm outside of tegra. ARM support is coming (back), apparently: https://www.nextplatform.com/2019/06/17/nvidia-makes-arm-a-peer-to-x86-and-power-for-gpu-acceleration/ https://www.nextplatform.com/2019/06/17/nvidia-makes-arm-a-p...
pdpi 7y ago
> I wonder if we'll eventually have storage cheap enough and fast enough At a fundamental level, RAM is faster than flash/hdd for two basic reasons: it's allowed to be volatile, and your working set is usually much smaller than your total storage requirements, so RAM is allowed to be more expensive on a byte-for-byte basis than persistent storage. Unless something weird happens with relative research paces, you can basically expect RAM to be faster than persistent storage for those two reasons. The question then becomes: at what point whether there is a point where even persistent storage is fast enough that your CPU/GPU/*PU can't make use of any further bandwidth/latency improvements. Looking at the relative performance of L1, L2, L3, and RAM sets some milestones you need to hit before this even becomes a long-term goal.
luma 7y ago
This can only happen if someone develops a storage technology that is simultaneously the lowest cost, lowest latency, highest bandwidth, lowest MTBF, lowest power, smallest footprint, and includes every extra feature (ECC, encryption, etc). Until that unicorn exists, engineers need to select a storage technology that meets requirements for the use case at hand. If these requirements result in a mix of storage technologies, then at some point it's likely to require moving data from one storage device to another. HPE believes (believed?) in your vision and made a lot of noise about "The Machine", a memristor-based system which incorporated a lot of ideas around the sorts of hardware and software architecture changes that would come from a radical change in the underlying storage technology.

ajflores1604 7y ago

That's the dream of Intel's "Optane DC Persistent Memory". As with most new technology though, it's still very expensive and currently targeted exclusively at enterprise solutions. Edit: 'exclusively' is a bit heavy handed, but for the sizes that a typical consumer would think of (250gb-2tb), it still isn't available outside of a simple cache drive. This link has a good breakdown: https://www.pcworld.com/article/3388135/everything-you-need-to-know-about-optane-right-now.html https://www.pcworld.com/article/3388135/everything-you-need-...

rl3 7y ago
>... but for the sizes that a typical consumer would think of (250gb-2tb), it still isn't available outside of a simple cache drive. The 900P and 905P series are Optane devices targeted towards prosumers presumably; they offer 280GB-1.5TB of storage. That said, they're still very expensive. I really wish Optane would get cheaper, or at least refresh faster. It's so utterly superior to NAND devices in terms of random access latency and endurance. All of my systems are Optane and it's amazing, but I feel the capacity squeeze since I can't afford the higher capacity drives. Intel's decision to offer small capacity cache devices was a mistake in my opinion. It was too confusing for the average consumer, and too complicated on a vendor support level. Their prosumer 900P series was initially PCI-E x4 AIC form factor, which was also a mistake in my opinion since they later followed up with proper U.2 and M.2 form factors. Likewise their enterprise offerings aren't really great in terms of availability. It's too many SKUs that represent poor vision and execution. In hindsight they should've just directly competed with Samsung NAND in M.2 form factor somehow. Sell at a loss for all I care; the tech is so superior to NAND that it deserves a much larger foothold than it currently has.
- ajflores1604 7y ago
  I haven't had the benefit of experiencing optane first hand, but the specs look insane to me. And I completely agree, Intel bumbled the whole strategy for it. And on top of the strategy, the marketing is abysmal, not just for optane but even their processor lines. This video laid out their new 10nm processor naming scheme (https://youtu.be/hzXKDQR_1d4 https://youtu.be/hzXKDQR_1d4) and at this point it seems malicious. Like they're trying to make it so consumers accidentally get the highest margin chip on the stack by getting a low power 'i7'. Or maybe hope consumers just stop trying to compare between processor skus, or head to head benchmarks against AMD, and instead focus on the brand that is 'intel inside'.

proverbialbunny 7y ago

Latency is more important than bandwidth when it comes to reading and writing data, except for specialized work loads. I don't know the delay for current gen graphics cards, but when CUDA first came out in '07 vram had a roughly a 10x delay to system ram. vram would work great as virtual memory, where chunks of ram can be loaded and unloaded when the systems main memory runs out, but using vram as system ram today isn't ideal. Likewise running an ssd as system ram wouldn't be ideal either. This is one of the reasons why cache sizes have increased so much throughout the years. It's not just more cores. The cpu is branching out further pulling in data that will be hypothetically necessary in a few ms. Using vram you'd need the cpu to prefetch hundreds of ms ahead of time. This would add a lot of complexity to the cpu. AMD announced a cpu with a 256MB L3 cache. Imagine, in 10 years a cpu with 1GB of cache.

kbumsik 7y ago
> Latency is more important than bandwidth when it comes to reading and writing data, except for specialized work loads. It is also worth noting that there is pretty big delay between CPU and PCIe.

shaklee3 7y ago

Ideally we'd get rid of the discrete CPU altogether for systems like this. Mellanox attempted to do this with their bluefield box, which essentially put arm cores on the nics, and let them control the transfers to storage. There's really no reason to have a dual Xeon system when you're just rdma'ing to storage. Now, if we can only have that for GPU, but CUDA no longer supports arm outside of tegra.

rrss 7y ago
> CUDA no longer supports arm outside of tegra. ARM support is coming (back), apparently: https://www.nextplatform.com/2019/06/17/nvidia-makes-arm-a-peer-to-x86-and-power-for-gpu-acceleration/ https://www.nextplatform.com/2019/06/17/nvidia-makes-arm-a-p...

pdpi 7y ago

> I wonder if we'll eventually have storage cheap enough and fast enough At a fundamental level, RAM is faster than flash/hdd for two basic reasons: it's allowed to be volatile, and your working set is usually much smaller than your total storage requirements, so RAM is allowed to be more expensive on a byte-for-byte basis than persistent storage. Unless something weird happens with relative research paces, you can basically expect RAM to be faster than persistent storage for those two reasons. The question then becomes: at what point whether there is a point where even persistent storage is fast enough that your CPU/GPU/*PU can't make use of any further bandwidth/latency improvements. Looking at the relative performance of L1, L2, L3, and RAM sets some milestones you need to hit before this even becomes a long-term goal.

luma 7y ago

This can only happen if someone develops a storage technology that is simultaneously the lowest cost, lowest latency, highest bandwidth, lowest MTBF, lowest power, smallest footprint, and includes every extra feature (ECC, encryption, etc). Until that unicorn exists, engineers need to select a storage technology that meets requirements for the use case at hand. If these requirements result in a mix of storage technologies, then at some point it's likely to require moving data from one storage device to another. HPE believes (believed?) in your vision and made a lot of noise about "The Machine", a memristor-based system which incorporated a lot of ideas around the sorts of hardware and software architecture changes that would come from a radical change in the underlying storage technology.