2 ms·

I wonder if there's any possibility that an RDMA expansion device could exist in the future - i.e. a box full of RAM on the other end of a thunderbolt cable. Al

by Retr0id 11mo ago

I wonder if there's any possibility that an RDMA expansion device could exist in the future - i.e. a box full of RAM on the other end of a thunderbolt cable. Although I guess such a device would cost almost as much as a mac mini in any case...

roadbuster 11mo ago
You still need an interface which does at least two things: handles incoming read/write requests using some kind of network protocol, and operates as a memory controller for the RAM. Texas Memory Systems was in the business of making large 'RAM Drives'. They had a product line known as "RamSan" which made many gigabytes/terabytes of DDR available via a block storage interface over infiniband and fibre channel. The control layer was implemented via FPGA. I recall a press release from 2004 which publicized the US govt purchase of a 2.5TB RamSan. They later expanded into SSDs and were acquired by IBM in 2012. https://en.wikipedia.org/wiki/Texas_Memory_Systems https://en.wikipedia.org/wiki/Texas_Memory_Systems https://www.lhcomp.com/vendors/tms/TMS-RamSan300-DataSheet.pdf https://www.lhcomp.com/vendors/tms/TMS-RamSan300-DataSheet.p... https://gizmodo.com/u-s-government-purchases-worlds-largest-ram-disk-8754 https://gizmodo.com/u-s-government-purchases-worlds-largest-... https://www.lhcomp.com/vendors/tms/TMS-RamSan20-DataSheet.pdf https://www.lhcomp.com/vendors/tms/TMS-RamSan20-DataSheet.pd... https://www.ibm.com/support/pages/ibm-plans-acquire-texas-memory-systems https://www.ibm.com/support/pages/ibm-plans-acquire-texas-me...
RantyDave 11mo ago
Couldn't you "just" use a honking fast SSD and set it as a swap drive?
- Retr0id 11mo ago
  You might get close in peak bandwidth, but not in random access and latency.
amluto 11mo ago
RDMA is not really intended for this. RDMA is really just a bunch of functionality of a PCIe device, and even PCIe isn’t really quite right to use like RAM because its cache semantics aren’t intended for this use case. But the industry knows this, and there’s a technology that is electrically compatible with PCIe that is intended for use as RAM among other things: CXL. I wonder if a anyone will ever build CXL over USB-C.

by Retr0id 11mo ago

roadbuster 11mo ago

You still need an interface which does at least two things: handles incoming read/write requests using some kind of network protocol, and operates as a memory controller for the RAM. Texas Memory Systems was in the business of making large 'RAM Drives'. They had a product line known as "RamSan" which made many gigabytes/terabytes of DDR available via a block storage interface over infiniband and fibre channel. The control layer was implemented via FPGA. I recall a press release from 2004 which publicized the US govt purchase of a 2.5TB RamSan. They later expanded into SSDs and were acquired by IBM in 2012. https://en.wikipedia.org/wiki/Texas_Memory_Systems https://en.wikipedia.org/wiki/Texas_Memory_Systems https://www.lhcomp.com/vendors/tms/TMS-RamSan300-DataSheet.pdf https://www.lhcomp.com/vendors/tms/TMS-RamSan300-DataSheet.p... https://gizmodo.com/u-s-government-purchases-worlds-largest-ram-disk-8754 https://gizmodo.com/u-s-government-purchases-worlds-largest-... https://www.lhcomp.com/vendors/tms/TMS-RamSan20-DataSheet.pdf https://www.lhcomp.com/vendors/tms/TMS-RamSan20-DataSheet.pd... https://www.ibm.com/support/pages/ibm-plans-acquire-texas-memory-systems https://www.ibm.com/support/pages/ibm-plans-acquire-texas-me...

RantyDave 11mo ago

Couldn't you "just" use a honking fast SSD and set it as a swap drive?

Retr0id 11mo ago
You might get close in peak bandwidth, but not in random access and latency.

amluto 11mo ago

RDMA is not really intended for this. RDMA is really just a bunch of functionality of a PCIe device, and even PCIe isn’t really quite right to use like RAM because its cache semantics aren’t intended for this use case. But the industry knows this, and there’s a technology that is electrically compatible with PCIe that is intended for use as RAM among other things: CXL. I wonder if a anyone will ever build CXL over USB-C.