4 ms·

I've got to imagine that at some point in the future when it's been common on server stuff for a while it'll end up as part of a next-gen "thunderbolt" style co

I've got to imagine that at some point in the future when it's been common on server stuff for a while it'll end up as part of a next-gen "thunderbolt" style connection, simply because the silicon by then will be a commodity and it'll be cheaper to use the same thing everywhere. I'm imagining a docking station that doubles the ram, and gives it a powerful GPU/NPU/TPU for a workstation. Since the ram would be close to the accelerator over CXL they can talk directly and make full bandwidth use of it even if the laptop uses the remote ram as a "very very very fast swap" or something else to prioritize things. I'm not so sure it'd be useful for persistent storage at that level but it'd make for some really cool options for a portable workstation since you wouldn't need everything inside the device.

jauntywundrkind 3y ago
CXL 3.0 is where it gets interesting, where you can start to have switched fabric where many hosts can talk to many devices. Having that pool of ram & GPUs be on demand usable by whomever has some attractive possibilities. Also, one can just imagine having some pools of data in CXL memory that a host can just attach to & read, which seems like a cool possibility. It does kind of upset me a bit that CXL 3.0 still seems purely host-to-switch-to-device oriented. If you have your formerly PCIe slots on your cores speaking CXL and doing directory memory over fabric, I'd really really love to be able to talk to other hosts. Maybe that happens & is possible in 3.0, but it feels like CXL isnt paving that cowpath, isn't making is obvious, and that there will be a bunch of proprietary nasty ways to bridge computers & chat over CXL that are all non-standard, & I wish CXL had been more direct about making themselves & their upcoming switched fabric viable & interesting for host-to-host.
- simcop2387 3y ago
  I think a large part of that is probably because the needs for doing that kind of host to host thing aren't well understood yet for the general cases. Right now I think that's being fleshed out in the market using the whole DPU model still. I think sadly you're going to be right about how it turns out for a while and it won't be until something like CXL 6.0 or whatever comes after it that starts to really address that. I'm wondering when we'll see the first CXL DPU style device that uses some of the fancy new 800Gbps or even 1.6Tbps networking stuff that's being developed right now. That'd be enough bandwidth to put things on the other end of a data-center with very little other than the latency penalty.

jauntywundrkind 3y ago

CXL 3.0 is where it gets interesting, where you can start to have switched fabric where many hosts can talk to many devices. Having that pool of ram & GPUs be on demand usable by whomever has some attractive possibilities. Also, one can just imagine having some pools of data in CXL memory that a host can just attach to & read, which seems like a cool possibility. It does kind of upset me a bit that CXL 3.0 still seems purely host-to-switch-to-device oriented. If you have your formerly PCIe slots on your cores speaking CXL and doing directory memory over fabric, I'd really really love to be able to talk to other hosts. Maybe that happens & is possible in 3.0, but it feels like CXL isnt paving that cowpath, isn't making is obvious, and that there will be a bunch of proprietary nasty ways to bridge computers & chat over CXL that are all non-standard, & I wish CXL had been more direct about making themselves & their upcoming switched fabric viable & interesting for host-to-host.

simcop2387 3y ago
I think a large part of that is probably because the needs for doing that kind of host to host thing aren't well understood yet for the general cases. Right now I think that's being fleshed out in the market using the whole DPU model still. I think sadly you're going to be right about how it turns out for a while and it won't be until something like CXL 6.0 or whatever comes after it that starts to really address that. I'm wondering when we'll see the first CXL DPU style device that uses some of the fancy new 800Gbps or even 1.6Tbps networking stuff that's being developed right now. That'd be enough bandwidth to put things on the other end of a data-center with very little other than the latency penalty.