6 ms·

Sounds like a disaster in terms of potential bugs.

jerf 4y ago
It is, but it's the same disaster of bugs we already have from multiple independent cores sharing the same memory space, not a brand new disaster or anything. It's a disaster of latency issues too, but it's not like that's surprising anyone either, and we already have NUMA on some multi-core systems which is the same problem. We have existing tools that can be extended in straightforward ways to deal with these issues. And it's not like there's a silver bullet here; having separate address spaces everywhere comes with its own disaster of issues. Pick your poison.
sidewndr46 4y ago
Instead of having another thread improperly manipulating your pointers and scribbling all over memory, now you can have an entire cluster of distributed machines doing it. This is a clear step forward.
rektide 4y ago
Not that the industry doesnt broadly deserve this FUD take/takedown, but perhaps possibly maybe it might end up being really good & useful & clean & clear & lead to very high functioning very performant very highly observable systems, for some. Having a single system image has many potential upsides, and understandability & reasonability are high among them.
yencabulator 4y ago
Just because you can refer to the identity of a thing anywhere in the cluster doesn't mean it can't also be memory-safe, capability-based, and just an RPC.
- lazide 4y ago
  Then why make said identity a fixed size number pretending to be a flat address space, instead of some other type of key?
  - yencabulator 4y ago
    For example, reads (when allowed) might just be transparent RAM accesses. I'm not advocating 128-bit pointers, or saying they're useful or realistic. I'm just saying, what if.
    - lazide 4y ago
      The issue is latency. If you don’t want to care if the read or write will take 10ns or 10000ns, then that can work, assuming a flat permission structure too. Latency is a fundamental restriction for any computer that is > zero size, and the larger the ‘computer’ (data center), the more noticeable it is. (For that matter, what happens when different segments of memory have complex access controls? What about needing to retry to failures like network partitions that don’t happen in a normal memory space?) If latency matters, which it usually does, then you need some kind of memory access hierarchy, copying things back and forth, etc. and then you’ll almost certainly need a library of some kind to manage all this, prefetch from a slow range of memory and populate some of your fast local memory, etc. At that point, we’ve done a lot of work, and are still pretending there is no network or the like, even though it’s there. It isn’t free, anyway. We’d also need checksums, cross network/fabric/access error handling, etc. And with 128 bits, we could also use something like IPv6 with the lower 64 bits being byte address hah.
      - yencabulator 4y ago
        Data centers are already doing "disaggregated memory" (page out to remote RAM), as it's been faster than local disk for years now (or at least was in the pre-NVMe world). The HPC world is all about RDMA to direct-access their huge data sets, and likely hyperscaler clouds are starting to do that too. The latency gap is just treated as yet another layer of the cache model. And we already have error correction for local RAM.
        lazide 4y ago
        Which as you note, NVME and the latest PCI generations turns on it’s head. Those were due to bottlenecks that don’t exist anymore, and even in your example were even then only emergency measures due to local resource shortages. ECC is also not reliable/sufficient in the face of issues that arrive when networks start playing their part.

jerf 4y ago

It is, but it's the same disaster of bugs we already have from multiple independent cores sharing the same memory space, not a brand new disaster or anything. It's a disaster of latency issues too, but it's not like that's surprising anyone either, and we already have NUMA on some multi-core systems which is the same problem. We have existing tools that can be extended in straightforward ways to deal with these issues. And it's not like there's a silver bullet here; having separate address spaces everywhere comes with its own disaster of issues. Pick your poison.

sidewndr46 4y ago

Instead of having another thread improperly manipulating your pointers and scribbling all over memory, now you can have an entire cluster of distributed machines doing it. This is a clear step forward.

rektide 4y ago

Not that the industry doesnt broadly deserve this FUD take/takedown, but perhaps possibly maybe it might end up being really good & useful & clean & clear & lead to very high functioning very performant very highly observable systems, for some. Having a single system image has many potential upsides, and understandability & reasonability are high among them.

yencabulator 4y ago

Just because you can refer to the identity of a thing anywhere in the cluster doesn't mean it can't also be memory-safe, capability-based, and just an RPC.

lazide 4y ago
Then why make said identity a fixed size number pretending to be a flat address space, instead of some other type of key?
- yencabulator 4y ago
  For example, reads (when allowed) might just be transparent RAM accesses. I'm not advocating 128-bit pointers, or saying they're useful or realistic. I'm just saying, what if.
  - lazide 4y ago
    The issue is latency. If you don’t want to care if the read or write will take 10ns or 10000ns, then that can work, assuming a flat permission structure too. Latency is a fundamental restriction for any computer that is > zero size, and the larger the ‘computer’ (data center), the more noticeable it is. (For that matter, what happens when different segments of memory have complex access controls? What about needing to retry to failures like network partitions that don’t happen in a normal memory space?) If latency matters, which it usually does, then you need some kind of memory access hierarchy, copying things back and forth, etc. and then you’ll almost certainly need a library of some kind to manage all this, prefetch from a slow range of memory and populate some of your fast local memory, etc. At that point, we’ve done a lot of work, and are still pretending there is no network or the like, even though it’s there. It isn’t free, anyway. We’d also need checksums, cross network/fabric/access error handling, etc. And with 128 bits, we could also use something like IPv6 with the lower 64 bits being byte address hah.
    - yencabulator 4y ago
      Data centers are already doing "disaggregated memory" (page out to remote RAM), as it's been faster than local disk for years now (or at least was in the pre-NVMe world). The HPC world is all about RDMA to direct-access their huge data sets, and likely hyperscaler clouds are starting to do that too. The latency gap is just treated as yet another layer of the cache model. And we already have error correction for local RAM.
      - lazide 4y ago
        Which as you note, NVME and the latest PCI generations turns on it’s head. Those were due to bottlenecks that don’t exist anymore, and even in your example were even then only emergency measures due to local resource shortages. ECC is also not reliable/sufficient in the face of issues that arrive when networks start playing their part.