3 ms·

as someone whos really interested in this kind of thing, what kind of memory semantics specifically does c++ support compared to rust?

jandrewrogers 6y ago
A couple examples: Rust requires references to memory to be visible at compile time. If you use direct DMA for I/O then the silicon holds mutable memory references at runtime that are not visible in code at compile time. In C++ you can make ownership objects (in the vein of unique_ptr) that automagically track the lifetime of DMA references. Exposing most of your address space to DMA is not optional in databases if you care about performance. DMA has other nuances as well e.g. it can modify memory that is not part of the object you care about and which might be "owned" by someone else. There are safety models wherein it is provably safe, and optimal code-wise, for multiple mutable references to the same object to coexist at runtime. Modern databases often use schedule-based safety, which dynamically reorders operations on references at a fine-grained level, for performance reasons, and which guarantee correctness of the aggregate execution graph without using locks. This wasn't common a decade ago but it is today. A developer can grab all the mutable references they need to objects, knowing that the execution scheduler can see through and arbitrate any potential conflicts. (This is a sophisticated reimagining of the old "deadlock-free" locking structures in databases without the locks, which didn't prevent deadlocks per se but transparently detect and resolve them.) You can make all of this work in Rust by using a lot of "unsafe" and inelegant data structure workarounds but it voids most of the safety benefits of Rust and requires more complex code.
- andrekandre 6y ago
  thanks for the reply! > In C++ you can make ownership objects (in the vein of unique_ptr) that automagically track the lifetime of DMA references. does that require os support to make it work? i assume the tracking is handled by some specific inturrupts? > You can make all of this work in Rust by using a lot of "unsafe" and inelegant data structure workarounds but it voids most of the safety benefits of Rust and requires more complex code. i see... so i guess rust would have to take one small step out of static, and into dynamic territory with a smart pointer "trait"?
  - jandrewrogers 6y ago
    The OS has to provide syscalls and other APIs for scheduling DMA operations into your address space. Linux has supported this to some degree for a very long time (storage DMA was introduced in the 2.5 kernels). In terms of behavior and APIs, storage is very different than network, since the former is schedule-driven and the latter is interrupt-driven. There is a lot of implementation detail, but you need object references that are aware when a DMA operation is outstanding on address space they “own”, the type of DMA operation, and rules about when new DMA operations can be scheduled with respect to object reference lifecycles. There are no fixed set of rules but there are a common set of design patterns. The scheduler design does a lot of the heavy lifting.
    - andrekandre 6y ago
      i see, very interesting, thanks ^^
- Communitivity 6y ago
  It takes work, but you can write safe DMA code in Rust. A good post by Jorge Aparicio explaining how is at his blog[1]. [1] https://blog.japaric.io/safe-dma/#:~:text=DMA%20stands%20for%20Direct%20Memory,as%20providing%20asynchronous%20memcpy%20functionality https://blog.japaric.io/safe-dma/#:~:text=DMA%20stands%20for...

jandrewrogers 6y ago

A couple examples: Rust requires references to memory to be visible at compile time. If you use direct DMA for I/O then the silicon holds mutable memory references at runtime that are not visible in code at compile time. In C++ you can make ownership objects (in the vein of unique_ptr) that automagically track the lifetime of DMA references. Exposing most of your address space to DMA is not optional in databases if you care about performance. DMA has other nuances as well e.g. it can modify memory that is not part of the object you care about and which might be "owned" by someone else. There are safety models wherein it is provably safe, and optimal code-wise, for multiple mutable references to the same object to coexist at runtime. Modern databases often use schedule-based safety, which dynamically reorders operations on references at a fine-grained level, for performance reasons, and which guarantee correctness of the aggregate execution graph without using locks. This wasn't common a decade ago but it is today. A developer can grab all the mutable references they need to objects, knowing that the execution scheduler can see through and arbitrate any potential conflicts. (This is a sophisticated reimagining of the old "deadlock-free" locking structures in databases without the locks, which didn't prevent deadlocks per se but transparently detect and resolve them.) You can make all of this work in Rust by using a lot of "unsafe" and inelegant data structure workarounds but it voids most of the safety benefits of Rust and requires more complex code.

andrekandre 6y ago
thanks for the reply! > In C++ you can make ownership objects (in the vein of unique_ptr) that automagically track the lifetime of DMA references. does that require os support to make it work? i assume the tracking is handled by some specific inturrupts? > You can make all of this work in Rust by using a lot of "unsafe" and inelegant data structure workarounds but it voids most of the safety benefits of Rust and requires more complex code. i see... so i guess rust would have to take one small step out of static, and into dynamic territory with a smart pointer "trait"?
- jandrewrogers 6y ago
  The OS has to provide syscalls and other APIs for scheduling DMA operations into your address space. Linux has supported this to some degree for a very long time (storage DMA was introduced in the 2.5 kernels). In terms of behavior and APIs, storage is very different than network, since the former is schedule-driven and the latter is interrupt-driven. There is a lot of implementation detail, but you need object references that are aware when a DMA operation is outstanding on address space they “own”, the type of DMA operation, and rules about when new DMA operations can be scheduled with respect to object reference lifecycles. There are no fixed set of rules but there are a common set of design patterns. The scheduler design does a lot of the heavy lifting.
  - andrekandre 6y ago
    i see, very interesting, thanks ^^
Communitivity 6y ago
It takes work, but you can write safe DMA code in Rust. A good post by Jorge Aparicio explaining how is at his blog[1]. [1] https://blog.japaric.io/safe-dma/#:~:text=DMA%20stands%20for%20Direct%20Memory,as%20providing%20asynchronous%20memcpy%20functionality https://blog.japaric.io/safe-dma/#:~:text=DMA%20stands%20for...