5 ms·

The caveat here is that some correct software architectures require littering so much "unsafe" in the code (due to incompatible safety models, not actual unsafe

The caveat here is that some correct software architectures require littering so much "unsafe" in the code (due to incompatible safety models, not actual unsafe-ness) that it largely defeats the purpose, and a software architecture that lets you avoid most "unsafe" produces a worse product while requiring more lines of code to accomplish the same thing. Rust will always leave plenty of room for C++ to the extent that it tacitly encourages suboptimal software architecture for some types of applications, such as database engines, that commonly rely on safety models Rust was not designed to express. I do see Rust potentially replacing a lot of backend Java, eventually.

Verdex 7y ago
I'm glad you commented because this is what I wanted to say, but couldn't think of a good way to lead into it. IIRC while Rust does allow you to switch up allocators, it doesn't let you mix and match your runtime with multiple allocators. (Each artifact can be linked with at most one allocator at a time.) There are applications where you want to have multiple allocation strategies for performance reasons. Uniqueness is really useful for most applications, but there are times that you want data structures that allow multiple pointers to the same data. Having to do this in Rust is going to be a bigger chore than doing it in a language that doesn't support uniqueness. It will be possible for Rust to still participate in these areas (especially because you can use Rust for only part of your project, so you can use it where you don't have allocation or nonuniqueness constraints in your problem space). However, other options are going to offer a better programmer experience.
- steveklabnik 7y ago
  > IIRC while Rust does allow you to switch up allocators, it doesn't let you mix and match your runtime with multiple allocators. (Each artifact can be linked with at most one allocator at a time.) There are applications where you want to have multiple allocation strategies for performance reasons. So, sort of yes, and sort of no. Like, you can swap the global allocator, but there's no way to parameterize standard library stuff over anything but the given allocator. But for your own code, you can write and use allocators however you want. Arenas are often popular, for example.
bluejekyll 7y ago
> such as database engines, that commonly rely on safety models Rust was not designed to express. Do you think you could expand on this? From my experience Rust facilitates all the same operations as either C or C++, and generally without even needing to turn to unsafe. What I've found in my own (not DB, but networking) work, is that Rust generally asks you to restate the problem in a way that will allow it to be best expressed in Rust. This often differs from the down the middle of the road implementations people have grown used to in other languages, but it doesn't in any meaningful way prevent you from solving the problem, in a safe way.
- jandrewrogers 7y ago
  There are a couple common architectural patterns that are easy to express in C++ but tend to violate assumptions in Rust's safety model. Database engines run into them frequently due to the nature of their core operations, and this ignores that most data structures are intrinsically globals (which Rust doesn't like) due to tight hardware coupling. Rust assumes that all references to memory are visible at compile-time, and the safety analysis can be applied in cases where this is true with the usual caveats around borrow-checking. The "unsafe" facility is designed to interface with code written in other languages that don't respect Rust's model, and it works well for that. But how do you express the case, common in database engines (because direct storage-backed), where hardware can hold mutable references to most of your address space? There is no way to determine at compile-time if a mutable reference will be unique at runtime or to even sandbox it to a small bit of code. As a consequence, most memory reference are effectively mutable. There are workarounds that will minimize the quantity of unsafe code in Rust if you are willing to sacrifice performance and elegance. In databases, having many mutable references to the same memory has few safety implications because ownership of memory is dynamically assigned at runtime by a scheduler that guarantees safe access without locking or blocking. This safety model solves the hardware ownership problem, which is why it is used, but it also enables quite a bit of dynamic optimization even if all your references are in software so you'd want to do things this way anyway. In C++, you can make all of this largely transparent on top of explicitly mutable references to memory. Again, you can produce a minimally unsafe version of this in Rust but it is going to be significantly uglier and slower. As more server software moves to userspace I/O and scheduling models (for performance and scale reasons) it will be interesting to see how this impedance mismatch problem is addressed in Rust.
apta 7y ago
> I do see Rust potentially replacing a lot of backend Java, eventually. I'm not quite so sure. Rust can be a good fit for services that need to be very optimized for memory or CPU usage. Java brings so many other extremely important benefits like introspection, management, profiling, hot swapping, and being generally more productive due to the fact that it's a GC'd language. Not to mention the huge ecosystem behind it.
nickpsecurity 7y ago
That's all or nothing thinking. Enough people do it that what you say will probably happen. Thing is, one can always use multiple tools to achieve their goal. Anything Rust's safety model can't handle might be done with a different model, analyzer, etc. One recommendation I keep at is using "unsafe" Rust, porting it to identical C, throwing every tool we have in C ecosystem at it, and port what passes back to Rust with safe wrappers if possible. Rust couldn't prove it safe, it's externally proven safe (or safe enough), and optionally has protections during interactions via wrappers. You get Rust's benefits on everything else you code in the app plus whatever you include that others manage to get past borrow checker. I call this general concept Brute-Force Assurance where you just modify the form of a program to fit existing tools to get their benefits. Just throw every sound and/or complete analyzer plus a lot of test generators at it. Also, code in a way that helps those tools wherever possible. If one can't, then use them on a version designed for verification first to get the algorithm right, step it toward optimized version, equivalence tests, repeat, etc.

Verdex 7y ago

I'm glad you commented because this is what I wanted to say, but couldn't think of a good way to lead into it. IIRC while Rust does allow you to switch up allocators, it doesn't let you mix and match your runtime with multiple allocators. (Each artifact can be linked with at most one allocator at a time.) There are applications where you want to have multiple allocation strategies for performance reasons. Uniqueness is really useful for most applications, but there are times that you want data structures that allow multiple pointers to the same data. Having to do this in Rust is going to be a bigger chore than doing it in a language that doesn't support uniqueness. It will be possible for Rust to still participate in these areas (especially because you can use Rust for only part of your project, so you can use it where you don't have allocation or nonuniqueness constraints in your problem space). However, other options are going to offer a better programmer experience.

steveklabnik 7y ago
> IIRC while Rust does allow you to switch up allocators, it doesn't let you mix and match your runtime with multiple allocators. (Each artifact can be linked with at most one allocator at a time.) There are applications where you want to have multiple allocation strategies for performance reasons. So, sort of yes, and sort of no. Like, you can swap the global allocator, but there's no way to parameterize standard library stuff over anything but the given allocator. But for your own code, you can write and use allocators however you want. Arenas are often popular, for example.

bluejekyll 7y ago

> such as database engines, that commonly rely on safety models Rust was not designed to express. Do you think you could expand on this? From my experience Rust facilitates all the same operations as either C or C++, and generally without even needing to turn to unsafe. What I've found in my own (not DB, but networking) work, is that Rust generally asks you to restate the problem in a way that will allow it to be best expressed in Rust. This often differs from the down the middle of the road implementations people have grown used to in other languages, but it doesn't in any meaningful way prevent you from solving the problem, in a safe way.

jandrewrogers 7y ago
There are a couple common architectural patterns that are easy to express in C++ but tend to violate assumptions in Rust's safety model. Database engines run into them frequently due to the nature of their core operations, and this ignores that most data structures are intrinsically globals (which Rust doesn't like) due to tight hardware coupling. Rust assumes that all references to memory are visible at compile-time, and the safety analysis can be applied in cases where this is true with the usual caveats around borrow-checking. The "unsafe" facility is designed to interface with code written in other languages that don't respect Rust's model, and it works well for that. But how do you express the case, common in database engines (because direct storage-backed), where hardware can hold mutable references to most of your address space? There is no way to determine at compile-time if a mutable reference will be unique at runtime or to even sandbox it to a small bit of code. As a consequence, most memory reference are effectively mutable. There are workarounds that will minimize the quantity of unsafe code in Rust if you are willing to sacrifice performance and elegance. In databases, having many mutable references to the same memory has few safety implications because ownership of memory is dynamically assigned at runtime by a scheduler that guarantees safe access without locking or blocking. This safety model solves the hardware ownership problem, which is why it is used, but it also enables quite a bit of dynamic optimization even if all your references are in software so you'd want to do things this way anyway. In C++, you can make all of this largely transparent on top of explicitly mutable references to memory. Again, you can produce a minimally unsafe version of this in Rust but it is going to be significantly uglier and slower. As more server software moves to userspace I/O and scheduling models (for performance and scale reasons) it will be interesting to see how this impedance mismatch problem is addressed in Rust.

apta 7y ago

> I do see Rust potentially replacing a lot of backend Java, eventually. I'm not quite so sure. Rust can be a good fit for services that need to be very optimized for memory or CPU usage. Java brings so many other extremely important benefits like introspection, management, profiling, hot swapping, and being generally more productive due to the fact that it's a GC'd language. Not to mention the huge ecosystem behind it.

nickpsecurity 7y ago

That's all or nothing thinking. Enough people do it that what you say will probably happen. Thing is, one can always use multiple tools to achieve their goal. Anything Rust's safety model can't handle might be done with a different model, analyzer, etc. One recommendation I keep at is using "unsafe" Rust, porting it to identical C, throwing every tool we have in C ecosystem at it, and port what passes back to Rust with safe wrappers if possible. Rust couldn't prove it safe, it's externally proven safe (or safe enough), and optionally has protections during interactions via wrappers. You get Rust's benefits on everything else you code in the app plus whatever you include that others manage to get past borrow checker. I call this general concept Brute-Force Assurance where you just modify the form of a program to fit existing tools to get their benefits. Just throw every sound and/or complete analyzer plus a lot of test generators at it. Also, code in a way that helps those tools wherever possible. If one can't, then use them on a version designed for verification first to get the algorithm right, step it toward optimized version, equivalence tests, repeat, etc.