4 ms·

No the solution isn't to rewrite it in Rust. The solution is to have the option of compiling your C/C++ program with memory safety whenever things go loopy. ASA

by jart 2y ago

No the solution isn't to rewrite it in Rust. The solution is to have the option of compiling your C/C++ program with memory safety whenever things go loopy. ASAN, MSAN, and UBSAN are one great way to do that. Another up and coming solution that promises even more memory safety is Fil-C which is being made by Epic Games. https://github.com/pizlonator/llvm-project-deluge/blob/deluge/Manifesto.md https://github.com/pizlonator/llvm-project-deluge/blob/delug...

kelnos 2y ago
The solution is usually not to do a rewrite, but I think for greenfield projects we should stop using C or C++ unless there is a compelling reason to do so. Memory-safe systems languages are available today; IMO it's professionally irresponsible to not use them, without a good reason. MSAN, ASAN, and UBSAN are great tools that have saved me a lot of time and headaches, but they don't catch everything that the compiler of a memory safe language can, at least not today.
- jart 2y ago
  Rust isn't standardized. Last time I checked, everyone who uses it depends on its nightly build. Their toolchain is enormous and isn't vendorable. The binaries it builds are quite large. Programs take a very long time to compile. You need to depend on web servers to do your development and use lots of third party libraries maintained by people you've never heard of, because Rust adopted NodeJS' quadratic dependency model. Choosing Rust will greatly limit your audience if you're doing an open source project, since your users need to install Rust to build your program, and there are many platforms Rust doesn't support. Rust programs use unsafe a lot in practice. One of the greatest difficulties I've had in supporting Rust with Cosmopolitan Libc is that Rust libraries all try to be clever by using raw assembly system calls rather than using libc. So our Rust binaries will break mysteriously when I run them on other OSes. Everyone who does AI or scientific computing with Rust, if you profile their programs, I guarantee you 99% of the time it's going to be inside C/C++ code. If better C/C++ tools can give us memory safety, then how much difference does it really make if it's baked into the language syntax. Rust can't prove everything at compile time. Some of the Rust programs I've used like Alacrity will runtime panic all the time. Because the language doesn't actually save you. What saves you is smart people spending 5+ years hammering out all the bugs. That's why old tools we depend on every day like GNU programs never crash and their memory bugs are rare enough to be newsworthy. The Rust community has a reputation for toxic behavior that raises questions about its the reliability of its governance. Rust evangelizes its ideas by attacking other languages and socially ostracizing the developers who use them. Software development is the process of manipulating memory, so do you really want to be relinquishing control over your memory to these kinds of people?
  - fc417fc802 2y ago
    > Not standardized. > Dependency soup. Exactly why I don't use it. I don't really feel like including the source for the entire toolchain as part of my project and building it all myself. At least if I write standards conforming C++ there are multiple compiler implementations that can all handle it. I also have a reasonable expectation that a few decades from now I will be able to `apt get somecompiler` and the code will still just work (aside from any API changes at the OS level, for which compatibility shims will almost certainly exist). If I can't build something starting from a repo in a network isolated environment then I want absolutely nothing to do with it. (Emscripten I am looking at you. I will not be downloading sketchy binary blobs from cloud storage to "build from source" that is not a source build that is binary distribution you liars.)
  - vvanders 2y ago
    You may want to refresh your familiarity with Rust, I haven't touched nightly in ages and much of what you mention doesn't really resonate with what I've seen in practice. Not saying the language doesn't have issues and things that aren't frustrating but in my experience unless you're going to go to the nines in testing/validation/etc (which is the first thing that's cut when schedules/etc are in peril) I've seen Rust code scale better than C++ ever did. More tools in the C/C++ realm are always welcome but I've yet to see more than 50% of projects I've worked on be able to successfully use ASAN(assuming you've got the time to burn to configure them and all their dependencies properly). I've used ASAN, CBMC and other tools to good effect but find Rust more productive overall.
  - saagarjha 2y ago
    > One of the greatest difficulties I've had in supporting Rust with Cosmopolitan Libc is that Rust libraries all try to be clever by using raw assembly system calls rather than using libc. I’m sorry, this is coming from Justine “the magic syscall numbers are my god given right to use” Tunney?
    - fc417fc802 2y ago
      Seems like it depends entirely on context. I'd expect code which intends to be portable to use some sort of dynamically linked wrapper, even if that wrapper isn't libc.
AlotOfReading 2y ago
Ubsan is fantastic, but ASAN and the rest have serious caveats. They're not suitable for production use and they have a tendency to break in mysterious, intermittent ways. For example, Ubuntu 24.04 unknowingly broke Clang <=15ish when it increased mmap_rnd_bits. ASAN on Windows will actually check if you have ASLR enabled, disable it, and restart at entry. They interact in fun ways with LD_PRELOAD too.
- james_promoted 2y ago
  I'm on Clang 19 and still have a bunch of those sysctl commands sitting around.
  - AlotOfReading 2y ago
    I'm not in a position to look up exactly when it was merged, but I'm pretty confident that shouldn't be needed anymore. The entry point on 19 should do the same restart juggling it does on Windows if the environment isn't correct for some other reason. I can double check later if you want to provide details. I encountered the issue when our (not Ubuntu, not 24.04) LTS upstream backported security fixes that included the mmap changes without updating universe to include a clang version with the fixes. Any developers diligent enough to update and run sanitisers locally started seeing intermittent crashes.
saagarjha 2y ago
None of those sanitizers provide full memory safety for C/C++.
ryao 2y ago
That would not catch this. This is technically undefined behavior, but UBSAN does not have a check for this: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#ubsan-checks https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#... What would catch this would be a C++ compiler warning. I filed bug reports against both GCC and LLVM asking for one: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=118263 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=118263 https://github.com/llvm/llvm-project/issues/121427 https://github.com/llvm/llvm-project/issues/121427

by jart 2y ago

kelnos 2y ago

The solution is usually not to do a rewrite, but I think for greenfield projects we should stop using C or C++ unless there is a compelling reason to do so. Memory-safe systems languages are available today; IMO it's professionally irresponsible to not use them, without a good reason. MSAN, ASAN, and UBSAN are great tools that have saved me a lot of time and headaches, but they don't catch everything that the compiler of a memory safe language can, at least not today.

jart 2y ago
Rust isn't standardized. Last time I checked, everyone who uses it depends on its nightly build. Their toolchain is enormous and isn't vendorable. The binaries it builds are quite large. Programs take a very long time to compile. You need to depend on web servers to do your development and use lots of third party libraries maintained by people you've never heard of, because Rust adopted NodeJS' quadratic dependency model. Choosing Rust will greatly limit your audience if you're doing an open source project, since your users need to install Rust to build your program, and there are many platforms Rust doesn't support. Rust programs use unsafe a lot in practice. One of the greatest difficulties I've had in supporting Rust with Cosmopolitan Libc is that Rust libraries all try to be clever by using raw assembly system calls rather than using libc. So our Rust binaries will break mysteriously when I run them on other OSes. Everyone who does AI or scientific computing with Rust, if you profile their programs, I guarantee you 99% of the time it's going to be inside C/C++ code. If better C/C++ tools can give us memory safety, then how much difference does it really make if it's baked into the language syntax. Rust can't prove everything at compile time. Some of the Rust programs I've used like Alacrity will runtime panic all the time. Because the language doesn't actually save you. What saves you is smart people spending 5+ years hammering out all the bugs. That's why old tools we depend on every day like GNU programs never crash and their memory bugs are rare enough to be newsworthy. The Rust community has a reputation for toxic behavior that raises questions about its the reliability of its governance. Rust evangelizes its ideas by attacking other languages and socially ostracizing the developers who use them. Software development is the process of manipulating memory, so do you really want to be relinquishing control over your memory to these kinds of people?
- fc417fc802 2y ago
  > Not standardized. > Dependency soup. Exactly why I don't use it. I don't really feel like including the source for the entire toolchain as part of my project and building it all myself. At least if I write standards conforming C++ there are multiple compiler implementations that can all handle it. I also have a reasonable expectation that a few decades from now I will be able to `apt get somecompiler` and the code will still just work (aside from any API changes at the OS level, for which compatibility shims will almost certainly exist). If I can't build something starting from a repo in a network isolated environment then I want absolutely nothing to do with it. (Emscripten I am looking at you. I will not be downloading sketchy binary blobs from cloud storage to "build from source" that is not a source build that is binary distribution you liars.)
- vvanders 2y ago
  You may want to refresh your familiarity with Rust, I haven't touched nightly in ages and much of what you mention doesn't really resonate with what I've seen in practice. Not saying the language doesn't have issues and things that aren't frustrating but in my experience unless you're going to go to the nines in testing/validation/etc (which is the first thing that's cut when schedules/etc are in peril) I've seen Rust code scale better than C++ ever did. More tools in the C/C++ realm are always welcome but I've yet to see more than 50% of projects I've worked on be able to successfully use ASAN(assuming you've got the time to burn to configure them and all their dependencies properly). I've used ASAN, CBMC and other tools to good effect but find Rust more productive overall.
- saagarjha 2y ago
  > One of the greatest difficulties I've had in supporting Rust with Cosmopolitan Libc is that Rust libraries all try to be clever by using raw assembly system calls rather than using libc. I’m sorry, this is coming from Justine “the magic syscall numbers are my god given right to use” Tunney?
  - fc417fc802 2y ago
    Seems like it depends entirely on context. I'd expect code which intends to be portable to use some sort of dynamically linked wrapper, even if that wrapper isn't libc.

AlotOfReading 2y ago

Ubsan is fantastic, but ASAN and the rest have serious caveats. They're not suitable for production use and they have a tendency to break in mysterious, intermittent ways. For example, Ubuntu 24.04 unknowingly broke Clang <=15ish when it increased mmap_rnd_bits. ASAN on Windows will actually check if you have ASLR enabled, disable it, and restart at entry. They interact in fun ways with LD_PRELOAD too.

james_promoted 2y ago
I'm on Clang 19 and still have a bunch of those sysctl commands sitting around.
- AlotOfReading 2y ago
  I'm not in a position to look up exactly when it was merged, but I'm pretty confident that shouldn't be needed anymore. The entry point on 19 should do the same restart juggling it does on Windows if the environment isn't correct for some other reason. I can double check later if you want to provide details. I encountered the issue when our (not Ubuntu, not 24.04) LTS upstream backported security fixes that included the mmap changes without updating universe to include a clang version with the fixes. Any developers diligent enough to update and run sanitisers locally started seeing intermittent crashes.

saagarjha 2y ago

None of those sanitizers provide full memory safety for C/C++.

ryao 2y ago

That would not catch this. This is technically undefined behavior, but UBSAN does not have a check for this: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#ubsan-checks https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#... What would catch this would be a C++ compiler warning. I filed bug reports against both GCC and LLVM asking for one: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=118263 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=118263 https://github.com/llvm/llvm-project/issues/121427 https://github.com/llvm/llvm-project/issues/121427