6 ms·

So the prediction that incautious and unverified unsafe {} blocks would cause CVEs seems entirely accurate.

by themafia 11mo ago

K0nserv 11mo ago
Isn’t it obvious that the primary source of CVEs in Rust programs would be the portions of the program where the human is charge of correctness instead of the compiler? The relevant question is whether it results in fewer and less severe CVEs than code written in C. So far the answer seems to be a resounding yes
- Hemospectrum 11mo ago
  It is not obvious to those who refuse to understand, and who preemptively reject case studies on the grounds that the numbers are surely fabricated.
woodruffw 11mo ago
"Cause" seems unsubstantiated: I think to justify "cause," we'd need strong evidence that the equivalent bug (or worse) wouldn't have happened in C. Or another way to put it: clearly this is bad, and unsafe blocks deserve significant scrutiny. But it's unclear how this would have been made better by the code being entirely unsafe, rather than a particular source of unsafety being incorrect.
- ganelonhb 11mo ago
  The definition of cause is quite clear. “Rust” is obviously not the cause, but it did fail to be the solution, here. You can’t avoid that.
  - woodruffw 11mo ago
    Except that it didn't fail to be the solution: the bug is localized to an explicit escape hatch in Rust's safety rules, rather than being a latent property of the system. (I think the underlying philosophical disagreement here is this: I think software is always going to have bugs, and that Rust can't - and doesn't promise - to perfectly eliminate them. Instead, what Rust does promise - and deliver on - is that the entire class of memory safety bugs can be eliminated by construction in safe Rust, and localized when present to errors in unsafe Rust. Insofar as that's the promise, Rust has delivered here.)
    - uecker 11mo ago
      You can label something an "explicit escape hatch" or a "latent property of the system", but in the end such labels are irrelevant. While I agree that it may be easier to review unsafe blocks in Rust compared to reviewing pointer arithmetic, union accesses, and free in C because "unsafe" is a bit more obvious in the source, I think selling this as a game changer was always an exaggeration.
      - woodruffw 11mo ago
        Having written lots of C and C++ before Rust, this kind of local reasoning + correctness by construction is absolutely a game changer. It's just not a silver bullet, and efforts to miscast Rust as incorrectly claiming to be one seem heavy-handed.
        tialaramex 11mo ago
        Google's feedback seems to suggest Rust actually might be a silver bullet, in the specific sense meant in the "No Silver Bullet" essay. That essay doesn't say that silver bullets are a panacea or cure all, instead they're a decimal order of magnitude improvement. The essay gives the example of Structured Programming, an idea which feels so obvious to us today that it's unspoken, but it's really true that once upon a time people wrote unstructured programs (today the only "language" where you even could do this is assembly and nobody does it) where you just jump arbitrarily to unrelated code and resume execution. The result is fucking chaos and languages where you never do that delivered a huge improvement even before I wrote my first line of code in the 1980s. Google did find that sort of effect in Rust over C++.
        uecker 11mo ago
        As a scientist, I would not trust self reports from the industry too much. Even if those are honest, there are too many things that could bias this.
  - bigstrat2003 11mo ago
    But it didn't promise to be the solution either. Rust has never claimed, nor have its advocates claimed, that unsafe Rust can eliminate memory bugs. Safe Rust can do that (assuming any unsafe code relied upon is sound), but unsafe cannot be and has never promised to be bug free.
    - nottorp 11mo ago
      Ok so why is Rust in the kernel if you can't use safe rust there and what's left is not what Rust promises?
      - woodruffw 11mo ago
        Most of the kernel's Rust usage is safe Rust. The only unsafe Rust is there to interact with the rest of the kernel and to provide safe abstractions.
n2d4 11mo ago
Sure, but that's not really that interesting or controversial. The more useful question is, how many CVEs were prevented because unsafe {} blocks receive more caution and scrutiny?
- themafia 11mo ago
  If you could find a way to actually measure that it would be useful. I doubt this is actually achievable in our Universe. If all of C is effectively "unsafe" then wouldn't it receive the _most_ scrutiny? Since this didn't work then I don't understand Rust's overall strategy.
  - goku12 11mo ago
    That's not how it works. A larger codebase to scrutinize means that there's more chance of missing a memory safety bug. If you can keep the Rust unsafe block bug-free, you don't need to worry about them anymore in safe Rust. They're talking about attention getting divided all over the code where this distinction is not there (like C code). They always have been. On top of that, there is something else they say. You have to uphold the invariants inside the unsafe blocks. Rust for Linux documents these invariants as well. The invariant was wrong in this case. The reason I mention this is because this practice has forced even C developers to rethink and improve their code. Rust specifies very clearly what sort of error it eliminates and where it does that. It reduces the surface area of memory safety bugs to unsafe blocks, and gives you clear guidelines on what you need to ensure manually within the unsafe block to avoid any memory safety bugs. And even when you make a human error in that task, Rust makes it easy to identify them. There are clear advantages here in terms of the effort required to prevent memory safety bugs, and in making your responsibilities explicit. This has been their claim consistently. Yet, I find that these have to be repeated in every discussion about Rust. It feels like some critics don't care about these arguments at all.
mustache_kimono 11mo ago
> So the prediction that incautious and unverified unsafe {} blocks would cause CVEs seems entirely accurate. This is one/the first CVE caused by a mistake made using unsafe Rust. But it was revealed along with 159 new kernel CVEs found in C code.[0] It may just be me, but it seems wildly myopic to draw conclusions about Rust, or even, unsafe Rust from one CVE. More CVEs will absolutely happen. But even true Rust haters have to recognize that tide of CVEs in kernel C code runs something like 19+ CVEs per day? What kind of case can you make that "incautious and unverified unsafe {} blocks" is worse than that? [0]: https://social.kernel.org/notice/B1JLrtkxEBazCPQHDM https://social.kernel.org/notice/B1JLrtkxEBazCPQHDM
- uecker 11mo ago
  Github says 0.3% of the kernel code is Rust. But even normalized to lines of code, I think counting CVEs would not measure anything meaningful.
  - mustache_kimono 11mo ago
    > Github says 0.3% of the kernel code is Rust. But even normalized to lines of code, I think counting CVEs would not measure anything meaningful. Your sense seems more than a little unrigorous. 1/160 = 0.00625. So, several orders of magnitude fewer CVEs per line of code. And remember this also the first Rust kernel CVE, and any fair metric would count both any new C kernel code CVEs, as well as those which have already accrued against the same C code, if comparing raw lines of code. But taking a one week snapshot and saying Rust doesn't compare favorably to C, when Rust CVEs are 1/160, and C CVEs are 159/160 is mostly nuts.
    - mustache_kimono 11mo ago
      > Your sense seems more than a little unrigorous. 1/160 = 0.00625. So, several orders of magnitude fewer CVEs per line of code. This is incorrect. Chalk it up to the flu and fever! Sorry. 0.00625 == .625%. or about twice the instance of Rust code however as stated above these are just the metric from one patch cycle.
crote 11mo ago
Obviously. If you use a language which inherently makes memory safety bugs in regular code impossible, all memory safety bugs will be contained to the "trust me, I know what I'm doing - no need to check this" bypass sections. Similarly, all drownings happen in the presence of water. The important thing to remember is that in this context C code is one giant unsafe {} block, and you're more likely to drown in the sea than in a puddle.

So the prediction that incautious and unverified unsafe {} blocks would cause CVEs seems entirely accurate.

Isn’t it obvious that the primary source of CVEs in Rust programs would be the portions of the program where the human is charge of correctness instead of the compiler? The relevant question is whether it results in fewer and less severe CVEs than code written in C. So far the answer seems to be a resounding yes

Hemospectrum 11mo ago
It is not obvious to those who refuse to understand, and who preemptively reject case studies on the grounds that the numbers are surely fabricated.

woodruffw 11mo ago

"Cause" seems unsubstantiated: I think to justify "cause," we'd need strong evidence that the equivalent bug (or worse) wouldn't have happened in C. Or another way to put it: clearly this is bad, and unsafe blocks deserve significant scrutiny. But it's unclear how this would have been made better by the code being entirely unsafe, rather than a particular source of unsafety being incorrect.

ganelonhb 11mo ago
The definition of cause is quite clear. “Rust” is obviously not the cause, but it did fail to be the solution, here. You can’t avoid that.
- woodruffw 11mo ago
  Except that it didn't fail to be the solution: the bug is localized to an explicit escape hatch in Rust's safety rules, rather than being a latent property of the system. (I think the underlying philosophical disagreement here is this: I think software is always going to have bugs, and that Rust can't - and doesn't promise - to perfectly eliminate them. Instead, what Rust does promise - and deliver on - is that the entire class of memory safety bugs can be eliminated by construction in safe Rust, and localized when present to errors in unsafe Rust. Insofar as that's the promise, Rust has delivered here.)
  - uecker 11mo ago
    You can label something an "explicit escape hatch" or a "latent property of the system", but in the end such labels are irrelevant. While I agree that it may be easier to review unsafe blocks in Rust compared to reviewing pointer arithmetic, union accesses, and free in C because "unsafe" is a bit more obvious in the source, I think selling this as a game changer was always an exaggeration.
    - woodruffw 11mo ago
      Having written lots of C and C++ before Rust, this kind of local reasoning + correctness by construction is absolutely a game changer. It's just not a silver bullet, and efforts to miscast Rust as incorrectly claiming to be one seem heavy-handed.
      - tialaramex 11mo ago
        Google's feedback seems to suggest Rust actually might be a silver bullet, in the specific sense meant in the "No Silver Bullet" essay. That essay doesn't say that silver bullets are a panacea or cure all, instead they're a decimal order of magnitude improvement. The essay gives the example of Structured Programming, an idea which feels so obvious to us today that it's unspoken, but it's really true that once upon a time people wrote unstructured programs (today the only "language" where you even could do this is assembly and nobody does it) where you just jump arbitrarily to unrelated code and resume execution. The result is fucking chaos and languages where you never do that delivered a huge improvement even before I wrote my first line of code in the 1980s. Google did find that sort of effect in Rust over C++.
        uecker 11mo ago
        As a scientist, I would not trust self reports from the industry too much. Even if those are honest, there are too many things that could bias this.
- bigstrat2003 11mo ago
  But it didn't promise to be the solution either. Rust has never claimed, nor have its advocates claimed, that unsafe Rust can eliminate memory bugs. Safe Rust can do that (assuming any unsafe code relied upon is sound), but unsafe cannot be and has never promised to be bug free.
  - nottorp 11mo ago
    Ok so why is Rust in the kernel if you can't use safe rust there and what's left is not what Rust promises?
    - woodruffw 11mo ago
      Most of the kernel's Rust usage is safe Rust. The only unsafe Rust is there to interact with the rest of the kernel and to provide safe abstractions.

n2d4 11mo ago

Sure, but that's not really that interesting or controversial. The more useful question is, how many CVEs were prevented because unsafe {} blocks receive more caution and scrutiny?

themafia 11mo ago
If you could find a way to actually measure that it would be useful. I doubt this is actually achievable in our Universe. If all of C is effectively "unsafe" then wouldn't it receive the _most_ scrutiny? Since this didn't work then I don't understand Rust's overall strategy.
- goku12 11mo ago
  That's not how it works. A larger codebase to scrutinize means that there's more chance of missing a memory safety bug. If you can keep the Rust unsafe block bug-free, you don't need to worry about them anymore in safe Rust. They're talking about attention getting divided all over the code where this distinction is not there (like C code). They always have been. On top of that, there is something else they say. You have to uphold the invariants inside the unsafe blocks. Rust for Linux documents these invariants as well. The invariant was wrong in this case. The reason I mention this is because this practice has forced even C developers to rethink and improve their code. Rust specifies very clearly what sort of error it eliminates and where it does that. It reduces the surface area of memory safety bugs to unsafe blocks, and gives you clear guidelines on what you need to ensure manually within the unsafe block to avoid any memory safety bugs. And even when you make a human error in that task, Rust makes it easy to identify them. There are clear advantages here in terms of the effort required to prevent memory safety bugs, and in making your responsibilities explicit. This has been their claim consistently. Yet, I find that these have to be repeated in every discussion about Rust. It feels like some critics don't care about these arguments at all.

mustache_kimono 11mo ago

> So the prediction that incautious and unverified unsafe {} blocks would cause CVEs seems entirely accurate. This is one/the first CVE caused by a mistake made using unsafe Rust. But it was revealed along with 159 new kernel CVEs found in C code.[0] It may just be me, but it seems wildly myopic to draw conclusions about Rust, or even, unsafe Rust from one CVE. More CVEs will absolutely happen. But even true Rust haters have to recognize that tide of CVEs in kernel C code runs something like 19+ CVEs per day? What kind of case can you make that "incautious and unverified unsafe {} blocks" is worse than that? [0]: https://social.kernel.org/notice/B1JLrtkxEBazCPQHDM https://social.kernel.org/notice/B1JLrtkxEBazCPQHDM

uecker 11mo ago
Github says 0.3% of the kernel code is Rust. But even normalized to lines of code, I think counting CVEs would not measure anything meaningful.
- mustache_kimono 11mo ago
  > Github says 0.3% of the kernel code is Rust. But even normalized to lines of code, I think counting CVEs would not measure anything meaningful. Your sense seems more than a little unrigorous. 1/160 = 0.00625. So, several orders of magnitude fewer CVEs per line of code. And remember this also the first Rust kernel CVE, and any fair metric would count both any new C kernel code CVEs, as well as those which have already accrued against the same C code, if comparing raw lines of code. But taking a one week snapshot and saying Rust doesn't compare favorably to C, when Rust CVEs are 1/160, and C CVEs are 159/160 is mostly nuts.
  - mustache_kimono 11mo ago
    > Your sense seems more than a little unrigorous. 1/160 = 0.00625. So, several orders of magnitude fewer CVEs per line of code. This is incorrect. Chalk it up to the flu and fever! Sorry. 0.00625 == .625%. or about twice the instance of Rust code however as stated above these are just the metric from one patch cycle.

crote 11mo ago

Obviously. If you use a language which inherently makes memory safety bugs in regular code impossible, all memory safety bugs will be contained to the "trust me, I know what I'm doing - no need to check this" bypass sections. Similarly, all drownings happen in the presence of water. The important thing to remember is that in this context C code is one giant unsafe {} block, and you're more likely to drown in the sea than in a puddle.