23 ms·

Limiting to the "top 5" vulnerabilities by number of CVEs feels like cherry-picking. It's true that spatial memory safety is lower-hanging fruit than temporal m

Limiting to the "top 5" vulnerabilities by number of CVEs feels like cherry-picking. It's true that spatial memory safety is lower-hanging fruit than temporal memory safety, sure; but the CVE list is dominated by vulnerabilities in web applications which are mostly already written in fully memory-safe languages (additionally, these vulnerabilities are typically mitigated by library/API design rather than by programming language design, in which case Rust gives you quite a lot more tools than Zig for designing APIs that have concepts of unsanitized/untrusted data, but I digress). If you filter for vulnerabilities relevant to applications that would typically be written in C/C++/Rust/Zig, use-after-free is easily within the top 5. The exact positioning within the top 10 is also quite noisy: if you look at last year's list, UAF is in the #1 spot for actively exploited vulnerabilities, even beating out all the web ones: https://cwe.mitre.org/top25/archive/2023/2023_kev_list.html https://cwe.mitre.org/top25/archive/2023/2023_kev_list.html Lastly, filtering on CVEs has a high selection bias: just because security researchers go for the easy vulnerabilities first doesn't mean that the harder ones can be ignored. As high-profile projects adopt tooling and mitigations to reduce the impact of spatial memory safety problems, it's common to see a sudden increase in CVEs related to temporal memory safety. This is not because use-after-free bugs somehow became more common or severe -- it's because once you eliminate the easy source of vulnerabilities, the attackers shift their attention to the new lowest-hanging fruit.

pron 11mo ago
> Limiting to the "top 5" vulnerabilities by number of CVEs feels like cherry-picking. The point isn't about limiting to the top 5. The point is that once you get to the things Rust prevents and Zig doesn't, there are quite a few more things that neither prevents, so it's just silly to draw a a particular sharp line between Rust and Zig because they perform exactly the same (in terms of sound guarantees; we're ignoring any softer effects) for most top weaknesses. Even if you think that difference is so important that it justifies downsides that Rust may have in comparison, you still have to admit that Zig is much, much closer to Rust than to C by that measure. And this "closer" matters because Rust's memory safety is also not absolute, and Rust proponents must accept that the cost of memory safety is an important factor, too, and sometimes not worth it, or else Rust wouldn't have been invented in the first place. After all, languages that are as memory-safe as Rust and more were more popular than Rust will ever be before it was even invented. So Rust proponents must accept that eliminating dangerous vulnerabilities is good (unlike C), that productivity and cost do matter (unlike ATS), and that non-absolute memory safety is acceptable. And Zig satisfies all of these points, too. The reason it's hard to find an objective metric to draw the line between Rust and Zig is because they're actually quite close to each other, at least on this front of trying to find a useful compromise between productivity and guarantees. > Lastly, filtering on CVEs has a high selection bias: just because security researchers go for the easy vulnerabilities first doesn't mean that the harder ones can be ignored. Sure, but then you might as well also consider softer effects. For example, maybe a language that's easier to review because it's more explicit, or a language that's faster to compile and is easier to test wins. And I agree that we should consider all these, but then we start seeing why correctness is such a complicated topic, and we could speculate just as easily that it is Zig that "clearly" wins. Anyway, it's perfectly fine for people to prefer Rust because they like it. But the attempt to find objective reasons for this preference is not based on any truly objective foundations, and just looks like some desperate rationalisation. And BTW, > relevant to applications that would typically be written in C/C++/Rust/Zig If you think that Rust and Zig are designed to target the exact same domains, then some of the "softer" aspects I mentioned could play even a larger role. I mean, the portion of software written in low level languages has been declining steadily for a long time with no sign of a change in the trend. To me it seems that Zig has internalised the narrower and more focused and role of low-level languages today compared to what C++ imagined it would be in the eighties.
- NobodyNada 11mo ago
  > it's just silly to draw a a particular sharp line between Rust and Zig because they perform exactly the same (in terms of sound guarantees; we're ignoring any softer effects) for most top weaknesses. There is a very clear sharp line between them in that Rust has no undefined behavior outside of an unsafe block. This matters because the effects of undefined behavior -- particularly memory-corrupting undefined behavior -- are unbounded. Security issues caused by logic bugs are limited to only having local effects on the program -- a SQL injection bug can lead to the wrong SQL query being executed, a path traversal bug can lead to the wrong path being accessed, and a shell escaping bug can lead to execution of the wrong shell command. These are severe problems that can lead to data exfiltration or remote code execution, but the relationship between bug and effect is straightforward: "I'm passing user input to a shell here; this could have catastrophic consequences so I'd better review this carefully." In contrast, undefined behavior can happen anywhere in your program, and it can do anything. That is a clear and measurable difference, and emperical evidence from the last 10 years clearly indicates both that it is nigh impossible to write large C/C++ applications without both spatial and temporal memory safety vulnerabilities, and that adopting Rust measurably decreases the incidence of such vulnerabilities. Zig is certainly an improvement over C on this front, but it is still a UB-heavy language, and that's where the sharp line is. It's hard to make emperical comparisons because Zig is so young, but the comparision between Deno and Bun in the article is a reasonably strong demonstration that Zig has not achieved a comparable level of memory safety to Rust. > Sure, but then you might as well also consider softer effects. For example, maybe a language that's easier to review because it's more explicit, or a language that's faster to compile and is easier to test wins. > And I agree that we should consider all these, but then we start seeing why correctness is such a complicated topic, and we could speculate just as easily that it is Zig that "clearly" wins. This doesn't really have anything to do with my point that the CVE list does not provide evidence to dismiss temporal memory safety as irrelevant; it's more of a general statement on writing correct software. But regardless, "Zig has nullability and bounds checks" and "Rust has an affine type system, statically-checked immutability and exclusivity, language-level resource management, and no undefined behavior" are not in the same league of program correctness. Rust wasn't designed after some ideal of memory safety at the expense of clarity and correctness: the lifetime and type system came from a goal of reducing logic bugs in complex concurrent programs, and the fact that it is powerful enough to achieve memory safety without garbage collection was a happy accident. The emperical results that Rust programs have fewer memory-safety vulnerabilities demonstrate that Rust's static approach to software correctness is successful, and not just for the specific problem of memory safety, because Rust's tooling for achieving program correctness is generalizable to arbitrary application invariants. This lines up with my own experience; software I write using Rust is far easier to get right, more reliable, and easier to successfully maintain and refactor than anything else I've done. Certainly Zig has its strong points as well -- explicitness can reduce complexity and compile times, but it also has downsides of pushing complexity into application code (thus making it harder to review and introducing more opportunities to create mistakes). A proper comparison of the two approaches is worthwhile, but just as "Rust is more memory safe" is an overly reductive generalization of the langauges' approach to software correctness, "there's a rather small difference between Rust and Zig" simply isn't true.

pron 11mo ago

> Limiting to the "top 5" vulnerabilities by number of CVEs feels like cherry-picking. The point isn't about limiting to the top 5. The point is that once you get to the things Rust prevents and Zig doesn't, there are quite a few more things that neither prevents, so it's just silly to draw a a particular sharp line between Rust and Zig because they perform exactly the same (in terms of sound guarantees; we're ignoring any softer effects) for most top weaknesses. Even if you think that difference is so important that it justifies downsides that Rust may have in comparison, you still have to admit that Zig is much, much closer to Rust than to C by that measure. And this "closer" matters because Rust's memory safety is also not absolute, and Rust proponents must accept that the cost of memory safety is an important factor, too, and sometimes not worth it, or else Rust wouldn't have been invented in the first place. After all, languages that are as memory-safe as Rust and more were more popular than Rust will ever be before it was even invented. So Rust proponents must accept that eliminating dangerous vulnerabilities is good (unlike C), that productivity and cost do matter (unlike ATS), and that non-absolute memory safety is acceptable. And Zig satisfies all of these points, too. The reason it's hard to find an objective metric to draw the line between Rust and Zig is because they're actually quite close to each other, at least on this front of trying to find a useful compromise between productivity and guarantees. > Lastly, filtering on CVEs has a high selection bias: just because security researchers go for the easy vulnerabilities first doesn't mean that the harder ones can be ignored. Sure, but then you might as well also consider softer effects. For example, maybe a language that's easier to review because it's more explicit, or a language that's faster to compile and is easier to test wins. And I agree that we should consider all these, but then we start seeing why correctness is such a complicated topic, and we could speculate just as easily that it is Zig that "clearly" wins. Anyway, it's perfectly fine for people to prefer Rust because they like it. But the attempt to find objective reasons for this preference is not based on any truly objective foundations, and just looks like some desperate rationalisation. And BTW, > relevant to applications that would typically be written in C/C++/Rust/Zig If you think that Rust and Zig are designed to target the exact same domains, then some of the "softer" aspects I mentioned could play even a larger role. I mean, the portion of software written in low level languages has been declining steadily for a long time with no sign of a change in the trend. To me it seems that Zig has internalised the narrower and more focused and role of low-level languages today compared to what C++ imagined it would be in the eighties.

NobodyNada 11mo ago
> it's just silly to draw a a particular sharp line between Rust and Zig because they perform exactly the same (in terms of sound guarantees; we're ignoring any softer effects) for most top weaknesses. There is a very clear sharp line between them in that Rust has no undefined behavior outside of an unsafe block. This matters because the effects of undefined behavior -- particularly memory-corrupting undefined behavior -- are unbounded. Security issues caused by logic bugs are limited to only having local effects on the program -- a SQL injection bug can lead to the wrong SQL query being executed, a path traversal bug can lead to the wrong path being accessed, and a shell escaping bug can lead to execution of the wrong shell command. These are severe problems that can lead to data exfiltration or remote code execution, but the relationship between bug and effect is straightforward: "I'm passing user input to a shell here; this could have catastrophic consequences so I'd better review this carefully." In contrast, undefined behavior can happen anywhere in your program, and it can do anything. That is a clear and measurable difference, and emperical evidence from the last 10 years clearly indicates both that it is nigh impossible to write large C/C++ applications without both spatial and temporal memory safety vulnerabilities, and that adopting Rust measurably decreases the incidence of such vulnerabilities. Zig is certainly an improvement over C on this front, but it is still a UB-heavy language, and that's where the sharp line is. It's hard to make emperical comparisons because Zig is so young, but the comparision between Deno and Bun in the article is a reasonably strong demonstration that Zig has not achieved a comparable level of memory safety to Rust. > Sure, but then you might as well also consider softer effects. For example, maybe a language that's easier to review because it's more explicit, or a language that's faster to compile and is easier to test wins. > And I agree that we should consider all these, but then we start seeing why correctness is such a complicated topic, and we could speculate just as easily that it is Zig that "clearly" wins. This doesn't really have anything to do with my point that the CVE list does not provide evidence to dismiss temporal memory safety as irrelevant; it's more of a general statement on writing correct software. But regardless, "Zig has nullability and bounds checks" and "Rust has an affine type system, statically-checked immutability and exclusivity, language-level resource management, and no undefined behavior" are not in the same league of program correctness. Rust wasn't designed after some ideal of memory safety at the expense of clarity and correctness: the lifetime and type system came from a goal of reducing logic bugs in complex concurrent programs, and the fact that it is powerful enough to achieve memory safety without garbage collection was a happy accident. The emperical results that Rust programs have fewer memory-safety vulnerabilities demonstrate that Rust's static approach to software correctness is successful, and not just for the specific problem of memory safety, because Rust's tooling for achieving program correctness is generalizable to arbitrary application invariants. This lines up with my own experience; software I write using Rust is far easier to get right, more reliable, and easier to successfully maintain and refactor than anything else I've done. Certainly Zig has its strong points as well -- explicitness can reduce complexity and compile times, but it also has downsides of pushing complexity into application code (thus making it harder to review and introducing more opportunities to create mistakes). A proper comparison of the two approaches is worthwhile, but just as "Rust is more memory safe" is an overly reductive generalization of the langauges' approach to software correctness, "there's a rather small difference between Rust and Zig" simply isn't true.