7 ms·

> cannot help but feel like the strong push for rust in the kernel is overstepping some sort of boundary There's no "strong" push; the RIIR narrative doesn't a

> cannot help but feel like the strong push for rust in the kernel is overstepping some sort of boundary There's no "strong" push; the RIIR narrative doesn't actually belong to the Rust community (which I've found very pragramatic), but it's so inherently inflammatory that the general public associated it to the Rust community nonetheless. What's actually happening is that some major projects are evaluating - slowly - whether Rust could be integrated in some areas. Besides the kernel, another high-profile project is QEMU. > Why, if C is so bad, wasnt one of the alternatives introduced to slowly replace C in the kernel years ago? Zig, D, Ada, ... they all offer massive benefits over C. Because of the "well-known value proposition" that Rust brings. I'm continuously mystified, now that unsafe memory management has been unambiguously shown to be the major factor in vulnerabilities, how safe memory management is still not recognized as the highest priority (which doesn't imply to rewrite in Rust/Golang/etc) by everybody.

WJW 5y ago
> I'm continuously mystified, now that unsafe memory management has been unambiguously shown to be the major factor in vulnerabilities, how safe memory management is still not recognized as the highest priority It is very straightforward to recognize the reason for this as soon as you realize that vulnerabilities themselves almost never have the highest priority either. Therefore, preventing them also can't have the highest priority.
lionkor 5y ago
I entirely agree that safe memory management is one of the most important topics today. I just wonder if our time would be better spent developing tools to formally verify memory safety of existing languages, instead of trying to "rewrite everything in Rust" to get safety. I see your point, though, and I was definitely being snarky for the sake of being snarky.
- bawolff 5y ago
  People have been trying the formally verify things route for way longer than rust has been a thing.
  - johnisgood 5y ago
    People really ought to read about Ada/SPARK before jumping on the Rust bandwagon.
- Yoric 5y ago
  > I just wonder if our time would be better spent developing tools to formally verify memory safety of existing languages, instead of trying to "rewrite everything in Rust" to get safety. In theory, that makes sense. Now, as a sibling post points out, such tools have been around for dozens of years. Microsoft started using these tools for drivers around the time of Windows XP, if I recall correctly. I've used such tools. Heck, I've developed such tools. In my experience, they require orders of magnitude more experience (both of the tools, of the underlying theory and of the codebase) than Rust (or Ada, or other safe languages) to achieve comparable levels of safety.
- kaba0 5y ago
  But metadata is needed for proper formal reasoning, and C has barely any additional information content over some very vague “typing” hints, everything else being only conventions enforced more or less by the linux community. So Rust’s lifecycle annotations are what makes proper proofs possible in the first place. (As for proofs though, they are no silver bullets either — a program of linux’s complexity is unlikely to be ever completely verified, but rust may indeed give valuable guarantees for most of the program)
  - pjmlp 5y ago
    On Windows we have SAL for that metadata. https://docs.microsoft.com/en-us/cpp/c-runtime-library/sal-annotations?view=msvc-160 https://docs.microsoft.com/en-us/cpp/c-runtime-library/sal-a... Naturally it doesn't help for portable code, nor if the people don't make use of it, or the static analysers on the code.
- jasode 5y ago
  >I just wonder if our time would be better spent developing tools to formally verify memory safety of existing languages, The industry can't implement your suggestion because while computer science can formally verify a restricted subset of some C source code, we can't write a generalized tool to scan any C code to prove memory safety. Consider memory operations of existing language such as C Language with these statements: char *dst; // value is set anywhere in the program char *src; // value is set anywhere in the program size_t sz; // value is set anywhere in the program // ... bunch of other code that may alter dst/src/sz ... memcpy(dst, src, sz); // is that memory safe? Nobody has yet written analysis tools to prove & verify that the memcpy() is "safe" because predicting with perfect accuracy how variables dst, src, sz will get set & change during runtime are variations of the unsolvable Halting Problem: https://en.wikipedia.org/wiki/Halting_problem https://en.wikipedia.org/wiki/Halting_problem That's why there's still no computer science research paper showing a tool that can scan general C source code to prove that all calls to memcpy(), strcpy(), and any writes to RAM via aliases & pointer dereferencing are "memory safe". We're hoping for "Sufficiently Smart Compiler"[1] type of tool that can verify C code is safe but the source itself doesn't have enough information embedded in it to prove that. So the alternatives so far have been: - static approach: extra syntax for manual annotation (C++ unique_ptr<>, or Rust borrow &ampersand, or manual mutex lock/unlock) - dynamic approach: runtime tests with extra memory checks such as Valgrind + UBSAN + ASAN + debugger memory fences, etc. This can catch some but not all memory errors. 100% provable is not possible for non-trivial programs because of the combinatorial explosion of runtime execution paths and memory values. - GC garbage collection in a different "managed" language. The tradeoff is higher RAM footprint, extra cpu and GC pauses for tracing scanners. [1] http://wiki.c2.com/?SufficientlySmartCompiler http://wiki.c2.com/?SufficientlySmartCompiler
  - kaba0 5y ago
    > GC garbage collection in a different "managed" language. The tradeoff is higher RAM footprint, extra cpu and GC pauses for tracing scanners It would hardly mean a problem for most programs, including OS kernels, especially that kernels are themselves GCs for resources like processes, file handlers, etc. Also, GC pauses are overblown, most of the tasks can be done in parallel.
- pjmlp 5y ago
  It was obvious from the yearly days that C was a sharp knife, that is why lint was created in 1979. How many people bother to actually use such tooling on their C projects? Not many, most surveys place that number around 11%. Just as an exercise, Github offers such tooling for C and C++ projects, try to find how many have CodeQL workflows enabled for pull requests.
  - bregma 5y ago
    I work with safety-certified systems written in C. The tooling around such things is extensive and mature: static analysis, extensive test runs with runtime analysis instrumentation (test coverage at the MC/DC level, valgrind for memory and concurrency analysis) it's all there and used. It's true that a lot of projects written in C do the equivalent of using Rust's "unsafe". Then again, a lot of Rust projects use Rust's "unsafe".
    - pjmlp 5y ago
      Welcome to the 11% mentioned on my comment. JOVIAL and ESPOOL were the first system languages with a notion of unsafe code blocks, both about 10 years older than C. It is a big difference to have unsafe code jump to our eye and being easily searchable, or having each line of code as a possible source of integer overflows, memory corruption, or UB optimization exploit of the day.
2pEXgD0fZ5cF 5y ago
> the RIIR narrative doesn't actually belong to the Rust community As someone who has been the target of aggressive "rewrite it in rust" comments and "pressure" (multiple issues and useless discussions) on one of my projects that probably completely depends on how you define "the Rust community". The highly annoying RIIR crowd certainly does stem from Rust users. As a Rust user myself I'd like to distance myself from them too, but that doesn't really change that they do come from the Rust community.
- lionkor 5y ago
  To be fair, elitists exist in every language. This is not a fault of Rust, but annoying either way.
  - 2pEXgD0fZ5cF 5y ago
    True
- brabel 5y ago
  It annoys me a lot when people try to describe a community, specially a completely open one as programming language communities where literally anyone can come an go at any time, as having certain characteristics. It's similar to when you try to generalize over nationalities. While there may be some small truth in it over very large numbers, there's always going to be plenty of exceptions so that making such generalizations should ALWAYS be considered inappropriate, be it "negative" or not. Even more so when there are large minorities within the community. I do agree that the RIIR is likely to be one of these minorities, but that doesn't make them NOT part of the Rust community. Another "minority" in the Rust community that I have noticed a lot in my projects/blog posts is made up of those who are never satisfied with any Rust code: there's always some better way to write something, or some more efficient implementation, and your code definitely sucks... Rust is a very large language with many features, so there's always a lot of ways to do things, but even two extremely highly skilled Rust developers are highly likely to disagree on which way is "the best" for a certain non-trivial design decision... the fear of not using the best alternative and being ridiculed by other Rust developers claiming my approach is bad makes me quite cautious about showing Rust code in any blog post I write. But I still don't want to claim this is a characteristic of "the Rust community", just like I would hesitate to say "the Rust community is very welcoming". We probably just need to stop the human impulse to generalize about groups of people, that's all.
nivenkos 5y ago
> now that unsafe memory management has been unambiguously shown to be the major factor in vulnerabilities, how safe memory management is still not recognized as the highest priority It is a shame it isn't taken more seriously, especially when you consider the recent Pegasus discoveries - https://digitalviolence.org/ https://digitalviolence.org/ A memory management error can literally be a matter of life and death for some.
- pjmlp 5y ago
  Therac-25.
  - bregma 5y ago
    The Therac-25 tragedy was not a memory-management issue. If its had been written in Rust the state machine error would still have cooked a lot of people.
    - mratsim 5y ago
      Indeed, if we wanted to prevent it we would use ADA/Spark or a language with formal verification support say Frama-C.
coopierez 5y ago
> What's actually happening is that some major projects are evaluating - slowly - whether Rust could be integrated in some areas. Related to your comment: the writer of curl was investigating Rust. [https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyper/ https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyp...] The first comment on that article complains that Daniel is being pressured by the Rust community, and he responds: > “They” don’t change anything here – I do. This is a project in which I’m working on a change as described in this post. It’s not a conspiracy.
- pizza234 5y ago
  > The first comment on that article complains that Daniel is being pressured by the Rust community, and he responds: The first comment actually proves, in a typical way, the point of the RIIR misinformation. Here it is, in full: > This sounds like the usual EEE playbook. Some powerful party wants to take away your project. The new slightly incompatible curl will then be the new “curl”, and the current curl will be dead. We have seen it so many times. > Please Daniel stick to C-based curl, and don’t align with such parties. They should rename their work, and we will see if their rusted thingy gets any traction without using the well known and trusted “curl” brand. The author is not a Rust programmer, and they're thinking on behalf of Curl's author, pushing a pseudo-conspiracy theory narrative ("some powerful party wants to take away your project"). Who's this powerful "party"? The Rust Illuminati? /s Depending on the interpretation, the Curl author's answer could be actually a confirmation of the nonsense of the comment ("It’s not a conspiracy"). The author has actually been outspoken on the nutty people putting pressure in a way or another. Here's a fun post: https://daniel.haxx.se/blog/2021/05/20/i-could-rewrite-curl https://daniel.haxx.se/blog/2021/05/20/i-could-rewrite-curl There's no trace of Rust programmers, and even if they were, they'd be simply nuts as all the people mentioned in the post.

WJW 5y ago

> I'm continuously mystified, now that unsafe memory management has been unambiguously shown to be the major factor in vulnerabilities, how safe memory management is still not recognized as the highest priority It is very straightforward to recognize the reason for this as soon as you realize that vulnerabilities themselves almost never have the highest priority either. Therefore, preventing them also can't have the highest priority.

lionkor 5y ago

I entirely agree that safe memory management is one of the most important topics today. I just wonder if our time would be better spent developing tools to formally verify memory safety of existing languages, instead of trying to "rewrite everything in Rust" to get safety. I see your point, though, and I was definitely being snarky for the sake of being snarky.

bawolff 5y ago
People have been trying the formally verify things route for way longer than rust has been a thing.
- johnisgood 5y ago
  People really ought to read about Ada/SPARK before jumping on the Rust bandwagon.
Yoric 5y ago
> I just wonder if our time would be better spent developing tools to formally verify memory safety of existing languages, instead of trying to "rewrite everything in Rust" to get safety. In theory, that makes sense. Now, as a sibling post points out, such tools have been around for dozens of years. Microsoft started using these tools for drivers around the time of Windows XP, if I recall correctly. I've used such tools. Heck, I've developed such tools. In my experience, they require orders of magnitude more experience (both of the tools, of the underlying theory and of the codebase) than Rust (or Ada, or other safe languages) to achieve comparable levels of safety.
kaba0 5y ago
But metadata is needed for proper formal reasoning, and C has barely any additional information content over some very vague “typing” hints, everything else being only conventions enforced more or less by the linux community. So Rust’s lifecycle annotations are what makes proper proofs possible in the first place. (As for proofs though, they are no silver bullets either — a program of linux’s complexity is unlikely to be ever completely verified, but rust may indeed give valuable guarantees for most of the program)
- pjmlp 5y ago
  On Windows we have SAL for that metadata. https://docs.microsoft.com/en-us/cpp/c-runtime-library/sal-annotations?view=msvc-160 https://docs.microsoft.com/en-us/cpp/c-runtime-library/sal-a... Naturally it doesn't help for portable code, nor if the people don't make use of it, or the static analysers on the code.
jasode 5y ago
>I just wonder if our time would be better spent developing tools to formally verify memory safety of existing languages, The industry can't implement your suggestion because while computer science can formally verify a restricted subset of some C source code, we can't write a generalized tool to scan any C code to prove memory safety. Consider memory operations of existing language such as C Language with these statements: char *dst; // value is set anywhere in the program char *src; // value is set anywhere in the program size_t sz; // value is set anywhere in the program // ... bunch of other code that may alter dst/src/sz ... memcpy(dst, src, sz); // is that memory safe? Nobody has yet written analysis tools to prove & verify that the memcpy() is "safe" because predicting with perfect accuracy how variables dst, src, sz will get set & change during runtime are variations of the unsolvable Halting Problem: https://en.wikipedia.org/wiki/Halting_problem https://en.wikipedia.org/wiki/Halting_problem That's why there's still no computer science research paper showing a tool that can scan general C source code to prove that all calls to memcpy(), strcpy(), and any writes to RAM via aliases & pointer dereferencing are "memory safe". We're hoping for "Sufficiently Smart Compiler"[1] type of tool that can verify C code is safe but the source itself doesn't have enough information embedded in it to prove that. So the alternatives so far have been: - static approach: extra syntax for manual annotation (C++ unique_ptr<>, or Rust borrow &ampersand, or manual mutex lock/unlock) - dynamic approach: runtime tests with extra memory checks such as Valgrind + UBSAN + ASAN + debugger memory fences, etc. This can catch some but not all memory errors. 100% provable is not possible for non-trivial programs because of the combinatorial explosion of runtime execution paths and memory values. - GC garbage collection in a different "managed" language. The tradeoff is higher RAM footprint, extra cpu and GC pauses for tracing scanners. [1] http://wiki.c2.com/?SufficientlySmartCompiler http://wiki.c2.com/?SufficientlySmartCompiler
- kaba0 5y ago
  > GC garbage collection in a different "managed" language. The tradeoff is higher RAM footprint, extra cpu and GC pauses for tracing scanners It would hardly mean a problem for most programs, including OS kernels, especially that kernels are themselves GCs for resources like processes, file handlers, etc. Also, GC pauses are overblown, most of the tasks can be done in parallel.
pjmlp 5y ago
It was obvious from the yearly days that C was a sharp knife, that is why lint was created in 1979. How many people bother to actually use such tooling on their C projects? Not many, most surveys place that number around 11%. Just as an exercise, Github offers such tooling for C and C++ projects, try to find how many have CodeQL workflows enabled for pull requests.
- bregma 5y ago
  I work with safety-certified systems written in C. The tooling around such things is extensive and mature: static analysis, extensive test runs with runtime analysis instrumentation (test coverage at the MC/DC level, valgrind for memory and concurrency analysis) it's all there and used. It's true that a lot of projects written in C do the equivalent of using Rust's "unsafe". Then again, a lot of Rust projects use Rust's "unsafe".
  - pjmlp 5y ago
    Welcome to the 11% mentioned on my comment. JOVIAL and ESPOOL were the first system languages with a notion of unsafe code blocks, both about 10 years older than C. It is a big difference to have unsafe code jump to our eye and being easily searchable, or having each line of code as a possible source of integer overflows, memory corruption, or UB optimization exploit of the day.

2pEXgD0fZ5cF 5y ago

> the RIIR narrative doesn't actually belong to the Rust community As someone who has been the target of aggressive "rewrite it in rust" comments and "pressure" (multiple issues and useless discussions) on one of my projects that probably completely depends on how you define "the Rust community". The highly annoying RIIR crowd certainly does stem from Rust users. As a Rust user myself I'd like to distance myself from them too, but that doesn't really change that they do come from the Rust community.

lionkor 5y ago
To be fair, elitists exist in every language. This is not a fault of Rust, but annoying either way.
- 2pEXgD0fZ5cF 5y ago
  True
brabel 5y ago
It annoys me a lot when people try to describe a community, specially a completely open one as programming language communities where literally anyone can come an go at any time, as having certain characteristics. It's similar to when you try to generalize over nationalities. While there may be some small truth in it over very large numbers, there's always going to be plenty of exceptions so that making such generalizations should ALWAYS be considered inappropriate, be it "negative" or not. Even more so when there are large minorities within the community. I do agree that the RIIR is likely to be one of these minorities, but that doesn't make them NOT part of the Rust community. Another "minority" in the Rust community that I have noticed a lot in my projects/blog posts is made up of those who are never satisfied with any Rust code: there's always some better way to write something, or some more efficient implementation, and your code definitely sucks... Rust is a very large language with many features, so there's always a lot of ways to do things, but even two extremely highly skilled Rust developers are highly likely to disagree on which way is "the best" for a certain non-trivial design decision... the fear of not using the best alternative and being ridiculed by other Rust developers claiming my approach is bad makes me quite cautious about showing Rust code in any blog post I write. But I still don't want to claim this is a characteristic of "the Rust community", just like I would hesitate to say "the Rust community is very welcoming". We probably just need to stop the human impulse to generalize about groups of people, that's all.

nivenkos 5y ago

> now that unsafe memory management has been unambiguously shown to be the major factor in vulnerabilities, how safe memory management is still not recognized as the highest priority It is a shame it isn't taken more seriously, especially when you consider the recent Pegasus discoveries - https://digitalviolence.org/ https://digitalviolence.org/ A memory management error can literally be a matter of life and death for some.

pjmlp 5y ago
Therac-25.
- bregma 5y ago
  The Therac-25 tragedy was not a memory-management issue. If its had been written in Rust the state machine error would still have cooked a lot of people.
  - mratsim 5y ago
    Indeed, if we wanted to prevent it we would use ADA/Spark or a language with formal verification support say Frama-C.

coopierez 5y ago

> What's actually happening is that some major projects are evaluating - slowly - whether Rust could be integrated in some areas. Related to your comment: the writer of curl was investigating Rust. [https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyper/ https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyp...] The first comment on that article complains that Daniel is being pressured by the Rust community, and he responds: > “They” don’t change anything here – I do. This is a project in which I’m working on a change as described in this post. It’s not a conspiracy.

pizza234 5y ago
> The first comment on that article complains that Daniel is being pressured by the Rust community, and he responds: The first comment actually proves, in a typical way, the point of the RIIR misinformation. Here it is, in full: > This sounds like the usual EEE playbook. Some powerful party wants to take away your project. The new slightly incompatible curl will then be the new “curl”, and the current curl will be dead. We have seen it so many times. > Please Daniel stick to C-based curl, and don’t align with such parties. They should rename their work, and we will see if their rusted thingy gets any traction without using the well known and trusted “curl” brand. The author is not a Rust programmer, and they're thinking on behalf of Curl's author, pushing a pseudo-conspiracy theory narrative ("some powerful party wants to take away your project"). Who's this powerful "party"? The Rust Illuminati? /s Depending on the interpretation, the Curl author's answer could be actually a confirmation of the nonsense of the comment ("It’s not a conspiracy"). The author has actually been outspoken on the nutty people putting pressure in a way or another. Here's a fun post: https://daniel.haxx.se/blog/2021/05/20/i-could-rewrite-curl https://daniel.haxx.se/blog/2021/05/20/i-could-rewrite-curl There's no trace of Rust programmers, and even if they were, they'd be simply nuts as all the people mentioned in the post.