5 ms·

I understand why people want to rewrite C libraries and such in Rust. Software like glibc is battle-tested---it is _widely_ used on hundreds of thousands of sy

I understand why people want to rewrite C libraries and such in Rust. Software like glibc is battle-tested---it is _widely_ used on hundreds of thousands of systems around the world, and has been used (though not at today's massive scale) for decades. I understand that glibc is under active development and there is a lot of new code, but let's keep this in perspective: Writing a new implementation of a system is a huge opportunity to introduce bugs. There is focus on these specific problems, but in the broader scheme of things, glibc is remarkably stable, performant, and feature-rich. A new implementation will have bugs, and those bugs might be less likely to be caught simply because the system will not be as widely used for quite a long time. Even formally proven systems don't address flaws in the actual program specification. (See "The Limits of Correctness" by Brian Cantwell Smith for a good discussion). Also relevant (which I'm reminded of in part because of his recent death): Peter Naur's Programming as Theory Building. So even _new_ code to glibc has the benefit of a huge community of both developers are users to eyeball it and test it out in production on a huge number of systems. So rewriting glibc may solve certain problems, but it's bound to create a whole lot more, considering the narrow range of issues that are being focused on. New Rust code will have undiscovered issues too, even if they're not memory or stack related. I feel that this effort might be better spent fixing and finding problems with glibc---and continuing the development of tools to find those problems, to benefit _all_ of our old C libraries and programs---than rewriting for the sake of rewriting.

pcwalton 11y ago
> New Rust code will have undiscovered issues too, even if they're not memory or stack related. Memory issues have a tendency to be the most severe types of issues from a security perspective. (Not always, but with a high frequency.) Other issues that may occur with a rewrite may be security problems as well, but the security track records of network-facing software written in old-timey C speak for themselves: memory safety issues make up a huge fraction of their vulnerabilities. > I feel that this effort might be better spent fixing and finding problems with glibc---and continuing the development of tools to find those problems, to benefit _all_ of our old C libraries and programs---than rewriting for the sake of rewriting. So we've been trying that for the past 35 years since C was released in 1978, and we've completely failed to move beyond making the same basic memory management errors again and again and again. I think that if we couldn't do it in 35 years, we aren't likely to be able to by just trying a little harder this one time.
- gpderetta 11y ago
  One problem with rewriting libc in a safer language is that interfaces will still be memory unsafe. No matter how many times you rewrite 'gets' it will still be broken, str* will still be problematic, the heap can be still trivially corrupted by the application, etc. As long as the public interfaces are still unsafe, it doesn't seem to me that there is significant latitude to qualitatively improve the security of a libc implementation.
  - pcwalton 11y ago
    > As long as the public interfaces are still unsafe, it doesn't seem to me that there is significant latitude to qualitatively improve the security of a libc implementation. Of course there is. The glibc bug we're talking about would not have happened if libresolv were memory safe.
    - gpderetta 11y ago
      I was too negative in my last message. To be more precise: I claim that, for a library for an unsafe interface, if the ratio of library interface area over the library implementation volume is high enough, it is not worth re-implementing in a safe language. I also claim that a libc implementation is generally well above this threshold. I agree that this might not be the case for selected subsets of the library like libresolv.
      - pcwalton 11y ago
        We can resolve this empirically, by looking at the list of glibc CVEs: https://www.cvedetails.com/vulnerability-list/vendor_id-72/product_id-767/GNU-Glibc.html https://www.cvedetails.com/vulnerability-list/vendor_id-72/p... The vast majority of the memory safety issues here are not at the public interface boundaries.
wvenable 11y ago
> Software like glibc is battle-tested This seems a bit counter-intuitive given that serious problems are still being found years or even decades later. Perhaps battle-tested is insufficient. And while new code is likely to have bugs, assuming it's already based on the original C code it would be less than if it was re-written from scratch based on a spec. New code can take advantage of the battle-tested nature of the original code even if it's in a different language.
- mikegerwitz 11y ago
  > This seems a bit counter-intuitive given that serious problems are still being found years or even decades later. Perhaps battle-tested is insufficient. Bugs will always exist. But that statement will apply equally to any software, regardless of language. > New code can take advantage of the battle-tested nature of the original code even if it's in a different language. That's why I referenced the Peter Naur paper on Programming as Theory Building---in practice, that may very well not be the case.
  - dikaiosune 11y ago
    > Bugs will always exist. But that statement will apply equally to any software, regardless of language. I'd be curious to see some research suggesting that the prevalence and severity of those bugs is identical regardless of development tool/language/runtime, but I'd be surprised if that's demonstrable. The question at hand is not "will safer languages eliminate all bugs?" It's "will safer languages reduce the prevalence and severity of important classes of bugs?" I'd wager it's probably yes, but even if you disagree, I don't think that it's reasonable to suggest that because there will always be bugs we should never improve.
    - mikegerwitz 11y ago
      > I'd be curious to see some research suggesting that the prevalence and severity of those bugs is identical regardless of development tool/language/runtime, but I'd be surprised if that's demonstrable. I intended to convey that software written in any language will have bugs, not that all languages will produce the same types of bugs.
gsanghera 11y ago
Battle tested in the past != fit for future wars. The nature of systems has evolved. Attackers are more sophisticated, have better tools and probably understand C code better than most who still write it. So our tools need to evolve too. Admittedly rewriting full libs into another language sounds scary, but hey - since when did fear of the future stop it from coming?
- mikegerwitz 11y ago
  I'm unconvinced by the argument that C is unfit for future wars.
  - pcwalton 11y ago
    It's simple: 1. Memory safety issues are the cause of a very large number of security vulnerabilities (often most of them for projects written in C or C++, depending on the software). 2. Memory safety-related issues have a relatively high probability of being turned into remote code execution, which is one of the most if not the most severe outcomes. 3. C and C++ projects have been empirically observed to have orders of magnitude more memory safety problems than projects written in other languages do. 4. The additional classes of security vulnerabilities that managed languages tend to foster do not have the combined prevalence and severity of memory safety problems. So, we would be better served in security by moving to memory-safe languages.
    - falcolas 11y ago
      Memory safe programs require 1) Runtime overhead for some form of GC (D, Lisp, etc) 2) Rephrasing a program to satisfy a memory constraint checker (Rust) 3) Disciplined memory usage (i.e. Nasa C coding guidelines) We don't have enough experience with 2 to indicate whether it will create new classes of bugs. We also don't understand the knock-on effect of managing memory differently - will functionally identical programs require more or fewer resources, more or fewer programmers hours, etc. Rust may very well be the future, but we don't know for sure yet. One thing we do know: options 1 and 3 have been available for years, but not widely utilized. What lessons can we learn from this fact to apply to Rust?
steveklabnik 11y ago
> Software like glibc is battle-tested "The best time to plant a tree is 20 years ago; the second best time is now."

pcwalton 11y ago

> New Rust code will have undiscovered issues too, even if they're not memory or stack related. Memory issues have a tendency to be the most severe types of issues from a security perspective. (Not always, but with a high frequency.) Other issues that may occur with a rewrite may be security problems as well, but the security track records of network-facing software written in old-timey C speak for themselves: memory safety issues make up a huge fraction of their vulnerabilities. > I feel that this effort might be better spent fixing and finding problems with glibc---and continuing the development of tools to find those problems, to benefit _all_ of our old C libraries and programs---than rewriting for the sake of rewriting. So we've been trying that for the past 35 years since C was released in 1978, and we've completely failed to move beyond making the same basic memory management errors again and again and again. I think that if we couldn't do it in 35 years, we aren't likely to be able to by just trying a little harder this one time.

gpderetta 11y ago
One problem with rewriting libc in a safer language is that interfaces will still be memory unsafe. No matter how many times you rewrite 'gets' it will still be broken, str* will still be problematic, the heap can be still trivially corrupted by the application, etc. As long as the public interfaces are still unsafe, it doesn't seem to me that there is significant latitude to qualitatively improve the security of a libc implementation.
- pcwalton 11y ago
  > As long as the public interfaces are still unsafe, it doesn't seem to me that there is significant latitude to qualitatively improve the security of a libc implementation. Of course there is. The glibc bug we're talking about would not have happened if libresolv were memory safe.
  - gpderetta 11y ago
    I was too negative in my last message. To be more precise: I claim that, for a library for an unsafe interface, if the ratio of library interface area over the library implementation volume is high enough, it is not worth re-implementing in a safe language. I also claim that a libc implementation is generally well above this threshold. I agree that this might not be the case for selected subsets of the library like libresolv.
    - pcwalton 11y ago
      We can resolve this empirically, by looking at the list of glibc CVEs: https://www.cvedetails.com/vulnerability-list/vendor_id-72/product_id-767/GNU-Glibc.html https://www.cvedetails.com/vulnerability-list/vendor_id-72/p... The vast majority of the memory safety issues here are not at the public interface boundaries.

wvenable 11y ago

> Software like glibc is battle-tested This seems a bit counter-intuitive given that serious problems are still being found years or even decades later. Perhaps battle-tested is insufficient. And while new code is likely to have bugs, assuming it's already based on the original C code it would be less than if it was re-written from scratch based on a spec. New code can take advantage of the battle-tested nature of the original code even if it's in a different language.

mikegerwitz 11y ago
> This seems a bit counter-intuitive given that serious problems are still being found years or even decades later. Perhaps battle-tested is insufficient. Bugs will always exist. But that statement will apply equally to any software, regardless of language. > New code can take advantage of the battle-tested nature of the original code even if it's in a different language. That's why I referenced the Peter Naur paper on Programming as Theory Building---in practice, that may very well not be the case.
- dikaiosune 11y ago
  > Bugs will always exist. But that statement will apply equally to any software, regardless of language. I'd be curious to see some research suggesting that the prevalence and severity of those bugs is identical regardless of development tool/language/runtime, but I'd be surprised if that's demonstrable. The question at hand is not "will safer languages eliminate all bugs?" It's "will safer languages reduce the prevalence and severity of important classes of bugs?" I'd wager it's probably yes, but even if you disagree, I don't think that it's reasonable to suggest that because there will always be bugs we should never improve.
  - mikegerwitz 11y ago
    > I'd be curious to see some research suggesting that the prevalence and severity of those bugs is identical regardless of development tool/language/runtime, but I'd be surprised if that's demonstrable. I intended to convey that software written in any language will have bugs, not that all languages will produce the same types of bugs.

gsanghera 11y ago

Battle tested in the past != fit for future wars. The nature of systems has evolved. Attackers are more sophisticated, have better tools and probably understand C code better than most who still write it. So our tools need to evolve too. Admittedly rewriting full libs into another language sounds scary, but hey - since when did fear of the future stop it from coming?

mikegerwitz 11y ago
I'm unconvinced by the argument that C is unfit for future wars.
- pcwalton 11y ago
  It's simple: 1. Memory safety issues are the cause of a very large number of security vulnerabilities (often most of them for projects written in C or C++, depending on the software). 2. Memory safety-related issues have a relatively high probability of being turned into remote code execution, which is one of the most if not the most severe outcomes. 3. C and C++ projects have been empirically observed to have orders of magnitude more memory safety problems than projects written in other languages do. 4. The additional classes of security vulnerabilities that managed languages tend to foster do not have the combined prevalence and severity of memory safety problems. So, we would be better served in security by moving to memory-safe languages.
  - falcolas 11y ago
    Memory safe programs require 1) Runtime overhead for some form of GC (D, Lisp, etc) 2) Rephrasing a program to satisfy a memory constraint checker (Rust) 3) Disciplined memory usage (i.e. Nasa C coding guidelines) We don't have enough experience with 2 to indicate whether it will create new classes of bugs. We also don't understand the knock-on effect of managing memory differently - will functionally identical programs require more or fewer resources, more or fewer programmers hours, etc. Rust may very well be the future, but we don't know for sure yet. One thing we do know: options 1 and 3 have been available for years, but not widely utilized. What lessons can we learn from this fact to apply to Rust?

steveklabnik 11y ago

> Software like glibc is battle-tested "The best time to plant a tree is 20 years ago; the second best time is now."