5 ms·

I think that since you're making concrete claims about actual cost and benefit, perhaps it's appropriate to suggest that you cite some sources (postmortems, whi

I think that since you're making concrete claims about actual cost and benefit, perhaps it's appropriate to suggest that you cite some sources (postmortems, white papers, journal articles) which have found these assertions to be true?

pron 10y ago
You're suggesting that rewriting billions of lines of code that probably contain lots of bugs somewhere every time a new safer language comes along is the most sensible investment of effort by the software industry in order to improve the quality of our software infrastructure, while I'm saying, wait, maybe we should use some inspection tools first before we decide on an approach, so at least we know where the worst of the problem is, and you think I'm the one who needs to cite sources? Alternatively, do you think that "rewrite all legacy code!" (an effort that would probably cost billions of dollars) does not need to be justified by concrete claims about actual cost and benefit? But fine, here's a taste (look for what is required to use the tool as well as results): * http://mir.cs.illinois.edu/marinov/publications/SenETAL05CUTE.pdf http://mir.cs.illinois.edu/marinov/publications/SenETAL05CUT... * http://lcamtuf.coredump.cx/afl/ http://lcamtuf.coredump.cx/afl/ (see the "The bug-o-rama trophy case") * http://research.microsoft.com/pubs/144848/2011%20CAV%20SLAyer_%20Memory%20Safety%20for%20Systems-level%20Code.pdf http://research.microsoft.com/pubs/144848/2011%20CAV%20SLAye... * https://code.facebook.com/posts/1648953042007882 https://code.facebook.com/posts/1648953042007882 * http://link.springer.com/chapter/10.1007/978-3-319-17524-9_1 http://link.springer.com/chapter/10.1007/978-3-319-17524-9_1 Note that there are some strengths, some weaknesses, but I hope those sources make it clear that using those tools first is at least more sensible than "rewriting everything", as they require orders of magnitude less effort, and they do uncover bugs (you're free to guess what percentage of those bugs would be found in a rewrite, and whether the bug density demands a rewrite). We may decide that a rewrite of some libraries is the best approach once we have more information. Now, do you have any sources suggesting that a complete rewrite is cost-effective?
- kbenson 10y ago
  The biggest problem I (not being the people you've been conversing with so far, for the most part) have with relying on static analysis tools on top of C, instead of a rewrite, is that the analysis tools are not enforceable. You can do all the analysis you want, and put in all the procedures you want, but in the end people are submitting changes to these libraries, and you need to make sure that these people are ensuring the tools get run prior to release. Additionally, from the perspective of a user of this code, my ability to look at the source and know that it's been scanned ranges from uncertain to non-existent. Switching to a compilation system where these constraints are enforced (such as a new language that enforces it) has a benefit in that respect. You could get a similar benefit by releasing a specialized C compiler that enforced some static analysis tools and required a very small amount of syntax change such that the code would not compile on a vanilla C compiler, and then I think the there would be much less of a case for a rewrite in Rust from a safety point of view (even if they would handle slightly different aspects of safety, as you've pointed out). That said, I'm not sure it makes sense to rewrite "everything" in Rust, but it would be nice to see one or two representatives of each thing in rust, so you can rely on the assurances Rust provides as much as possible as far down the stack as you can, if that's desired. In other words, I don't think we need every compression library ported to Rust, but I would like to see one or two choices for each type of compression, as applicable. An easy way to jump-start this is to take a library and port it.
  - pron 10y ago
    1. We are not talking about code that is under heavy development. I specifically said that rewriting code may be sensible if you intend to make big changes. 2. Obviously, verification tools (and I'm not just talking about static analysis, but also whitebox fuzzing and concolic testing) do have their downside. The question is not which approach yields the best overall result, but which is more economically sensible given the goal. > it would be nice to see one or two representatives of each thing in rust Absolutely, as long as it's done while analyzing the cost vs. benefit. What isn't so nice is to have the software industry waste precious efforts rewriting instead of developing new software, without analyzing where a rewrite would yield the most benefit. Obviously, anyone is free to choose what they want to spend their time on, but I think that as an industry we should at least encourage projects with higher impact first.
- dikaiosune 10y ago
  > You're suggesting that rewriting billions of lines of code that probably contain lots of bugs somewhere every time a new safer language comes along is the most sensible investment of effort by the software industry in order to improve the quality of our software infrastructure From my own post: > ... there are a bunch of calls to just rewrite the world in Rust, but even given unlimited resources, is it actually a reasonable thing to do? How well can Rust actually replace C in C’s native domains? What are best practices for managing the risks inherent in a project like that (injecting new bugs/vulnerabilities, missing important edge case functionality, etc.)? > I don’t really know good answers to these questions... So I'm not sure exactly where you're getting that argument. This also isn't "every time a new safer language comes along," Rust (AFAIK) is one of very few languages which is generally usable, memory safe, and without a garbage collection. > ... do you think that "rewrite all legacy code!" (an effort that would probably cost billions of dollars) does not need to be justified by concrete claims about actual cost and benefit? Absolutely not. But I don't see anywhere that I or others are making that claim. Part of what interests me here is to explore possible avenues of these rewrites. > so at least we know where the worst of the problem is I think that CVE's already give us a pretty good idea of places we might start if we were to seriously undertake this endeavor (as opposed to the hobbyist exploration I've started). > But fine, here's a taste (look for what is required to use the tool as well as results): ... Some of these look quite promising (and I suspect most on HN are familiar these days with AFL), but the only one which purports to present a cost/benefit analysis or postmortem requires paying $30 for the chapter. I'm not saying that analysis tools are a bad thing (and I would argue are quite badly needed because no matter how badly we want to we could never rewrite "everything" in Rust). I'm not saying that they aren't useful, or that they don't improve existing projects. I'm just saying that I don't yet see evidence that they are strictly always more cost-effective for needed improvements than a safe-language rewrite. I just don't know. I think that the software community barely knows how to properly quantify these things, so it's no surprise that there aren't any good metrics for making these decisions. > Now, do you have any sources suggesting that a complete rewrite is cost-effective? It's a small part of what I'm investigating with this project, specifically with regard to Rust. I think that looking into techniques for managing risk in any project (whether the changes are happening due to static analysis warnings or do to a rewrite) is useful. Although, regarding Rust, one company (MaidSafe, talk about this here: https://www.youtube.com/watch?v=t8z5rA3A1RA https://www.youtube.com/watch?v=t8z5rA3A1RA) has already reported very promising results from a C++ to Rust rewrite. Dropbox rewrote a core piece of their Magic Pocket infrastructure from Go to Rust to get a handle on memory usage and has also generally reported positive results (wired article: http://www.wired.com/2016/03/epic-story-dropboxs-exodus-amazon-cloud-empire/ http://www.wired.com/2016/03/epic-story-dropboxs-exodus-amaz..., subsequent HN conversation with Dropbox engineers: https://news.ycombinator.com/item?id=11282948 https://news.ycombinator.com/item?id=11282948). All that said, I wouldn't generally advocate for complete ground-up rewrites without an incremental plan. Which is why I'm experimenting with ways to do so successfully and ergonomically between C and Rust.

pron 10y ago

You're suggesting that rewriting billions of lines of code that probably contain lots of bugs somewhere every time a new safer language comes along is the most sensible investment of effort by the software industry in order to improve the quality of our software infrastructure, while I'm saying, wait, maybe we should use some inspection tools first before we decide on an approach, so at least we know where the worst of the problem is, and you think I'm the one who needs to cite sources? Alternatively, do you think that "rewrite all legacy code!" (an effort that would probably cost billions of dollars) does not need to be justified by concrete claims about actual cost and benefit? But fine, here's a taste (look for what is required to use the tool as well as results): * http://mir.cs.illinois.edu/marinov/publications/SenETAL05CUTE.pdf http://mir.cs.illinois.edu/marinov/publications/SenETAL05CUT... * http://lcamtuf.coredump.cx/afl/ http://lcamtuf.coredump.cx/afl/ (see the "The bug-o-rama trophy case") * http://research.microsoft.com/pubs/144848/2011%20CAV%20SLAyer_%20Memory%20Safety%20for%20Systems-level%20Code.pdf http://research.microsoft.com/pubs/144848/2011%20CAV%20SLAye... * https://code.facebook.com/posts/1648953042007882 https://code.facebook.com/posts/1648953042007882 * http://link.springer.com/chapter/10.1007/978-3-319-17524-9_1 http://link.springer.com/chapter/10.1007/978-3-319-17524-9_1 Note that there are some strengths, some weaknesses, but I hope those sources make it clear that using those tools first is at least more sensible than "rewriting everything", as they require orders of magnitude less effort, and they do uncover bugs (you're free to guess what percentage of those bugs would be found in a rewrite, and whether the bug density demands a rewrite). We may decide that a rewrite of some libraries is the best approach once we have more information. Now, do you have any sources suggesting that a complete rewrite is cost-effective?

kbenson 10y ago
The biggest problem I (not being the people you've been conversing with so far, for the most part) have with relying on static analysis tools on top of C, instead of a rewrite, is that the analysis tools are not enforceable. You can do all the analysis you want, and put in all the procedures you want, but in the end people are submitting changes to these libraries, and you need to make sure that these people are ensuring the tools get run prior to release. Additionally, from the perspective of a user of this code, my ability to look at the source and know that it's been scanned ranges from uncertain to non-existent. Switching to a compilation system where these constraints are enforced (such as a new language that enforces it) has a benefit in that respect. You could get a similar benefit by releasing a specialized C compiler that enforced some static analysis tools and required a very small amount of syntax change such that the code would not compile on a vanilla C compiler, and then I think the there would be much less of a case for a rewrite in Rust from a safety point of view (even if they would handle slightly different aspects of safety, as you've pointed out). That said, I'm not sure it makes sense to rewrite "everything" in Rust, but it would be nice to see one or two representatives of each thing in rust, so you can rely on the assurances Rust provides as much as possible as far down the stack as you can, if that's desired. In other words, I don't think we need every compression library ported to Rust, but I would like to see one or two choices for each type of compression, as applicable. An easy way to jump-start this is to take a library and port it.
- pron 10y ago
  1. We are not talking about code that is under heavy development. I specifically said that rewriting code may be sensible if you intend to make big changes. 2. Obviously, verification tools (and I'm not just talking about static analysis, but also whitebox fuzzing and concolic testing) do have their downside. The question is not which approach yields the best overall result, but which is more economically sensible given the goal. > it would be nice to see one or two representatives of each thing in rust Absolutely, as long as it's done while analyzing the cost vs. benefit. What isn't so nice is to have the software industry waste precious efforts rewriting instead of developing new software, without analyzing where a rewrite would yield the most benefit. Obviously, anyone is free to choose what they want to spend their time on, but I think that as an industry we should at least encourage projects with higher impact first.
dikaiosune 10y ago
> You're suggesting that rewriting billions of lines of code that probably contain lots of bugs somewhere every time a new safer language comes along is the most sensible investment of effort by the software industry in order to improve the quality of our software infrastructure From my own post: > ... there are a bunch of calls to just rewrite the world in Rust, but even given unlimited resources, is it actually a reasonable thing to do? How well can Rust actually replace C in C’s native domains? What are best practices for managing the risks inherent in a project like that (injecting new bugs/vulnerabilities, missing important edge case functionality, etc.)? > I don’t really know good answers to these questions... So I'm not sure exactly where you're getting that argument. This also isn't "every time a new safer language comes along," Rust (AFAIK) is one of very few languages which is generally usable, memory safe, and without a garbage collection. > ... do you think that "rewrite all legacy code!" (an effort that would probably cost billions of dollars) does not need to be justified by concrete claims about actual cost and benefit? Absolutely not. But I don't see anywhere that I or others are making that claim. Part of what interests me here is to explore possible avenues of these rewrites. > so at least we know where the worst of the problem is I think that CVE's already give us a pretty good idea of places we might start if we were to seriously undertake this endeavor (as opposed to the hobbyist exploration I've started). > But fine, here's a taste (look for what is required to use the tool as well as results): ... Some of these look quite promising (and I suspect most on HN are familiar these days with AFL), but the only one which purports to present a cost/benefit analysis or postmortem requires paying $30 for the chapter. I'm not saying that analysis tools are a bad thing (and I would argue are quite badly needed because no matter how badly we want to we could never rewrite "everything" in Rust). I'm not saying that they aren't useful, or that they don't improve existing projects. I'm just saying that I don't yet see evidence that they are strictly always more cost-effective for needed improvements than a safe-language rewrite. I just don't know. I think that the software community barely knows how to properly quantify these things, so it's no surprise that there aren't any good metrics for making these decisions. > Now, do you have any sources suggesting that a complete rewrite is cost-effective? It's a small part of what I'm investigating with this project, specifically with regard to Rust. I think that looking into techniques for managing risk in any project (whether the changes are happening due to static analysis warnings or do to a rewrite) is useful. Although, regarding Rust, one company (MaidSafe, talk about this here: https://www.youtube.com/watch?v=t8z5rA3A1RA https://www.youtube.com/watch?v=t8z5rA3A1RA) has already reported very promising results from a C++ to Rust rewrite. Dropbox rewrote a core piece of their Magic Pocket infrastructure from Go to Rust to get a handle on memory usage and has also generally reported positive results (wired article: http://www.wired.com/2016/03/epic-story-dropboxs-exodus-amazon-cloud-empire/ http://www.wired.com/2016/03/epic-story-dropboxs-exodus-amaz..., subsequent HN conversation with Dropbox engineers: https://news.ycombinator.com/item?id=11282948 https://news.ycombinator.com/item?id=11282948). All that said, I wouldn't generally advocate for complete ground-up rewrites without an incremental plan. Which is why I'm experimenting with ways to do so successfully and ergonomically between C and Rust.