6 ms·

> Yet none of that makes rewriting every line of C out there in Rust the most cost-effective way of improving software quality. What I know not to be cost-effe

> Yet none of that makes rewriting every line of C out there in Rust the most cost-effective way of improving software quality. What I know not to be cost-effective in the long run is working around C's inability to enforce its own abstractions. Anything that makes abstractions easier to protect is an improvement over what we have now.

pron 10y ago
> working around C's inability to enforce its own abstractions But you don't need to. We are talking about rewriting internal code that no one intends to touch, not about improving APIs. > Anything that makes abstractions easier to protect is an improvement over what we have now. Sure, so wrap C code with Rust. That's a whole other matter than rewriting it.
- catnaroek 10y ago
  > But you don't need to. We are talking about rewriting internal code that no one intends to touch, not about improving APIs. But it has to be touched, because it contains mistakes! > Sure, so wrap C code with Rust. That's a whole other matter than rewriting it. Reimplementing things in Rust from scratch has an important advantage: abstractions can be redesigned to maximize the extent to which the contract between API implementors and users is captured in types.
  - pron 10y ago
    > But it has to be touched, because it contains mistakes! Sure, but where exactly? Is manual translation into another language the best bang-for-the-buck approach you know for finding bugs? > Reimplementing things in Rust from scratch has an important advantage Of course it does! But is it the most cost-effective way to improve vast amounts of legacy code that isn't under active development? Just for the sake of argument, suppose some 10MLOC (which comprise a minuscule, nearly negligible, portion of all legacy C code) contain, say, 500 serious, dangerous bugs, 450 of them could be fixed in a Rust rewrite that would cost $10M, while, say, 400 of them could be fixed using C tools for the cost of, say $1M. Wouldn't it make much more sense not to do the rewrite for 10x the cost and 12% of added utility? Of course the situation could be much more in favor of Rust or in favor of C, but we don't know! Wouldn't it at least be far more responsible to use tools that would help us know where we stand for relatively little effort before we commit to one approach over the other?
    - lmm 10y ago
      > Just for the sake of argument, suppose some 10MLOC (which comprise a minuscule, nearly negligible, portion of all legacy C code) contain, say, 500 serious, dangerous bugs, 450 of them could be fixed in a Rust rewrite that would cost $10M, while, say, 400 of them could be fixed using C tools for the cost of, say $1M. Wouldn't it make much more sense not to do the rewrite for 10x the cost and 12% of added utility? I don't think 12% is the right way to think about it. The difference between 100 bugs and 50 bugs is at least as big as the difference between 200 bugs and 100 bugs, probably bigger - at least in terms of how much it costs. Suppose you go for the C approach, but turns out you really do need to get down to 50 outstanding bugs - at that point how much is it going to cost to do that in C? Suppose a year later you need to get down to 25? I think it's very easy to end up in a sunk costs situation where at each step improving the existing codebase is cheaper than rewriting, but overall you end up spending far more. Maybe we need to focus on migration paths. Maybe strategies like targeted rewrites of critical pieces can offer the same level of incrementalism as improving existing C code, but greater overall efficiency. Or maybe there's just no affordable way to salvage existing legacy codebases. IMO in the coming decades we're going to see attackers reach a level of sophistication where any bug in network-facing code is exploited; at that point more-or-less the only useful code will be (provably) bug-free code. I think the path from legacy C code to bug-free code that goes via rewrite-in-Rust-today is ultimately cheaper than the one that goes via use-tooling-to-improve-the-C-today. > Is manual translation into another language the best bang-for-the-buck approach you know for finding bugs? For finding a single bug, no. For finding bugs in bulk, yes (or rather, for reducing the bug rate below a certain threshold, if that required threshold is low enough). I honestly think the costs of translating between languages are very much overestimated - it's much more similar to refactoring code for clarity than it is to writing a program from scratch. (It doesn't have to be manual - there's no reason you can't do a semi-automated translation in the same way you'd do a semi-automated correctness proof.)
      - pron 10y ago
        > I think it's very easy to end up in a sunk costs situation where at each step improving the existing codebase is cheaper than rewriting, but overall you end up spending far more. I don't understand this analysis. First, a fixed bug is a fixed bug. You never ever have to rewrite a function that's correct. Second, the whole point of at least first trying to find the bugs, is that the whole effort is orders of magnitude smaller. Doing the verification step first and the rewrite later is as cheap as just doing the rewrite, if not cheaper (because then you at least know what to rewrite). If a codebase turns out to be broken beyond repair, by all means -- rewrite. But we have no reason to assume that's the case, so why make such a costly assumption? > at that point more-or-less the only useful code will be (provably) bug-free code In that case, why translate now? Proving bug-freedom is so hard regardless of the language[1], that translating the code first to Rust saves you very little and is just wasted effort that could have gone into improving formal verification tools. > For finding bugs in bulk, yes I must say I'm surprised. With all the automated test-generations and new static analysis tools out there, the first thing you reach for is a manual translation? > it's much more similar to refactoring code for clarity than it is to writing a program from scratch. I partly agree, but I think that refactoring largely untouched, largely correct code for clarity is also a waste of (much!) effort. > It doesn't have to be manual - there's no reason you can't do a semi-automated translation in the same way you'd do a semi-automated correctness proof. The main reason I can think of is that no such tool currently exists. If it did, it may significantly reduce the cost of translation and change the equation. I'd be very excited to have such a tool, and, in fact, I think creating one is a better investment than starting to manually rewrite old, largely untouched and largely correct code. [1]: I'm giving a talk about this in July at Curry On: http://curry-on.org/2016/sessions/why-writing-correct-software-is-hard-and-why-math-alone-wont-help-us.html http://curry-on.org/2016/sessions/why-writing-correct-softwa...

pron 10y ago

> working around C's inability to enforce its own abstractions But you don't need to. We are talking about rewriting internal code that no one intends to touch, not about improving APIs. > Anything that makes abstractions easier to protect is an improvement over what we have now. Sure, so wrap C code with Rust. That's a whole other matter than rewriting it.

catnaroek 10y ago
> But you don't need to. We are talking about rewriting internal code that no one intends to touch, not about improving APIs. But it has to be touched, because it contains mistakes! > Sure, so wrap C code with Rust. That's a whole other matter than rewriting it. Reimplementing things in Rust from scratch has an important advantage: abstractions can be redesigned to maximize the extent to which the contract between API implementors and users is captured in types.
- pron 10y ago
  > But it has to be touched, because it contains mistakes! Sure, but where exactly? Is manual translation into another language the best bang-for-the-buck approach you know for finding bugs? > Reimplementing things in Rust from scratch has an important advantage Of course it does! But is it the most cost-effective way to improve vast amounts of legacy code that isn't under active development? Just for the sake of argument, suppose some 10MLOC (which comprise a minuscule, nearly negligible, portion of all legacy C code) contain, say, 500 serious, dangerous bugs, 450 of them could be fixed in a Rust rewrite that would cost $10M, while, say, 400 of them could be fixed using C tools for the cost of, say $1M. Wouldn't it make much more sense not to do the rewrite for 10x the cost and 12% of added utility? Of course the situation could be much more in favor of Rust or in favor of C, but we don't know! Wouldn't it at least be far more responsible to use tools that would help us know where we stand for relatively little effort before we commit to one approach over the other?
  - lmm 10y ago
    > Just for the sake of argument, suppose some 10MLOC (which comprise a minuscule, nearly negligible, portion of all legacy C code) contain, say, 500 serious, dangerous bugs, 450 of them could be fixed in a Rust rewrite that would cost $10M, while, say, 400 of them could be fixed using C tools for the cost of, say $1M. Wouldn't it make much more sense not to do the rewrite for 10x the cost and 12% of added utility? I don't think 12% is the right way to think about it. The difference between 100 bugs and 50 bugs is at least as big as the difference between 200 bugs and 100 bugs, probably bigger - at least in terms of how much it costs. Suppose you go for the C approach, but turns out you really do need to get down to 50 outstanding bugs - at that point how much is it going to cost to do that in C? Suppose a year later you need to get down to 25? I think it's very easy to end up in a sunk costs situation where at each step improving the existing codebase is cheaper than rewriting, but overall you end up spending far more. Maybe we need to focus on migration paths. Maybe strategies like targeted rewrites of critical pieces can offer the same level of incrementalism as improving existing C code, but greater overall efficiency. Or maybe there's just no affordable way to salvage existing legacy codebases. IMO in the coming decades we're going to see attackers reach a level of sophistication where any bug in network-facing code is exploited; at that point more-or-less the only useful code will be (provably) bug-free code. I think the path from legacy C code to bug-free code that goes via rewrite-in-Rust-today is ultimately cheaper than the one that goes via use-tooling-to-improve-the-C-today. > Is manual translation into another language the best bang-for-the-buck approach you know for finding bugs? For finding a single bug, no. For finding bugs in bulk, yes (or rather, for reducing the bug rate below a certain threshold, if that required threshold is low enough). I honestly think the costs of translating between languages are very much overestimated - it's much more similar to refactoring code for clarity than it is to writing a program from scratch. (It doesn't have to be manual - there's no reason you can't do a semi-automated translation in the same way you'd do a semi-automated correctness proof.)
    - pron 10y ago
      > I think it's very easy to end up in a sunk costs situation where at each step improving the existing codebase is cheaper than rewriting, but overall you end up spending far more. I don't understand this analysis. First, a fixed bug is a fixed bug. You never ever have to rewrite a function that's correct. Second, the whole point of at least first trying to find the bugs, is that the whole effort is orders of magnitude smaller. Doing the verification step first and the rewrite later is as cheap as just doing the rewrite, if not cheaper (because then you at least know what to rewrite). If a codebase turns out to be broken beyond repair, by all means -- rewrite. But we have no reason to assume that's the case, so why make such a costly assumption? > at that point more-or-less the only useful code will be (provably) bug-free code In that case, why translate now? Proving bug-freedom is so hard regardless of the language[1], that translating the code first to Rust saves you very little and is just wasted effort that could have gone into improving formal verification tools. > For finding bugs in bulk, yes I must say I'm surprised. With all the automated test-generations and new static analysis tools out there, the first thing you reach for is a manual translation? > it's much more similar to refactoring code for clarity than it is to writing a program from scratch. I partly agree, but I think that refactoring largely untouched, largely correct code for clarity is also a waste of (much!) effort. > It doesn't have to be manual - there's no reason you can't do a semi-automated translation in the same way you'd do a semi-automated correctness proof. The main reason I can think of is that no such tool currently exists. If it did, it may significantly reduce the cost of translation and change the equation. I'd be very excited to have such a tool, and, in fact, I think creating one is a better investment than starting to manually rewrite old, largely untouched and largely correct code. [1]: I'm giving a talk about this in July at Curry On: http://curry-on.org/2016/sessions/why-writing-correct-software-is-hard-and-why-math-alone-wont-help-us.html http://curry-on.org/2016/sessions/why-writing-correct-softwa...