6 ms·

I'm personally not a fan of "rewrite the world in Rust" mentality, but that being said, if one is planning to port a project to a new language or platform, mech

I'm personally not a fan of "rewrite the world in Rust" mentality, but that being said, if one is planning to port a project to a new language or platform, mechanical translation is a poor means of doing so. Spend the time planning better architecture and designing a better software system, and find a way to replace it piece by piece. Don't build a castle in the sky, because it will never reach the ground. If you've decided to use Rust for this system, that's fine. But, write Rust. Don't try to back-port C into Rust. I think a far better and more mature process is to update C to modern C and use a model checker such as CBMC to verify memory, resource, and integer math safety. One gets the same safety as a gradual Rust rewrite, but the code base, knowledge base, and developers can be maintained.

pdimitar 2y ago
> I'm personally not a fan of "rewrite the world in Rust" mentality There is no such mentality anywhere. There is a ton of software that's much better off left alone in a dynamic language, or a statically typed language with a garbage collector (like Golang). Good engineers understand the idea of using the right tool for the job. The push is to start reducing those memory safety CVEs because they have been proven to be a real problem, many times over. > mechanical translation is a poor means of doing so Agreed. If we could automatically and reliably translate C/C++ to Rust it would have been done already. > Spend the time planning better architecture and designing a better software system, and find a way to replace it piece by piece. OK, I am just saying that somewhere along that process people might get a bout of confidence and tell themselves "oh, we're doing C much better now, we no longer write memory safety bugs, can't we stop here?" and they absolutely will. Cue another hilarious buffer overflow CVE 6 months later. > I think a far better and more mature process is to update C to modern C and use a model checker such as CBMC to verify memory, resource, and integer math safety. A huge investment. If you are going to do that then you might as well just move to Rust. > One gets the same safety as a gradual Rust rewrite Maybe, but that sounds fairly uncertain or far from a clear takeaway to me.
- nanolith 2y ago
  > A huge investment. If you are going to do that then you might as well just move to Rust. People say that, but the people who say this rarely have any practical experience using CBMC. It's very straight-forward to use. I could teach a developer to use it reliably, on practical software, in a month.
  - pdimitar 2y ago
    I am not denying it, nor am I claiming that "just move to Rust" is an universal escape hatch. What I am saying is that if it were as simple as "just learn CBMC" then maybe Microsoft and Google would have not published their studies demonstrating that 60% - 75% of all CVEs are memory safety errors like buffer under-/over-flows.
    - nanolith 2y ago
      These studies aren't wrong. But, that's also because neither Microsoft nor Google make use of practical formal methods in practice. Both have research teams and pie-in-the-sky projects, not dissimilar to this DARPA project. But, when it comes down to the nitty-gritty development cycle, both companies use decades old software development practices.
- uecker 2y ago
  Rewriting is rarely a good idea in general. Rust proponents like to pretend that it is impossible to avoid safety issues in C while it is automatically given in Rust. But this is not so simply in reality.
  - pdimitar 2y ago
    I don't like generalizations... in in general. :D (Addressing your "rewrites are rarely a good idea in general" here.) My experience tells me that if a tech stack supports certain safety guarantees by default that this leads to measurable reduction of those safety problems when you switch to the stack. People love convenient defaults, that's a fact of life. The apparently inconvenient truth is that most programmers are quite average and you can't rely on them going above and beyond to reduce memory safety errors. So I don't buy the good old argument of "just hire better C programmers". We still have a ton of buffer overflow CVEs regardless. And I never "pretended it's impossible to avoid safety issues in C". I'll appreciate if you don't clump me in some imaginary group of "Rust proponents". What I'm saying is this: use the right tool for the job. The C devs have been given decades and yet memory safety CVEs are still prevalent. What conclusion would you arrive at if you were in my place -- i.e. not coding C for a living for like 18 years now but still witnessing it periodically crapping the bed? I'm curious of your take on this. Again, what other conclusion would you arrive at?
    - uecker 2y ago
      I am complaining about the usual phrases which are part of the Rust marketing, like the "just hire better C programmer did not work" or the "why are there still CVEs" pseudo arguments, etc. For example, let's look at the "hire better C programmers does not work" argument. Like every good propaganda it starts with a truism: In this case that even highly skilled C/C++ programmers will make mistakes that could lead to exploitable memory safety issues. The problem comes from exaggerating this to the idea that "all hope is lost and nothing can be done". In reality one can obviously do a lot of things to improve safety in C/C++. And even one short look at CVEs should make it clear that there is often huge room for improvements even with relatively simple measures. For example, a lot of memory safety bugs in C/C++ come from open-coded string or buffer manipulation. But it is not exactly rocket science to abstract this away behind a safer interface. But once this is understood, the obvious conclusion is that addressing some of these low-hanging fruits would be far more effective in improving safety than wasting a lot of time and effort in rewriting in Rust.
- andrewflnr 2y ago
  > There is no such mentality anywhere. There definitely is. Mainstream and official Rust community material is generally sane, but the meme did not come from nowhere. The rewrite-everything people are out there.
  - pdimitar 2y ago
    > The rewrite-everything people are out there. Meh, there are zealots in every community -- we're not even talking programming language communities only. Not even programming either. Everywhere. No idea why people over-reacted so much to one particular 0.1% fanatics. It's a pretty normal state of affairs. Point me at your hobby group and even if it is only 20 people I can bet my balls at least 1 of them is a fanatic.
    - andrewflnr 2y ago
      Overreacting to fanatics is also a normal state of affairs, so don't act surprised. :) By their nature fanatics almost always make a disproportionate amount of noise, and if you're outside the community you often can't tell the difference: don't know which if any of the loudmouths members pay attention to, etc. And even more broadly, a small number of people can cause a lot of damage.
IshKebab 2y ago
> I think a far better and more mature process is to update C to modern C and use a model checker such as CBMC to verify memory, resource, and integer math safety. No chance. CBMC is amazing, but have you actually tried formally verifying a "real" program? I agree replacing with a hand-architected Rust version is clearly the better solution but also more expensive. I think they're going for an RLBox style "improve security significantly with little-to-no effort" type product here. That doesn't mean you shouldn't do a full manual rewrite if you have the resources, but it's better than nothing if you haven't.
- nanolith 2y ago
  > No chance. CBMC is amazing, but have you actually tried formally verifying a "real" program? Yes. Every day. It's actually quite easy to do. Write shadow methods covering the resources and function contracts of called functions, then verify the function. Repeat all of the way up and down the stack. It adds about 30% overhead over just TDD development.
  - PhilipRoman 2y ago
    Last time I tried CBMC, it ended up running out of memory for relatively small programs, do you encounter any resource usage issues with it? I'm learning Frama-C and I find it more predictable, although the non-determinism of solvers shocked me when I first tried to prove non-trivial programs. I guess ideally I would like something even more explicit than Frama-C.
    - nanolith 2y ago
      CBMC works best on functions, not programs. You want to isolate an individual function, then provide shadows of the functions it calls. The shadows should have nondeterministic behavior (cover every possible error condition) and otherwise follow the same memory and resource rules as the original function. For instance, if shadowing a function that reads a buffer, the shadow should ensure full buffer access as part of its assertions. The biggest issue you will run into with bounded model checking is recursion and looping. In these cases, you want to refactor the code to make it easier to formally verify outside of the loop. Capture and assert on loop variants / invariants, and feed these forward in assertions on code. There's no way I can capture all of this in an HN comment, but to get CBMC to work, you need to break down your code.
Apofis 2y ago
This is definitely a pie-in-the-sky DARPA challenge that would be great to have around as we migrate away from legacy systems, however, even taking your functions/methods in one language and giving them to ChatGPT and asking it to translate your method to a different language generally doesn't work. Asking ChatGPT the initial problem you're trying to solve, works more frequently, but still generally doesn't work. You still need to do a lot of tinkering and thinking to get even basic things to work that it outputs.
usrusr 2y ago
If you have dormant code, as in running everywhere but not getting worked on anywhere, a "translate to shitty rust before ever touching again" has a certain appeal. Not the appeal of an obviously good idea: chances are the "shitty rust" created through translation would be so much worse to work on than C with some level of background noise of bugs (that would also be present in the "shitty rust" thanks to faithful translation). In C, people have an idea about how to deal with the problems. In "shitty rust", it's, well, shitty, because rust people are not used to that stuff. But there's a non-zero chance that someone could develop a skillset for iteratively cleaning up into something tolerable. And then there are non-goal things that could grow out of the project, e.g. some form of linter feedback "can't translate into tolerable rust because of x, y and z". C people could look into that, and once the code is translatable into good rust, why translate. If that was an outcome of the project, some people might find it easier to describe their solution in runnable C and let the "translator/linter" guide them to a non-broken approach. I'd certainly consider all these positive outcomes quite unlikely, but isn't it pretty much the job description of DARPA to do the occasional dark horse bet?
- suprjami 2y ago
  In my experience (supporting a machine-translated codebase which resulted in shitty Java) your theory doesn't play out. If you give developers a shitty codebase then those developers will leave to work somewhere else. After a few years of working on this codebase we had 88% turnover. 1 in 10 developers remembered the original project's design philosophy and intention. It wasn't a good situation.
  - dcsommer 2y ago
    GP was proposing a different situation where the source code is not changing or changing very rarely. If you have a high churn codebase, obviously the maintenance experience will worsen dramatically after machine translation (at least with many current tools), so your experience is not unexpected.
pjmlp 2y ago
Modern C still has the same security exploits in arrays and strings as Classical C, nothing changed in 50 years.
- Y_Y 2y ago
  The programmers have changed, the machines have changed, the literature has changed, the compilers have changed a lot. You can still write and run the old insecure code, but you'll get warnings and hit stack canaries and your colleagues will gasp at you and your merge requests will be rejected.
  - pjmlp 2y ago
    Meaningless changes, as proven by the CVE database, or the kernel corruption by a bad pointer caused by Crowdstrike.
    - Y_Y 2y ago
      I respectfully disagree. GP claimed that nothing has changed [regarding string and array security bugs in C] in 50 years. I responded that many relevant factors have changed, such that people tend to write different code now which is less susceptible to those bugs. Of course the same old bugs are possible, and sometimes good coders will still write them. Still I argue that there has been meaningful change since there are more protections against writing bugs in the first place, less incentive to write dangerous code, and more security for when (some) bugs still appear.
      - pjmlp 2y ago
        ISO C89 is exactly like ISO C23 in that regard. CVE database proves that those kind of errors keep coming up in 2024, regardless of those changes. Not only do they keep coming up, the monetary cost of fixing those issues has raised up to a level that now even governments are looking into this.
        Y_Y 2y ago

pdimitar 2y ago

> I'm personally not a fan of "rewrite the world in Rust" mentality There is no such mentality anywhere. There is a ton of software that's much better off left alone in a dynamic language, or a statically typed language with a garbage collector (like Golang). Good engineers understand the idea of using the right tool for the job. The push is to start reducing those memory safety CVEs because they have been proven to be a real problem, many times over. > mechanical translation is a poor means of doing so Agreed. If we could automatically and reliably translate C/C++ to Rust it would have been done already. > Spend the time planning better architecture and designing a better software system, and find a way to replace it piece by piece. OK, I am just saying that somewhere along that process people might get a bout of confidence and tell themselves "oh, we're doing C much better now, we no longer write memory safety bugs, can't we stop here?" and they absolutely will. Cue another hilarious buffer overflow CVE 6 months later. > I think a far better and more mature process is to update C to modern C and use a model checker such as CBMC to verify memory, resource, and integer math safety. A huge investment. If you are going to do that then you might as well just move to Rust. > One gets the same safety as a gradual Rust rewrite Maybe, but that sounds fairly uncertain or far from a clear takeaway to me.

nanolith 2y ago
> A huge investment. If you are going to do that then you might as well just move to Rust. People say that, but the people who say this rarely have any practical experience using CBMC. It's very straight-forward to use. I could teach a developer to use it reliably, on practical software, in a month.
- pdimitar 2y ago
  I am not denying it, nor am I claiming that "just move to Rust" is an universal escape hatch. What I am saying is that if it were as simple as "just learn CBMC" then maybe Microsoft and Google would have not published their studies demonstrating that 60% - 75% of all CVEs are memory safety errors like buffer under-/over-flows.
  - nanolith 2y ago
    These studies aren't wrong. But, that's also because neither Microsoft nor Google make use of practical formal methods in practice. Both have research teams and pie-in-the-sky projects, not dissimilar to this DARPA project. But, when it comes down to the nitty-gritty development cycle, both companies use decades old software development practices.
uecker 2y ago
Rewriting is rarely a good idea in general. Rust proponents like to pretend that it is impossible to avoid safety issues in C while it is automatically given in Rust. But this is not so simply in reality.
- pdimitar 2y ago
  I don't like generalizations... in in general. :D (Addressing your "rewrites are rarely a good idea in general" here.) My experience tells me that if a tech stack supports certain safety guarantees by default that this leads to measurable reduction of those safety problems when you switch to the stack. People love convenient defaults, that's a fact of life. The apparently inconvenient truth is that most programmers are quite average and you can't rely on them going above and beyond to reduce memory safety errors. So I don't buy the good old argument of "just hire better C programmers". We still have a ton of buffer overflow CVEs regardless. And I never "pretended it's impossible to avoid safety issues in C". I'll appreciate if you don't clump me in some imaginary group of "Rust proponents". What I'm saying is this: use the right tool for the job. The C devs have been given decades and yet memory safety CVEs are still prevalent. What conclusion would you arrive at if you were in my place -- i.e. not coding C for a living for like 18 years now but still witnessing it periodically crapping the bed? I'm curious of your take on this. Again, what other conclusion would you arrive at?
  - uecker 2y ago
    I am complaining about the usual phrases which are part of the Rust marketing, like the "just hire better C programmer did not work" or the "why are there still CVEs" pseudo arguments, etc. For example, let's look at the "hire better C programmers does not work" argument. Like every good propaganda it starts with a truism: In this case that even highly skilled C/C++ programmers will make mistakes that could lead to exploitable memory safety issues. The problem comes from exaggerating this to the idea that "all hope is lost and nothing can be done". In reality one can obviously do a lot of things to improve safety in C/C++. And even one short look at CVEs should make it clear that there is often huge room for improvements even with relatively simple measures. For example, a lot of memory safety bugs in C/C++ come from open-coded string or buffer manipulation. But it is not exactly rocket science to abstract this away behind a safer interface. But once this is understood, the obvious conclusion is that addressing some of these low-hanging fruits would be far more effective in improving safety than wasting a lot of time and effort in rewriting in Rust.
andrewflnr 2y ago
> There is no such mentality anywhere. There definitely is. Mainstream and official Rust community material is generally sane, but the meme did not come from nowhere. The rewrite-everything people are out there.
- pdimitar 2y ago
  > The rewrite-everything people are out there. Meh, there are zealots in every community -- we're not even talking programming language communities only. Not even programming either. Everywhere. No idea why people over-reacted so much to one particular 0.1% fanatics. It's a pretty normal state of affairs. Point me at your hobby group and even if it is only 20 people I can bet my balls at least 1 of them is a fanatic.
  - andrewflnr 2y ago
    Overreacting to fanatics is also a normal state of affairs, so don't act surprised. :) By their nature fanatics almost always make a disproportionate amount of noise, and if you're outside the community you often can't tell the difference: don't know which if any of the loudmouths members pay attention to, etc. And even more broadly, a small number of people can cause a lot of damage.

IshKebab 2y ago

> I think a far better and more mature process is to update C to modern C and use a model checker such as CBMC to verify memory, resource, and integer math safety. No chance. CBMC is amazing, but have you actually tried formally verifying a "real" program? I agree replacing with a hand-architected Rust version is clearly the better solution but also more expensive. I think they're going for an RLBox style "improve security significantly with little-to-no effort" type product here. That doesn't mean you shouldn't do a full manual rewrite if you have the resources, but it's better than nothing if you haven't.

nanolith 2y ago
> No chance. CBMC is amazing, but have you actually tried formally verifying a "real" program? Yes. Every day. It's actually quite easy to do. Write shadow methods covering the resources and function contracts of called functions, then verify the function. Repeat all of the way up and down the stack. It adds about 30% overhead over just TDD development.
- PhilipRoman 2y ago
  Last time I tried CBMC, it ended up running out of memory for relatively small programs, do you encounter any resource usage issues with it? I'm learning Frama-C and I find it more predictable, although the non-determinism of solvers shocked me when I first tried to prove non-trivial programs. I guess ideally I would like something even more explicit than Frama-C.
  - nanolith 2y ago
    CBMC works best on functions, not programs. You want to isolate an individual function, then provide shadows of the functions it calls. The shadows should have nondeterministic behavior (cover every possible error condition) and otherwise follow the same memory and resource rules as the original function. For instance, if shadowing a function that reads a buffer, the shadow should ensure full buffer access as part of its assertions. The biggest issue you will run into with bounded model checking is recursion and looping. In these cases, you want to refactor the code to make it easier to formally verify outside of the loop. Capture and assert on loop variants / invariants, and feed these forward in assertions on code. There's no way I can capture all of this in an HN comment, but to get CBMC to work, you need to break down your code.

Apofis 2y ago

This is definitely a pie-in-the-sky DARPA challenge that would be great to have around as we migrate away from legacy systems, however, even taking your functions/methods in one language and giving them to ChatGPT and asking it to translate your method to a different language generally doesn't work. Asking ChatGPT the initial problem you're trying to solve, works more frequently, but still generally doesn't work. You still need to do a lot of tinkering and thinking to get even basic things to work that it outputs.

usrusr 2y ago

If you have dormant code, as in running everywhere but not getting worked on anywhere, a "translate to shitty rust before ever touching again" has a certain appeal. Not the appeal of an obviously good idea: chances are the "shitty rust" created through translation would be so much worse to work on than C with some level of background noise of bugs (that would also be present in the "shitty rust" thanks to faithful translation). In C, people have an idea about how to deal with the problems. In "shitty rust", it's, well, shitty, because rust people are not used to that stuff. But there's a non-zero chance that someone could develop a skillset for iteratively cleaning up into something tolerable. And then there are non-goal things that could grow out of the project, e.g. some form of linter feedback "can't translate into tolerable rust because of x, y and z". C people could look into that, and once the code is translatable into good rust, why translate. If that was an outcome of the project, some people might find it easier to describe their solution in runnable C and let the "translator/linter" guide them to a non-broken approach. I'd certainly consider all these positive outcomes quite unlikely, but isn't it pretty much the job description of DARPA to do the occasional dark horse bet?

suprjami 2y ago
In my experience (supporting a machine-translated codebase which resulted in shitty Java) your theory doesn't play out. If you give developers a shitty codebase then those developers will leave to work somewhere else. After a few years of working on this codebase we had 88% turnover. 1 in 10 developers remembered the original project's design philosophy and intention. It wasn't a good situation.
- dcsommer 2y ago
  GP was proposing a different situation where the source code is not changing or changing very rarely. If you have a high churn codebase, obviously the maintenance experience will worsen dramatically after machine translation (at least with many current tools), so your experience is not unexpected.

pjmlp 2y ago

Modern C still has the same security exploits in arrays and strings as Classical C, nothing changed in 50 years.

Y_Y 2y ago
The programmers have changed, the machines have changed, the literature has changed, the compilers have changed a lot. You can still write and run the old insecure code, but you'll get warnings and hit stack canaries and your colleagues will gasp at you and your merge requests will be rejected.
- pjmlp 2y ago
  Meaningless changes, as proven by the CVE database, or the kernel corruption by a bad pointer caused by Crowdstrike.
  - Y_Y 2y ago
    I respectfully disagree. GP claimed that nothing has changed [regarding string and array security bugs in C] in 50 years. I responded that many relevant factors have changed, such that people tend to write different code now which is less susceptible to those bugs. Of course the same old bugs are possible, and sometimes good coders will still write them. Still I argue that there has been meaningful change since there are more protections against writing bugs in the first place, less incentive to write dangerous code, and more security for when (some) bugs still appear.
    - pjmlp 2y ago
      ISO C89 is exactly like ISO C23 in that regard. CVE database proves that those kind of errors keep coming up in 2024, regardless of those changes. Not only do they keep coming up, the monetary cost of fixing those issues has raised up to a level that now even governments are looking into this.
      - Y_Y 2y ago