8 ms·

Perhaps worth mentioning is that someone attempted to port this to Rust and got about 60,000 lines of code into it before archiving the project. I feel like com

by suby 2y ago

Perhaps worth mentioning is that someone attempted to port this to Rust and got about 60,000 lines of code into it before archiving the project. I feel like comparing these two efforts would be an interesting case study on the impacts / benefits / limitations or difficulties, etc involved in rewriting from C++ to Rust. https://github.com/sidkshatriya/rd/ https://github.com/sidkshatriya/rd/

dwattttt 2y ago
It would be a good but difficult analysis; at a quick check, rr 1.0 took 3 years and signficant contributions from around 3 or 4 people (I saw at least 5 people contributing), and the rr we have today is 10 years further work on that.
tarruda 2y ago
I don't understand the "rewrite X/Y/Z in Rust" trend that has been going for a few years. I'm not familiar with Rust, but I'm almost sure it has a good C interoperability. If a certain piece of software is working well, what is the benefit of rewriting it in Rust?
- coldtea 2y ago
  >If a certain piece of software is working well, what is the benefit of rewriting it in Rust? Presumablly the idea here is to support Rust replay debugging, not just rewrite a C/C++ targetting replay debugger in Rust
  - icholy 2y ago
    It works with pretty much any compiled language.
    - coldtea 2y ago
      Does it understand the semantics, primitives, and structures of any compiled language? Or is it more of a lowest common denominator experience, where e.g. all of them are constrained to common semantics of C/C++?
      - icholy 2y ago
        That's up to debuggers to implement. lldb, gdb, and delve all support rr traces.
  - asveikau 2y ago
    I'm not so sure of this. Most code in a debugger doesn't have much to do with the source language. Line mapping and figuring out values for variables happens via symbol information, eg. DWARF, and compilers for multiple languages can produce that in the same format.
    - coldtea 2y ago
      Does that neutral way of debugging work as well as having first-class explicit support for the language, or it's more of a lowest common denominator (kind of like what e.g. language support over LSP can offer and how conveniently, compared to what a native Lisp or Smalltalk environment's language support can offer)?
      - asveikau 2y ago
        One debugger feature I thought of while writing the comment was the ?? command in windbg. That takes an expression and evaluates it. Gdb also does this which I've used with the "print" command to print a C expression, including pointer casts and such. That would obviously require language support. But then again you don't need to code everything in the same language either. You could write a rust parser in another language. Or a modular interface to dispatch knowledge of a programming language (does Microsoft's "language server" concept work this way?)
      - Veserv 2y ago
        The latter. The neutral way of debugging is really debugging the raw machine code of a process. This requires OS integration for your low level manipulation primitives. To add language support, you then need to figure out how to define the semantics of your manipulations in terms of the low-level primitives. If you have a rich runtime, you can add language-level debugging facilities that can operate at a higher level. However, this requires you to implement portions of a debugger in your runtime. Now you have to maintain a language, runtime, and debugger. It also means that if new debugger techniques are invented, such as time travel debugging, you do not get them for free since you embedded a debugger of your own design. So, like many similar things, it is a trade-off of specialization versus maintenance. The perennial question of use a library, or do it yourself.
- _dain_ 2y ago
  >If a certain piece of software is working well, what is the benefit of rewriting it in Rust? The "working" C program has a high risk of undiscovered bugs relating to concurrency and memory safety. Rust lets you rule out a large swathe of them by construction. Rust's type system is also far more expressive, which in many cases enables cleaner domain modelling.
  - jvanderbot 2y ago
    You're correct that this is the stated justification most of the time. Should be nuanced though, because the working C program has a risk, but the the risk is a function of the size of the codebase, its age, and the number of audits it has undergone. It is definitely easier to write bugs in C due to the additional freedom you have, but it is not necessarily a "high" risk for mature C libraries. It is definitely not as advisable to just replace all C with Rust, but it is advisable to prefer memory safety in new projects.
    - _dain_ 2y ago
      Definitely. Once you aren't adding new features, the only thing left to do is fix bugs. New rewrites can make the same old mistakes. But there is the potential to have a lower "bug floor" in a safer language.
- johnwatson11218 2y ago
  Recently the federal government issued a security advisory encouraging all new development to be done in Rust. I'm not sure the extent of which agencies this was meant to cover but it struck me as very unrealistic having just done years of java development for a state agency. https://www.nextgov.com/cybersecurity/2024/02/white-house-urges-software-developers-use-memory-safe-programming-languages/394455/ https://www.nextgov.com/cybersecurity/2024/02/white-house-ur...
  - detaro 2y ago
    It doesn't say that, no. Even in the title it says "memory-safe languages", which of course includes Java. Rust is "only" mentioned in the context of a C / C++ replacement, which tends to be a different area.
    - johnwatson11218 2y ago
      Yes that is correct, I had read a different article that had a more rust first focus. I just tried to google for a reference to the news release I was referencing, but yea it doesn't really support my comment :(
  - 77pt77 2y ago
    > the federal government issued a security advisory Well, I guess that's it... Maybe rust will take the same path as Ada.
    - deleted 2y ago
      [deleted]
- rtpg 2y ago
  There's a bit of a higher abstraction ceiling in Rust so in theory if you are successful rewriting a thing in Rust then you now have a codebase that's easier to change confidently. This sort of property is nice to have in huge codebases where you really start losing confidence in shipping changes that don't subtly break things. But of course a huge codebase is hard to rewrite in general...
  - taneq 2y ago
    Sounds like a perfect situation for a strangler pattern? Wrap or transpile the original code into a language with stronger refactoring support and the rest should become incrementally easier.
    - kvdveer 2y ago
      The advantages of rust only come when you actually use the rust-provides abstraction, especially those around allocation and concurrency. Even if transpiling is possible, the code would still not be structured I the rust way, and you wouldn't have any of the benefits. Same goes for wrapping.
  - nsajko 2y ago
    > There's a bit of a higher abstraction ceiling in Rust Compared to C, yes, but not compared to C++.
    - tialaramex 2y ago
      This isn't really true. Rust has a much better type system. When writing generic code the impact is enormous. C++ doesn't have a real Empty Type, and it thinks Units have non-zero size. In practical terms this makes it incredibly wasteful and in terms of a clear abstraction it encourages you to come up with a hack that's unclear but efficient.
      - saagarjha 2y ago
        …or you can just waste a few bytes? It's not a big deal.
- oguz-ismail 2y ago
  > If a certain piece of software is working well, what is the benefit of rewriting it in Rust? One benefit is it's easier to hide malicious code thanks to Rust's complicated syntax.
  - roca 2y ago
    It's far easier to hide malicious code in C or C++: just write some subtle undefined behavior that you can write an exploit against. Developers do that all the time even when they're not trying to be malicious. In Rust you'd have to wrap it in "unsafe" which draws attention.
khuey 2y ago
From the perspective of an rr maintainer, Sid's work was good and we were supportive of it. The main issues with migrating to that as the "blessed" version are that 1) rr has accumulated a decade of very hairy fixes for crazy kernel/process behavior that we feared could be lost during a port and 2) there's a closed source project (remix[0]) built on top of rr that would have needed to be ported too. [0] https://robert.ocallahan.org/2020/12/rr-remix-efficient-replay-only-binary.html https://robert.ocallahan.org/2020/12/rr-remix-efficient-repl...

by suby 2y ago

dwattttt 2y ago

It would be a good but difficult analysis; at a quick check, rr 1.0 took 3 years and signficant contributions from around 3 or 4 people (I saw at least 5 people contributing), and the rr we have today is 10 years further work on that.

tarruda 2y ago

I don't understand the "rewrite X/Y/Z in Rust" trend that has been going for a few years. I'm not familiar with Rust, but I'm almost sure it has a good C interoperability. If a certain piece of software is working well, what is the benefit of rewriting it in Rust?

coldtea 2y ago
>If a certain piece of software is working well, what is the benefit of rewriting it in Rust? Presumablly the idea here is to support Rust replay debugging, not just rewrite a C/C++ targetting replay debugger in Rust
- icholy 2y ago
  It works with pretty much any compiled language.
  - coldtea 2y ago
    Does it understand the semantics, primitives, and structures of any compiled language? Or is it more of a lowest common denominator experience, where e.g. all of them are constrained to common semantics of C/C++?
    - icholy 2y ago
      That's up to debuggers to implement. lldb, gdb, and delve all support rr traces.
- asveikau 2y ago
  I'm not so sure of this. Most code in a debugger doesn't have much to do with the source language. Line mapping and figuring out values for variables happens via symbol information, eg. DWARF, and compilers for multiple languages can produce that in the same format.
  - coldtea 2y ago
    Does that neutral way of debugging work as well as having first-class explicit support for the language, or it's more of a lowest common denominator (kind of like what e.g. language support over LSP can offer and how conveniently, compared to what a native Lisp or Smalltalk environment's language support can offer)?
    - asveikau 2y ago
      One debugger feature I thought of while writing the comment was the ?? command in windbg. That takes an expression and evaluates it. Gdb also does this which I've used with the "print" command to print a C expression, including pointer casts and such. That would obviously require language support. But then again you don't need to code everything in the same language either. You could write a rust parser in another language. Or a modular interface to dispatch knowledge of a programming language (does Microsoft's "language server" concept work this way?)
    - Veserv 2y ago
      The latter. The neutral way of debugging is really debugging the raw machine code of a process. This requires OS integration for your low level manipulation primitives. To add language support, you then need to figure out how to define the semantics of your manipulations in terms of the low-level primitives. If you have a rich runtime, you can add language-level debugging facilities that can operate at a higher level. However, this requires you to implement portions of a debugger in your runtime. Now you have to maintain a language, runtime, and debugger. It also means that if new debugger techniques are invented, such as time travel debugging, you do not get them for free since you embedded a debugger of your own design. So, like many similar things, it is a trade-off of specialization versus maintenance. The perennial question of use a library, or do it yourself.
_dain_ 2y ago
>If a certain piece of software is working well, what is the benefit of rewriting it in Rust? The "working" C program has a high risk of undiscovered bugs relating to concurrency and memory safety. Rust lets you rule out a large swathe of them by construction. Rust's type system is also far more expressive, which in many cases enables cleaner domain modelling.
- jvanderbot 2y ago
  You're correct that this is the stated justification most of the time. Should be nuanced though, because the working C program has a risk, but the the risk is a function of the size of the codebase, its age, and the number of audits it has undergone. It is definitely easier to write bugs in C due to the additional freedom you have, but it is not necessarily a "high" risk for mature C libraries. It is definitely not as advisable to just replace all C with Rust, but it is advisable to prefer memory safety in new projects.
  - _dain_ 2y ago
    Definitely. Once you aren't adding new features, the only thing left to do is fix bugs. New rewrites can make the same old mistakes. But there is the potential to have a lower "bug floor" in a safer language.
johnwatson11218 2y ago
Recently the federal government issued a security advisory encouraging all new development to be done in Rust. I'm not sure the extent of which agencies this was meant to cover but it struck me as very unrealistic having just done years of java development for a state agency. https://www.nextgov.com/cybersecurity/2024/02/white-house-urges-software-developers-use-memory-safe-programming-languages/394455/ https://www.nextgov.com/cybersecurity/2024/02/white-house-ur...
- detaro 2y ago
  It doesn't say that, no. Even in the title it says "memory-safe languages", which of course includes Java. Rust is "only" mentioned in the context of a C / C++ replacement, which tends to be a different area.
  - johnwatson11218 2y ago
    Yes that is correct, I had read a different article that had a more rust first focus. I just tried to google for a reference to the news release I was referencing, but yea it doesn't really support my comment :(
- 77pt77 2y ago
  > the federal government issued a security advisory Well, I guess that's it... Maybe rust will take the same path as Ada.
  - deleted 2y ago
    [deleted]
rtpg 2y ago
There's a bit of a higher abstraction ceiling in Rust so in theory if you are successful rewriting a thing in Rust then you now have a codebase that's easier to change confidently. This sort of property is nice to have in huge codebases where you really start losing confidence in shipping changes that don't subtly break things. But of course a huge codebase is hard to rewrite in general...
- taneq 2y ago
  Sounds like a perfect situation for a strangler pattern? Wrap or transpile the original code into a language with stronger refactoring support and the rest should become incrementally easier.
  - kvdveer 2y ago
    The advantages of rust only come when you actually use the rust-provides abstraction, especially those around allocation and concurrency. Even if transpiling is possible, the code would still not be structured I the rust way, and you wouldn't have any of the benefits. Same goes for wrapping.
- nsajko 2y ago
  > There's a bit of a higher abstraction ceiling in Rust Compared to C, yes, but not compared to C++.
  - tialaramex 2y ago
    This isn't really true. Rust has a much better type system. When writing generic code the impact is enormous. C++ doesn't have a real Empty Type, and it thinks Units have non-zero size. In practical terms this makes it incredibly wasteful and in terms of a clear abstraction it encourages you to come up with a hack that's unclear but efficient.
    - saagarjha 2y ago
      …or you can just waste a few bytes? It's not a big deal.
oguz-ismail 2y ago
> If a certain piece of software is working well, what is the benefit of rewriting it in Rust? One benefit is it's easier to hide malicious code thanks to Rust's complicated syntax.
- roca 2y ago
  It's far easier to hide malicious code in C or C++: just write some subtle undefined behavior that you can write an exploit against. Developers do that all the time even when they're not trying to be malicious. In Rust you'd have to wrap it in "unsafe" which draws attention.

khuey 2y ago

From the perspective of an rr maintainer, Sid's work was good and we were supportive of it. The main issues with migrating to that as the "blessed" version are that 1) rr has accumulated a decade of very hairy fixes for crazy kernel/process behavior that we feared could be lost during a port and 2) there's a closed source project (remix[0]) built on top of rr that would have needed to be ported too. [0] https://robert.ocallahan.org/2020/12/rr-remix-efficient-replay-only-binary.html https://robert.ocallahan.org/2020/12/rr-remix-efficient-repl...