31 ms·

Translating Quake 3 into Rust

456 points by K0nserv 7y ago

wyldfire 7y ago
I recall seeing a presentation of c2rust by Andrei at a conference a while back. It's great to hear that it's making good progress. I'm curious at how good c2rust is at producing idiomatic rust code. It seems like that would be difficult and yet very valuable. > we'd love to hear what you want to see translated next. qemu [1] is all in C (probably C89 or "gnu89") and would make an interesting project, IMO. [1] https://github.com/qemu/qemu https://github.com/qemu/qemu
- K0nserv 7y ago
  I looked for the transpiled source, but I couldn't find it. I suspect it's not particularly idomatic Rust since it has to maintain all details of the C code however I think it's a very promising project anyway. The fact that they found a memory safety bug is excellent. Maybe translating old C projects to Rust via this method can be used to find memory issues as well as to aid rewriting in Rust projects. EDIT: You can try the transpilation yourself on https://c2rust.com/ https://c2rust.com/ Here's an example of the output: #![allow(dead_code, mutable_transmutes, non_camel_case_types, non_snake_case, non_upper_case_globals, unused_assignments, unused_mut)] #![register_tool(c2rust)] #![feature(register_tool)] #[no_mangle] pub unsafe extern "C" fn insertion_sort(n: libc::c_int, p: *mut libc::c_int) { let mut i: libc::c_int = 1 as libc::c_int; while i < n { let tmp: libc::c_int = *p.offset(i as isize); let mut j: libc::c_int = i; while j > 0 as libc::c_int && *p.offset((j - 1 as libc::c_int) as isize) > tmp { *p.offset(j as isize) = *p.offset((j - 1 as libc::c_int) as isize); j -= 1 } *p.offset(j as isize) = tmp; i += 1 }; }
  - steveklabnik 7y ago
    Their plan is twofold, in my understanding: first to translate the code fairly literaly, and then second, to provide unsafe -> safe refactoring tools. As far as I know they’re still on step 1.
    - K0nserv 7y ago
      Cool, that seems like a very sensible approach. The fact that they are already uncovering memory issues in C is promising too.
      - steveklabnik 7y ago
        Yep, I think so too. That example output is more verbose than a human would write, but it’s a sensible start.
    - harpocrates 7y ago
      Step 2 has been happening concurrently, and pretty much since the start of the project. See https://github.com/immunant/c2rust/tree/master/c2rust-refactor https://github.com/immunant/c2rust/tree/master/c2rust-refact.... It is just a lot tougher to get right, so it gets less publicity.
      - steveklabnik 7y ago
        Ah awesome! Thanks for the link.
    - mikepurvis 7y ago
      Looks like it's meant to enable projects similar to how the Go compiler/toolchain was initially translated (with the like-named c2go) and then gradually rewritten over time into idiomatic golang. It's easy to see that this could be a much more tenable project then a grand rewrite that has two codebases iterating forward in parallel for a time.
- masklinn 7y ago
  > I'm curious at how good c2rust is at producing idiomatic rust code. It’s not. It should eventually improve but currently the goal is to > build a rock-solid translator that gets people up and running in Rust.
  - weberc2 7y ago
    What is the ceiling for how good (i.e., "close to generating idiomatic Rust") it could get? Given the differing semantics, it seems like it should be nearly impossible to generate anything close to idiomatic Rust, but I know very little about mechanically translating between programming languages and I would be delighted to be incorrect.
    - masklinn 7y ago
      I'd say "pretty low" on the idiomatic scale, however a very desirable thing would be for them to manage to make most of the generated code not unsafe. Limiting the scope of unsafe would make what would remain easier to convert to safe rust, and then you're off to the refactoring race with much more confidence. A pretty good go would be the ability to recognise wide-spread high-level C patterns and be able to convert them to idiomatic Rust pattern, but even that would be difficult, generally speaking idiomatic C and Rust would likely be structured quite differently, especially on the data(-structure) side.
    - deleted 7y ago
      [deleted]
- smolder 7y ago
  The long tail of patterns c2rust would need to recognize to reliably make idiomatic code seems like a much larger problem. With that as the goal, it might make sense to build a separate rust-to-rust "idiomaticizer" that can recognize opportunities to reduce code complexity through refactoring, and apply idiomatic patterns without changing application behavior.
  - PudgePacket 7y ago
    Clippy already sort is is that, though it seems to target smaller scope idiomatic changes. It's a much harder problem for a tool to recognise the design patterns C pushes you towards and refactor that to something more native to Rust.
  - bluejekyll 7y ago
    Maybe. Though there might be lowish hanging fruit that could potentially recognize arrays that could be translated to slices, and heap based allocations using pointer traversal that could be converted to Vec instead. The real issue with doing that isn’t imo, going to Rust, it’s understanding what still needs to be addressable in C. For example, a self-contained application has no external ABI dependencies, and so could assume everything that references that slice must only need to be Rust. A library on the other-hand might still need to present a C ABI.
  - harpocrates 7y ago
    As I've already posted elsewhere on this thread, such a tool has been developed in parallel (and with the intent to consume the output of) c2rust. Development on that occurs here: https://github.com/immunant/c2rust/tree/master/c2rust-refactor https://github.com/immunant/c2rust/tree/master/c2rust-refact....
    - smolder 7y ago
      Thanks, I hadn't seen this by the time I posted. Very cool.
- flohofwoe 7y ago
  Here's an example from another code base, the Linux (GLX+X11+GL) extract of an 3D API wrapper around OpenGL: https://github.com/not-fl3/miniquad/blob/c2rust/sokol-app-sys/src/sokol_app_linux.rs https://github.com/not-fl3/miniquad/blob/c2rust/sokol-app-sy... The generated code looks fairly straightforward, however I wouldn't call it "idiomatic". What's exciting though is that (at least) both Rust and Zig get this ability to translate C code into the "native" language (and sometimes even finding bugs in the process). This basically turns C into a "meta-language" to create "bedrock" cross-platform API modules for a variety of modern languages, essentially the next step from "C as a cross-platform assembler". And to be honest, this sort of "boring" low-level system API glue code which mainly talks to native C system APIs anyway isn't a joy to write in any language, and also doesn't benefit much from more modern language features, can just as well do it in C once, and then "transpile everywhere".
  - dodobirdlord 7y ago
    This seems pretty contrary to the purpose of Rust, since the generated code is full of non-idiomatic unsafe and possibly situationally unsound pointer manipulations that could definitely be done in idiomatic safe code. No reason I can see not to just write this sort of thing in Rust to begin with and expose it to other languages via the C ABI to call with FFI or even co-compile with other LLVM languages.
    - flohofwoe 7y ago
      Yeah theoretically, but then you have the same problem as using C libraries from Rust, the build process for those other languages suddenly becomes much more complicated because they must integrate with another language toolchain. Lets say I want to expose a Zig library to Rust or vice versa, now I must call the Zig compiler from Cargo, or Cargo from the Zig build process. With a transpiled library such a project is "clean", it only needs the Rust or Zig toolchain. The transpiling process only needs to happen once, and this can happen automatically on a CI service when the base library is updated and then made available automatically through the language's package repository. Rust probably won't get the ability to transpile to Zig, and Zig probably won't be able to transpile to Rust. Some sort of Lingua Franca makes sense in a multi-language world, and C (or even a subset of C) makes sense for that because it already is the Lingua France, and has a very small feature set to consider. I'm aware that this is a controversial opinion though ;)
- ZiiS 7y ago
  I would highly value something which turned code into idiomatic code; I don't care which language.
  - misterdata 7y ago
    Although not automatic, I find Rust's clippy to be very good at suggesting improvements to code that compiles but simply is not written the way it should.
kstenerud 7y ago
That's super cool! What I'd really like to see is benchmarks that compare the rust version to C.
- steveklabnik 7y ago
  Benchmarks would be interesting, in the sense that it would help make sure that the translation is doing the same thing, but it wouldn't say a ton about Rust vs C in a more general case. This translates to unsafe code, which means it's not really representative of Rust generally.
- AdmiralAsshat 7y ago
  What is the realistic best-case scenario? Has anyone demonstrated that well-written Rust code should run faster than C, or is the hope simply to get performance parity with C while maintaining memory safety?
  - K0nserv 7y ago
    Yes, the most recent example I can think of is http://cliffle.com/p/dangerust/ http://cliffle.com/p/dangerust/ in which the author translates heavily optimized C to Rust in a quite mechanical way. It comes out being slightly faster. In the last part they rewrite it in idomatic Rust and it ends up even faster again without any of the bespoke optimizations that the C code used or any unsafe Rust. In many instances the Rust type system should allow the compiler to emit faster code because it has more data to use for reasoning about valid and invalid states of the program. Depending on how good c2rust is the Rust version should be roughly as fast, refactored to idomatic Rust it could be meaningfully faster than C.
  - steveklabnik 7y ago
    This topic is really tricky, and there's a lot of different ways to slice it. In the most general sense, if you take a C expert, and a Rust expert, and ask them to do something, they should be able to get the exact same results. But that leaves out other interesting questions that inform this comparison. For example: * What about the average Rust programmer vs the average C programmer? Maybe two experts can produce identical results, but does one language vs the other make it so that a non-expert tends to produce faster code? * What about "no unsafe Rust other than what's in the implementation of the standard library"? Most Rust code doesn't use unsafe, so is it fair to let the Rust use it? How much? * What about "no code other than the standard libraries." I may not be an expert, but Rust's package ecosystem makes it easier to use code that's developed by them. This is sort of a slightly different take on the two previous questions at the same time. * Should there be a time limit on development? If so, should the C have the same requirements of demonstrating safety as the Rust, that is, sketching out an informal proof of correctness? * What about some sort of "how maintainable is this" metric. Mozilla tried to implement parallel CSS in C++ twice, and couldn't do it, because the concurrency bugs were just too many. Their Rust attempt succeeded, thanks to the safety guidelines. Is this comparison valid? Or could some sort of person who was better at C++ have implemented it in theory with no bugs? (this is sort of question 1, but slightly different.) Lots and lots of ways to do this.
bbmario 7y ago
Is the final code readable/maintainable?
- steveklabnik 7y ago
  There's an example of the output and a discussion of this elsewhere in this thread. TL;DR: not as much as it could be, yet. This is a work in progress.
- edflsafoiewq 7y ago
  No.
- ahomescu1 7y ago
  We added the transpiled code in a new branch at https://github.com/immunant/ioq3/tree/transpiled/quake3-rs https://github.com/immunant/ioq3/tree/transpiled/quake3-rs and also a more cleaned up refactored code base at https://github.com/immunant/ioq3/tree/refactored/quake3-rs https://github.com/immunant/ioq3/tree/refactored/quake3-rs
jtaft 7y ago
This is some awesome PR too! Didn't expect them to be a software security consultancy.
- steveklabnik 7y ago
  Some of this work was even funded by DARPA...
johnklos 7y ago
Why is there such a push to use this new language that's so inaccessible to much of the world? Not everyone has a multi-gigahertz, multi-core, multi-gigabyte computer to develop in Rust. It seems like unnecessary stratification for nebulous gain. Perhaps the Rust team should make their compiler work on reasonable resources, like 32 bit processors, before marching off to try to convert the world. Or perhaps we should have rust2c instead, so people can take all this Rust code that everyone is pushing and compile it on modest machines.
- steveklabnik 7y ago
  We distribute binaries of the compiler for a bunch of 32 bit targets. It will of course be slower on slower machines, but that’s just physics. mrustc is "rust to C", though it's not quite ready for the general case yet. It's primarily for translating the compiler, to ease bootstrapping.
- twblalock 7y ago
  > Perhaps the Rust team should make their compiler work on reasonable resources, like 32 bit processors, before marching off to try to convert the world. It's time to let 32-bit processors go and move on.
  - deleted 7y ago
    [deleted]
  - AceJohnny2 7y ago
    There are magnitudes more processors out there than desktop- or server-class.
    - tverbeure 7y ago
      How many of those processors are used to compile code? The complaint above is about that. Running compiled Rust code on a tiny embedded microcontroller is not a problem. https://medium.com/coinmonks/coding-the-stm32-blue-pill-with-rust-and-visual-studio-code-b21615d8a20 https://medium.com/coinmonks/coding-the-stm32-blue-pill-with...
deleted 7y ago
[deleted]
floatingatoll 7y ago
One highlight of this post is their work exposing a memory safety error in the original source code: > The Rust compiler also flagged a similar but actually buggy example in G_TryPushingEntity where the conditional is >, not >=. The out of bounds pointer was then dereferenced after the conditional, which is an actual memory safety bug.
- Avamander 7y ago
  Interestingly I've too found bugs in original source similar to this when rewriting in just TypeScript, so I suspect it's just someone taking a look that allows the bug to be discovered.
  - jzoch 7y ago
    This is a bit different since it isn't "someone" discovering this bug but the compiler
  - ShorsHammer 7y ago
    Sourcecode?
    - floatingatoll 7y ago
      Linked from that paragraph in the article.
      - vijaybritto 7y ago
        I think they are asking for the Typescript source
deleted 7y ago
[deleted]
willvarfar 7y ago
It would be really cool to see snippets or even the whole original and transpiled code side-by-side. When I was working on compilers I spent as much time making such a script in a web-browser with scrolling etc as I did studying the results.
- saldivarcher 7y ago
  https://c2rust.com https://c2rust.com will show you side by side comparisons, although it isn't the best looking code; in terms of readability. But as mentioned in other threads, the refactoring tool (https://github.com/immunant/c2rust/tree/master/c2rust-refactor https://github.com/immunant/c2rust/tree/master/c2rust-refact...) aims to fix that.
- ahomescu1 7y ago
  We added the transpiled code in a new branch at https://github.com/immunant/ioq3/tree/transpiled/quake3-rs https://github.com/immunant/ioq3/tree/transpiled/quake3-rs and also a more cleaned up refactored code base at https://github.com/immunant/ioq3/tree/refactored/quake3-rs https://github.com/immunant/ioq3/tree/refactored/quake3-rs
giovannibajo1 7y ago
Out of curiosity, how do compilation times look, for a fresh build and for the edit-compile cycle?
faebi 7y ago
Does it mean we could translate languages like Ruby and Python to Rust?
- johannes1234321 7y ago
  Technically it is possible. The gain won't be there for most cases, though as the generated code would be complex and bad as variables often have to be carried as "variant" types and you have to plug in to the library (which has a specific C API) and so on. Facebook for some time did this with their "HipHop for PHP" transpiler which translates PHP into C++ and then fed it to gcc. That worked, but due to all the type conversion and similar magic required on all calls this lead them to compilation times of 24 hours and interfaces which were hard to interact with for language bindings, thus they went back to write a scripting engine (HHVM) which now runs their forked PHP.
- K0nserv 7y ago
  While it might be possible my guess is it would be much more difficult than C. AFAIK Rust is more or less a superset of C whereas Python and Ruby have numerous advanced features that aren't present in Rust and would likely require a bespoke Rust runtime to emulate the behaviour. Types is also a whole can of worms here. I think in relation to Ruby, Python and other languages like them Rust's best application is leveraging the support for C FFIs to replace hot paths. While Rust has a significant learning curve it's probably much simpler, for say a ruby programmer, to pick it up and write safe performant code than C.
  - jashmatthews 7y ago
    There's an effort to write a Ruby implementation in Rust: https://github.com/artichoke/artichoke https://github.com/artichoke/artichoke At the moment it wraps the mruby VM, but I think Ryan's plan is to switch to the CRuby YARV VM and then replace it with a VM written in Rust. I'm also working on a CRuby JIT compiler written in Rust using CraneLift (a new compiler written in Rust).
- jashmatthews 7y ago
  CRuby's JIT basically works by translating Ruby to C.
arendtio 7y ago
I would love to see them translating the Linux kernel, but that might be a bit too ambitious.
tombert 7y ago
Out of curiosity, has anyone here tried taking something like Quake 2 or 3 and porting it to run on the JVM? This project kind of makes me want to try porting Quake 3 into Clojure, but I want to know the limitations in doing so.
- cs02rm0 7y ago
  I'd imagine the graphics libraries is where the JVM falls down.
  - flatiron 7y ago
    Not that Minecraft runs well or is a marvel of graphical technology but at least it proves you can do FPS 3D with the JVM enough to the the most popular game around.
    - phit_ 7y ago
      Minecraft uses https://www.lwjgl.org/ https://www.lwjgl.org/
  - agumonkey 7y ago
    Cant one bind to opengl directly ?
    - p1necone 7y ago
      Yes you can. Java works fine for even kinda high end 3d graphics, not many people do it though. The biggest thing in java gamedev is avoiding GC pauses.
      - tombert 7y ago
        I would think that with the newer GCs (like Shenandoah) for Java, the GC pause thing might become a non-issue soon enough. Not sure that works for Clojure though.
bookAlot 7y ago
Is it different or have the commands made a change to the memory mapping for it to be read only at least?
ahomescu1 7y ago
Author here: I wanted to let everyone know we made some small edits to the text, and added two new branches to our repository: transpiled containing the raw transpiler output [1], and refactored containing the same code [2] after a few refactoring commands. 1. https://github.com/immunant/ioq3/tree/transpiled/quake3-rs https://github.com/immunant/ioq3/tree/transpiled/quake3-rs 2. https://github.com/immunant/ioq3/tree/refactored/quake3-rs https://github.com/immunant/ioq3/tree/refactored/quake3-rs
- phkahler 7y ago
  Is there any progress on C++ to Rust? I know the whole language is huge, so maybe even a little support could be opening a can of worms.
  - simcop2387 7y ago
    Most of the language probably isn't too far from C to be done, but I can't even guess at what would have to be done for C++'s templates.
    - zozbot234 7y ago
      The closest Rusty feature to C++ templates is most likely macros. I don't think even const generics, specialization etc. (which are WIP anyway at the moment) would be enough to replicate templates in the fully general case.
      - heavenlyblue 7y ago
        I most surely misunderstand something: why are templates not replaceable with Rust generics?
        mathw 7y ago
        The simple answer is that C++ templates can do more than Rust generics can, including some things that in Rust you'd use the macro system for, and probably some things that you'd have to translate by expanding out the template and turning that result into Rust.
Waffle5 7y ago
Would be interested in a performance comparison - c (using opengl1, etc etc) vs c2rust.

456 points by K0nserv 7y ago

I recall seeing a presentation of c2rust by Andrei at a conference a while back. It's great to hear that it's making good progress. I'm curious at how good c2rust is at producing idiomatic rust code. It seems like that would be difficult and yet very valuable. > we'd love to hear what you want to see translated next. qemu [1] is all in C (probably C89 or "gnu89") and would make an interesting project, IMO. [1] https://github.com/qemu/qemu https://github.com/qemu/qemu

K0nserv 7y ago
I looked for the transpiled source, but I couldn't find it. I suspect it's not particularly idomatic Rust since it has to maintain all details of the C code however I think it's a very promising project anyway. The fact that they found a memory safety bug is excellent. Maybe translating old C projects to Rust via this method can be used to find memory issues as well as to aid rewriting in Rust projects. EDIT: You can try the transpilation yourself on https://c2rust.com/ https://c2rust.com/ Here's an example of the output: #![allow(dead_code, mutable_transmutes, non_camel_case_types, non_snake_case, non_upper_case_globals, unused_assignments, unused_mut)] #![register_tool(c2rust)] #![feature(register_tool)] #[no_mangle] pub unsafe extern "C" fn insertion_sort(n: libc::c_int, p: *mut libc::c_int) { let mut i: libc::c_int = 1 as libc::c_int; while i < n { let tmp: libc::c_int = *p.offset(i as isize); let mut j: libc::c_int = i; while j > 0 as libc::c_int && *p.offset((j - 1 as libc::c_int) as isize) > tmp { *p.offset(j as isize) = *p.offset((j - 1 as libc::c_int) as isize); j -= 1 } *p.offset(j as isize) = tmp; i += 1 }; }
- steveklabnik 7y ago
  Their plan is twofold, in my understanding: first to translate the code fairly literaly, and then second, to provide unsafe -> safe refactoring tools. As far as I know they’re still on step 1.
  - K0nserv 7y ago
    Cool, that seems like a very sensible approach. The fact that they are already uncovering memory issues in C is promising too.
    - steveklabnik 7y ago
      Yep, I think so too. That example output is more verbose than a human would write, but it’s a sensible start.
  - harpocrates 7y ago
    Step 2 has been happening concurrently, and pretty much since the start of the project. See https://github.com/immunant/c2rust/tree/master/c2rust-refactor https://github.com/immunant/c2rust/tree/master/c2rust-refact.... It is just a lot tougher to get right, so it gets less publicity.
    - steveklabnik 7y ago
      Ah awesome! Thanks for the link.
  - mikepurvis 7y ago
    Looks like it's meant to enable projects similar to how the Go compiler/toolchain was initially translated (with the like-named c2go) and then gradually rewritten over time into idiomatic golang. It's easy to see that this could be a much more tenable project then a grand rewrite that has two codebases iterating forward in parallel for a time.
masklinn 7y ago
> I'm curious at how good c2rust is at producing idiomatic rust code. It’s not. It should eventually improve but currently the goal is to > build a rock-solid translator that gets people up and running in Rust.
- weberc2 7y ago
  What is the ceiling for how good (i.e., "close to generating idiomatic Rust") it could get? Given the differing semantics, it seems like it should be nearly impossible to generate anything close to idiomatic Rust, but I know very little about mechanically translating between programming languages and I would be delighted to be incorrect.
  - masklinn 7y ago
    I'd say "pretty low" on the idiomatic scale, however a very desirable thing would be for them to manage to make most of the generated code not unsafe. Limiting the scope of unsafe would make what would remain easier to convert to safe rust, and then you're off to the refactoring race with much more confidence. A pretty good go would be the ability to recognise wide-spread high-level C patterns and be able to convert them to idiomatic Rust pattern, but even that would be difficult, generally speaking idiomatic C and Rust would likely be structured quite differently, especially on the data(-structure) side.
  - deleted 7y ago
    [deleted]
smolder 7y ago
The long tail of patterns c2rust would need to recognize to reliably make idiomatic code seems like a much larger problem. With that as the goal, it might make sense to build a separate rust-to-rust "idiomaticizer" that can recognize opportunities to reduce code complexity through refactoring, and apply idiomatic patterns without changing application behavior.
- PudgePacket 7y ago
  Clippy already sort is is that, though it seems to target smaller scope idiomatic changes. It's a much harder problem for a tool to recognise the design patterns C pushes you towards and refactor that to something more native to Rust.
- bluejekyll 7y ago
  Maybe. Though there might be lowish hanging fruit that could potentially recognize arrays that could be translated to slices, and heap based allocations using pointer traversal that could be converted to Vec instead. The real issue with doing that isn’t imo, going to Rust, it’s understanding what still needs to be addressable in C. For example, a self-contained application has no external ABI dependencies, and so could assume everything that references that slice must only need to be Rust. A library on the other-hand might still need to present a C ABI.
- harpocrates 7y ago
  As I've already posted elsewhere on this thread, such a tool has been developed in parallel (and with the intent to consume the output of) c2rust. Development on that occurs here: https://github.com/immunant/c2rust/tree/master/c2rust-refactor https://github.com/immunant/c2rust/tree/master/c2rust-refact....
  - smolder 7y ago
    Thanks, I hadn't seen this by the time I posted. Very cool.
flohofwoe 7y ago
Here's an example from another code base, the Linux (GLX+X11+GL) extract of an 3D API wrapper around OpenGL: https://github.com/not-fl3/miniquad/blob/c2rust/sokol-app-sys/src/sokol_app_linux.rs https://github.com/not-fl3/miniquad/blob/c2rust/sokol-app-sy... The generated code looks fairly straightforward, however I wouldn't call it "idiomatic". What's exciting though is that (at least) both Rust and Zig get this ability to translate C code into the "native" language (and sometimes even finding bugs in the process). This basically turns C into a "meta-language" to create "bedrock" cross-platform API modules for a variety of modern languages, essentially the next step from "C as a cross-platform assembler". And to be honest, this sort of "boring" low-level system API glue code which mainly talks to native C system APIs anyway isn't a joy to write in any language, and also doesn't benefit much from more modern language features, can just as well do it in C once, and then "transpile everywhere".
- dodobirdlord 7y ago
  This seems pretty contrary to the purpose of Rust, since the generated code is full of non-idiomatic unsafe and possibly situationally unsound pointer manipulations that could definitely be done in idiomatic safe code. No reason I can see not to just write this sort of thing in Rust to begin with and expose it to other languages via the C ABI to call with FFI or even co-compile with other LLVM languages.
  - flohofwoe 7y ago
    Yeah theoretically, but then you have the same problem as using C libraries from Rust, the build process for those other languages suddenly becomes much more complicated because they must integrate with another language toolchain. Lets say I want to expose a Zig library to Rust or vice versa, now I must call the Zig compiler from Cargo, or Cargo from the Zig build process. With a transpiled library such a project is "clean", it only needs the Rust or Zig toolchain. The transpiling process only needs to happen once, and this can happen automatically on a CI service when the base library is updated and then made available automatically through the language's package repository. Rust probably won't get the ability to transpile to Zig, and Zig probably won't be able to transpile to Rust. Some sort of Lingua Franca makes sense in a multi-language world, and C (or even a subset of C) makes sense for that because it already is the Lingua France, and has a very small feature set to consider. I'm aware that this is a controversial opinion though ;)
ZiiS 7y ago
I would highly value something which turned code into idiomatic code; I don't care which language.
- misterdata 7y ago
  Although not automatic, I find Rust's clippy to be very good at suggesting improvements to code that compiles but simply is not written the way it should.

kstenerud 7y ago

That's super cool! What I'd really like to see is benchmarks that compare the rust version to C.

steveklabnik 7y ago
Benchmarks would be interesting, in the sense that it would help make sure that the translation is doing the same thing, but it wouldn't say a ton about Rust vs C in a more general case. This translates to unsafe code, which means it's not really representative of Rust generally.
AdmiralAsshat 7y ago
What is the realistic best-case scenario? Has anyone demonstrated that well-written Rust code should run faster than C, or is the hope simply to get performance parity with C while maintaining memory safety?
- K0nserv 7y ago
  Yes, the most recent example I can think of is http://cliffle.com/p/dangerust/ http://cliffle.com/p/dangerust/ in which the author translates heavily optimized C to Rust in a quite mechanical way. It comes out being slightly faster. In the last part they rewrite it in idomatic Rust and it ends up even faster again without any of the bespoke optimizations that the C code used or any unsafe Rust. In many instances the Rust type system should allow the compiler to emit faster code because it has more data to use for reasoning about valid and invalid states of the program. Depending on how good c2rust is the Rust version should be roughly as fast, refactored to idomatic Rust it could be meaningfully faster than C.
- steveklabnik 7y ago
  This topic is really tricky, and there's a lot of different ways to slice it. In the most general sense, if you take a C expert, and a Rust expert, and ask them to do something, they should be able to get the exact same results. But that leaves out other interesting questions that inform this comparison. For example: * What about the average Rust programmer vs the average C programmer? Maybe two experts can produce identical results, but does one language vs the other make it so that a non-expert tends to produce faster code? * What about "no unsafe Rust other than what's in the implementation of the standard library"? Most Rust code doesn't use unsafe, so is it fair to let the Rust use it? How much? * What about "no code other than the standard libraries." I may not be an expert, but Rust's package ecosystem makes it easier to use code that's developed by them. This is sort of a slightly different take on the two previous questions at the same time. * Should there be a time limit on development? If so, should the C have the same requirements of demonstrating safety as the Rust, that is, sketching out an informal proof of correctness? * What about some sort of "how maintainable is this" metric. Mozilla tried to implement parallel CSS in C++ twice, and couldn't do it, because the concurrency bugs were just too many. Their Rust attempt succeeded, thanks to the safety guidelines. Is this comparison valid? Or could some sort of person who was better at C++ have implemented it in theory with no bugs? (this is sort of question 1, but slightly different.) Lots and lots of ways to do this.

bbmario 7y ago

Is the final code readable/maintainable?

steveklabnik 7y ago
There's an example of the output and a discussion of this elsewhere in this thread. TL;DR: not as much as it could be, yet. This is a work in progress.
edflsafoiewq 7y ago
No.
ahomescu1 7y ago
We added the transpiled code in a new branch at https://github.com/immunant/ioq3/tree/transpiled/quake3-rs https://github.com/immunant/ioq3/tree/transpiled/quake3-rs and also a more cleaned up refactored code base at https://github.com/immunant/ioq3/tree/refactored/quake3-rs https://github.com/immunant/ioq3/tree/refactored/quake3-rs

jtaft 7y ago

This is some awesome PR too! Didn't expect them to be a software security consultancy.

steveklabnik 7y ago
Some of this work was even funded by DARPA...

johnklos 7y ago

Why is there such a push to use this new language that's so inaccessible to much of the world? Not everyone has a multi-gigahertz, multi-core, multi-gigabyte computer to develop in Rust. It seems like unnecessary stratification for nebulous gain. Perhaps the Rust team should make their compiler work on reasonable resources, like 32 bit processors, before marching off to try to convert the world. Or perhaps we should have rust2c instead, so people can take all this Rust code that everyone is pushing and compile it on modest machines.

steveklabnik 7y ago
We distribute binaries of the compiler for a bunch of 32 bit targets. It will of course be slower on slower machines, but that’s just physics. mrustc is "rust to C", though it's not quite ready for the general case yet. It's primarily for translating the compiler, to ease bootstrapping.
twblalock 7y ago
> Perhaps the Rust team should make their compiler work on reasonable resources, like 32 bit processors, before marching off to try to convert the world. It's time to let 32-bit processors go and move on.
- deleted 7y ago
  [deleted]
- AceJohnny2 7y ago
  There are magnitudes more processors out there than desktop- or server-class.
  - tverbeure 7y ago
    How many of those processors are used to compile code? The complaint above is about that. Running compiled Rust code on a tiny embedded microcontroller is not a problem. https://medium.com/coinmonks/coding-the-stm32-blue-pill-with-rust-and-visual-studio-code-b21615d8a20 https://medium.com/coinmonks/coding-the-stm32-blue-pill-with...

deleted 7y ago

[deleted]

floatingatoll 7y ago

One highlight of this post is their work exposing a memory safety error in the original source code: > The Rust compiler also flagged a similar but actually buggy example in G_TryPushingEntity where the conditional is >, not >=. The out of bounds pointer was then dereferenced after the conditional, which is an actual memory safety bug.

Avamander 7y ago
Interestingly I've too found bugs in original source similar to this when rewriting in just TypeScript, so I suspect it's just someone taking a look that allows the bug to be discovered.
- jzoch 7y ago
  This is a bit different since it isn't "someone" discovering this bug but the compiler
- ShorsHammer 7y ago
  Sourcecode?
  - floatingatoll 7y ago
    Linked from that paragraph in the article.
    - vijaybritto 7y ago
      I think they are asking for the Typescript source

deleted 7y ago

[deleted]

willvarfar 7y ago

It would be really cool to see snippets or even the whole original and transpiled code side-by-side. When I was working on compilers I spent as much time making such a script in a web-browser with scrolling etc as I did studying the results.

saldivarcher 7y ago
https://c2rust.com https://c2rust.com will show you side by side comparisons, although it isn't the best looking code; in terms of readability. But as mentioned in other threads, the refactoring tool (https://github.com/immunant/c2rust/tree/master/c2rust-refactor https://github.com/immunant/c2rust/tree/master/c2rust-refact...) aims to fix that.
ahomescu1 7y ago
We added the transpiled code in a new branch at https://github.com/immunant/ioq3/tree/transpiled/quake3-rs https://github.com/immunant/ioq3/tree/transpiled/quake3-rs and also a more cleaned up refactored code base at https://github.com/immunant/ioq3/tree/refactored/quake3-rs https://github.com/immunant/ioq3/tree/refactored/quake3-rs

giovannibajo1 7y ago

Out of curiosity, how do compilation times look, for a fresh build and for the edit-compile cycle?

faebi 7y ago

Does it mean we could translate languages like Ruby and Python to Rust?

johannes1234321 7y ago
Technically it is possible. The gain won't be there for most cases, though as the generated code would be complex and bad as variables often have to be carried as "variant" types and you have to plug in to the library (which has a specific C API) and so on. Facebook for some time did this with their "HipHop for PHP" transpiler which translates PHP into C++ and then fed it to gcc. That worked, but due to all the type conversion and similar magic required on all calls this lead them to compilation times of 24 hours and interfaces which were hard to interact with for language bindings, thus they went back to write a scripting engine (HHVM) which now runs their forked PHP.
K0nserv 7y ago
While it might be possible my guess is it would be much more difficult than C. AFAIK Rust is more or less a superset of C whereas Python and Ruby have numerous advanced features that aren't present in Rust and would likely require a bespoke Rust runtime to emulate the behaviour. Types is also a whole can of worms here. I think in relation to Ruby, Python and other languages like them Rust's best application is leveraging the support for C FFIs to replace hot paths. While Rust has a significant learning curve it's probably much simpler, for say a ruby programmer, to pick it up and write safe performant code than C.
- jashmatthews 7y ago
  There's an effort to write a Ruby implementation in Rust: https://github.com/artichoke/artichoke https://github.com/artichoke/artichoke At the moment it wraps the mruby VM, but I think Ryan's plan is to switch to the CRuby YARV VM and then replace it with a VM written in Rust. I'm also working on a CRuby JIT compiler written in Rust using CraneLift (a new compiler written in Rust).
jashmatthews 7y ago
CRuby's JIT basically works by translating Ruby to C.

arendtio 7y ago

I would love to see them translating the Linux kernel, but that might be a bit too ambitious.

tombert 7y ago

Out of curiosity, has anyone here tried taking something like Quake 2 or 3 and porting it to run on the JVM? This project kind of makes me want to try porting Quake 3 into Clojure, but I want to know the limitations in doing so.

cs02rm0 7y ago
I'd imagine the graphics libraries is where the JVM falls down.
- flatiron 7y ago
  Not that Minecraft runs well or is a marvel of graphical technology but at least it proves you can do FPS 3D with the JVM enough to the the most popular game around.
  - phit_ 7y ago
    Minecraft uses https://www.lwjgl.org/ https://www.lwjgl.org/
- agumonkey 7y ago
  Cant one bind to opengl directly ?
  - p1necone 7y ago
    Yes you can. Java works fine for even kinda high end 3d graphics, not many people do it though. The biggest thing in java gamedev is avoiding GC pauses.
    - tombert 7y ago
      I would think that with the newer GCs (like Shenandoah) for Java, the GC pause thing might become a non-issue soon enough. Not sure that works for Clojure though.

bookAlot 7y ago

Is it different or have the commands made a change to the memory mapping for it to be read only at least?

ahomescu1 7y ago

Author here: I wanted to let everyone know we made some small edits to the text, and added two new branches to our repository: transpiled containing the raw transpiler output [1], and refactored containing the same code [2] after a few refactoring commands. 1. https://github.com/immunant/ioq3/tree/transpiled/quake3-rs https://github.com/immunant/ioq3/tree/transpiled/quake3-rs 2. https://github.com/immunant/ioq3/tree/refactored/quake3-rs https://github.com/immunant/ioq3/tree/refactored/quake3-rs

phkahler 7y ago
Is there any progress on C++ to Rust? I know the whole language is huge, so maybe even a little support could be opening a can of worms.
- simcop2387 7y ago
  Most of the language probably isn't too far from C to be done, but I can't even guess at what would have to be done for C++'s templates.
  - zozbot234 7y ago
    The closest Rusty feature to C++ templates is most likely macros. I don't think even const generics, specialization etc. (which are WIP anyway at the moment) would be enough to replicate templates in the fully general case.
    - heavenlyblue 7y ago
      I most surely misunderstand something: why are templates not replaceable with Rust generics?
      - mathw 7y ago
        The simple answer is that C++ templates can do more than Rust generics can, including some things that in Rust you'd use the macro system for, and probably some things that you'd have to translate by expanding out the template and turning that result into Rust.

Waffle5 7y ago

Would be interested in a performance comparison - c (using opengl1, etc etc) vs c2rust.