3 ms·

But it's not in C's standard library. So the exercise isn't merely to auto-translate one language's standard library to another language's standard library (say

But it's not in C's standard library. So the exercise isn't merely to auto-translate one language's standard library to another language's standard library (say, replacing C++ std::list with Rust LinkedList) — which would already be very hard. The exercise here is to auto-identify-and-refactor idioms open-coded in one language, into idioms suited for the other language's already-written standard library. Imagine refactoring your average C program to use GLib for all (all!) of its data structures. Now imagine doing that, but also translating it into Rust at the same time.

Animats 2y ago
> The exercise here is to auto-identify-and-refactor idioms open-coded in one language, into idioms suited for the other language's already-written standard library. That's what LLMs are for - idiom translation. You can't trust them to do it right, though. [Pan et al . 2024] find that while GPT-4 generates code that is more idiomatic than C2Rust, only 61% of it is correct (i.e., compiles and produces the expected result), compared to 95% for C2Rust. This problem needs both AI-type methods to help with the idioms and formal methods to insure that the guessed idioms correctly capture the semantics. A big advance in this project is that they can usually translate C pointer arithmetic into Rust slices. That's progress on of one of the hardest parts of the problem. C2Rust did not do that. That system just generates unsafe raw pointer arithmetic, yielding ugly Rust code that replicates C pointer semantics using function calls. DARPA is funding research in this area under the TRACTOR program. Program awards in April 2025, so this is just getting started. It's encouraging to see so much progress already. This looks do-able.
- saghm 2y ago
  Oh god, I can't even imagine trying to have formally-verified LLM-generated code. It's not surprising that even incremental progress for that would require quite a lot of ingenuity.
  - deleted 2y ago
    [deleted]
- fuhsnn 2y ago
  >That's what LLMs are for - idiom translation. You can't trust them to do it right, though. Optimizing C compilers also happened to be good at idiom recognition, and we can probably trust them a little more. The OP paper does mention future plan to use clang as well: >We have plans for a libclang-based frontend that consume actual C syntax. If such transformation can be done at IR level it might be more efficient to be to C-IR > idiom transform to Rust-IR > run safe-checks in Rust-IR > continue compilation in C-IR or Rust-IR or combining both for better optimization properties.
  - swiftcoder 2y ago
    I'm definitely bullish on this angle of compiling C down to LLVM assembly, and then "decompiling" it back to Rust (with some reference to the original C to reconstruct high-level idioms like for loops)
- immibis 2y ago
  Actually, LLMs are for generating humorous nonsense. Putting them in charge of the world economy was not intended, but we did it anyway.
  - dhosek 2y ago
    Given that in my (small, employer-mandated) explorations with Copilot autocompletions it’s offered incorrect suggestions about a third of the time and seems to like to also suggest deprecated APIs, I’m skeptical about the current generation’s ability to be useful at even this small task.
    - glouwbug 2y ago
      Sure but it takes two copilots to fly a plane
    - LeFantome 2y ago
      Have you seen O3? If your experience with something less than half as good as state-of-the-art is that it worked 66% of the time, I am not sure why you would be so dismissive about the future potential.
- CodesInChaos 2y ago
  Why does C2Rust produce so much incorrect code? Getting 5% wrong sounds terrible, for a 1:1 translation to unsafe Rust. What does it mis-translate? https://dl.acm.org/doi/pdf/10.1145/3597503.3639226 https://dl.acm.org/doi/pdf/10.1145/3597503.3639226 > As for C2Rust, the 5% unsuccessful translations were due to compilation errors, the majority of them caused by unused imports. I'm rather confused by what that's supposed to mean, since unused imports cause warnings, not errors in Rust.

Animats 2y ago

> The exercise here is to auto-identify-and-refactor idioms open-coded in one language, into idioms suited for the other language's already-written standard library. That's what LLMs are for - idiom translation. You can't trust them to do it right, though. [Pan et al . 2024] find that while GPT-4 generates code that is more idiomatic than C2Rust, only 61% of it is correct (i.e., compiles and produces the expected result), compared to 95% for C2Rust. This problem needs both AI-type methods to help with the idioms and formal methods to insure that the guessed idioms correctly capture the semantics. A big advance in this project is that they can usually translate C pointer arithmetic into Rust slices. That's progress on of one of the hardest parts of the problem. C2Rust did not do that. That system just generates unsafe raw pointer arithmetic, yielding ugly Rust code that replicates C pointer semantics using function calls. DARPA is funding research in this area under the TRACTOR program. Program awards in April 2025, so this is just getting started. It's encouraging to see so much progress already. This looks do-able.

saghm 2y ago
Oh god, I can't even imagine trying to have formally-verified LLM-generated code. It's not surprising that even incremental progress for that would require quite a lot of ingenuity.
- deleted 2y ago
  [deleted]
fuhsnn 2y ago
>That's what LLMs are for - idiom translation. You can't trust them to do it right, though. Optimizing C compilers also happened to be good at idiom recognition, and we can probably trust them a little more. The OP paper does mention future plan to use clang as well: >We have plans for a libclang-based frontend that consume actual C syntax. If such transformation can be done at IR level it might be more efficient to be to C-IR > idiom transform to Rust-IR > run safe-checks in Rust-IR > continue compilation in C-IR or Rust-IR or combining both for better optimization properties.
- swiftcoder 2y ago
  I'm definitely bullish on this angle of compiling C down to LLVM assembly, and then "decompiling" it back to Rust (with some reference to the original C to reconstruct high-level idioms like for loops)
immibis 2y ago
Actually, LLMs are for generating humorous nonsense. Putting them in charge of the world economy was not intended, but we did it anyway.
- dhosek 2y ago
  Given that in my (small, employer-mandated) explorations with Copilot autocompletions it’s offered incorrect suggestions about a third of the time and seems to like to also suggest deprecated APIs, I’m skeptical about the current generation’s ability to be useful at even this small task.
  - glouwbug 2y ago
    Sure but it takes two copilots to fly a plane
  - LeFantome 2y ago
    Have you seen O3? If your experience with something less than half as good as state-of-the-art is that it worked 66% of the time, I am not sure why you would be so dismissive about the future potential.
CodesInChaos 2y ago
Why does C2Rust produce so much incorrect code? Getting 5% wrong sounds terrible, for a 1:1 translation to unsafe Rust. What does it mis-translate? https://dl.acm.org/doi/pdf/10.1145/3597503.3639226 https://dl.acm.org/doi/pdf/10.1145/3597503.3639226 > As for C2Rust, the 5% unsuccessful translations were due to compilation errors, the majority of them caused by unused imports. I'm rather confused by what that's supposed to mean, since unused imports cause warnings, not errors in Rust.