4 ms·

C to Go translation tool supporting Go toolchain migration

85 points by thinxer 12y ago

WestCoastJustin 12y ago
GopherCon 2014 had a talk on "Go, from C to Go" by Russ Cox. Here's the slide deck and video for those interested [1, 2]. [1] http://talks.golang.org/2014/c2go.slide#1 http://talks.golang.org/2014/c2go.slide#1 [2] https://www.youtube.com/watch?v=QIE5nV5fDwA https://www.youtube.com/watch?v=QIE5nV5fDwA
- everettForth 12y ago
  For me, this was a key part of the sides: "Goal: Automated conversion of our C code to Go. Target: our C code, not all C code."
- zvrba 12y ago
  And the slides cite a paper from 1977: http://dl.acm.org/citation.cfm?id=321999 http://dl.acm.org/citation.cfm?id=321999 Nice to see that somebody knows how to dig through history.
kanwisher 12y ago
Would be nice if this could be made into a more general tool, would be a cool way to modernize older libraries and reinvigorate new development into them
- dsymonds 12y ago
  That's been proposed, but it won't happen. There's too many edge cases, and it only works on the Go compiler because it has a particularly constrained style and rsc can make the relevant hacks as a one-off effort.
  - marco2357 12y ago
    Why not? There's not that much of a difference between translating C to assembler or a high level language like Java/C#/Go. The main challenge is to make the code readable. But basically someone just has to sit down and do it right.
- lmm 12y ago
  Most libraries that are still in C are in C so that people can embed them from many languages, or because they need to be used in scenarios where GC is unacceptable. I don't think many libraries that have remained in C (and not used e.g. D, or OCaml, or Haskell) will be suitable for migration into Go.
- haches 12y ago
  Friend of mine does fully automatic translation from C to Java, including automatic OO reengineering. Some demos are available on his page: https://www.mtsystems.ch https://www.mtsystems.ch This work is based on the stuff he did for his PhD. I think his largest test case so far is a full translation of VIM.
  - marco2357 12y ago
    Main author of the tool here. If you have questions feel free to contact me: marco.trudel@mtsystems.ch
- rsc 12y ago
  I'd certainly be happy if someone wants to do that. I'm not planning to. For this kind of thing my instinct is that the delta from "useful tool that gets you 90-99% of the way there" to "general tool that just works" is at least 2x if not 10x the effort. For what I'm doing, the 90-99% is fine and then I can fix the rest by hand.
eklavya 12y ago
I wish for something similar for C++ -> Rust and C -> Rust.
- Daishiman 12y ago
  The semantics of those languages are so brutally complex and different that I find it difficult that we could see such a tool.
  - eklavya 12y ago
    I dare think I understand that. The down votes seem to imply that a guy can't even wish!
azernik 12y ago
For the casual reader, in case other comments don't make this clear - this is not a general-purpose translation library, and probably will not work for whatever C codebase you have. It is a one-off tool built to migrate the specific subset of C used in the original go compilers, as part of the process of making the go toolchain self-hosting (go compilers written in go). This translator produces a VERY literal translation, as the intention is for future human passes to clean it up and convert it to more idiomatic go. See here for the full plan: https://docs.google.com/document/d/1P3BLR31VA8cvLJLfMibSuTdwTuF7WWLux71CYD0eeD8/edit https://docs.google.com/document/d/1P3BLR31VA8cvLJLfMibSuTdw...
peterwaller 12y ago
I love the simplicity of the main function: rewriteTypes(cfg, prog) rewriteSyntax(cfg, prog) rewriteLen(cfg, prog) fixGoTypes(cfg, prog) renameDecls(cfg, prog) exportDecls(cfg, prog) writeGoFiles(cfg, prog) https://github.com/rsc/c2go/blob/f6c077dd7192b90f4a7ad0a4b8f4fa5b2649899b/main.go#L62-L76 https://github.com/rsc/c2go/blob/f6c077dd7192b90f4a7ad0a4b8f...
- SixSigma 12y ago
  Plan9 style.
qb45 12y ago
This should be written in C and have sufficient C language coverage to understand and translate itself. Functional tests for free :)
4ad 12y ago
I already reused part of this a few months ago to automatically refactor a large body of C code (around 10k LOC)[1]. Some time ago, Go converted (by hand) the various Plan 9 linkers used into a library, and I had to do the same for the arm64 linker. I'm not sure if I saved time or not (I think I did) for the effective work, but I definitely saved time (and stress) overall, as the conversion was without bugs. I know for sure that I would have introduced a lot of bugs if I did this manually. Tools that understand source code are great, and I'm really excited about the new Go compilers. [1] https://code.google.com/p/rbits/source/browse/7lfix/main.go https://code.google.com/p/rbits/source/browse/7lfix/main.go

85 points by thinxer 12y ago

GopherCon 2014 had a talk on "Go, from C to Go" by Russ Cox. Here's the slide deck and video for those interested [1, 2]. [1] http://talks.golang.org/2014/c2go.slide#1 http://talks.golang.org/2014/c2go.slide#1 [2] https://www.youtube.com/watch?v=QIE5nV5fDwA https://www.youtube.com/watch?v=QIE5nV5fDwA

everettForth 12y ago
For me, this was a key part of the sides: "Goal: Automated conversion of our C code to Go. Target: our C code, not all C code."
zvrba 12y ago
And the slides cite a paper from 1977: http://dl.acm.org/citation.cfm?id=321999 http://dl.acm.org/citation.cfm?id=321999 Nice to see that somebody knows how to dig through history.

kanwisher 12y ago

Would be nice if this could be made into a more general tool, would be a cool way to modernize older libraries and reinvigorate new development into them

dsymonds 12y ago
That's been proposed, but it won't happen. There's too many edge cases, and it only works on the Go compiler because it has a particularly constrained style and rsc can make the relevant hacks as a one-off effort.
- marco2357 12y ago
  Why not? There's not that much of a difference between translating C to assembler or a high level language like Java/C#/Go. The main challenge is to make the code readable. But basically someone just has to sit down and do it right.
lmm 12y ago
Most libraries that are still in C are in C so that people can embed them from many languages, or because they need to be used in scenarios where GC is unacceptable. I don't think many libraries that have remained in C (and not used e.g. D, or OCaml, or Haskell) will be suitable for migration into Go.
haches 12y ago
Friend of mine does fully automatic translation from C to Java, including automatic OO reengineering. Some demos are available on his page: https://www.mtsystems.ch https://www.mtsystems.ch This work is based on the stuff he did for his PhD. I think his largest test case so far is a full translation of VIM.
- marco2357 12y ago
  Main author of the tool here. If you have questions feel free to contact me: marco.trudel@mtsystems.ch
rsc 12y ago
I'd certainly be happy if someone wants to do that. I'm not planning to. For this kind of thing my instinct is that the delta from "useful tool that gets you 90-99% of the way there" to "general tool that just works" is at least 2x if not 10x the effort. For what I'm doing, the 90-99% is fine and then I can fix the rest by hand.

eklavya 12y ago

I wish for something similar for C++ -> Rust and C -> Rust.

Daishiman 12y ago
The semantics of those languages are so brutally complex and different that I find it difficult that we could see such a tool.
- eklavya 12y ago
  I dare think I understand that. The down votes seem to imply that a guy can't even wish!

azernik 12y ago

For the casual reader, in case other comments don't make this clear - this is not a general-purpose translation library, and probably will not work for whatever C codebase you have. It is a one-off tool built to migrate the specific subset of C used in the original go compilers, as part of the process of making the go toolchain self-hosting (go compilers written in go). This translator produces a VERY literal translation, as the intention is for future human passes to clean it up and convert it to more idiomatic go. See here for the full plan: https://docs.google.com/document/d/1P3BLR31VA8cvLJLfMibSuTdwTuF7WWLux71CYD0eeD8/edit https://docs.google.com/document/d/1P3BLR31VA8cvLJLfMibSuTdw...

peterwaller 12y ago

I love the simplicity of the main function: rewriteTypes(cfg, prog) rewriteSyntax(cfg, prog) rewriteLen(cfg, prog) fixGoTypes(cfg, prog) renameDecls(cfg, prog) exportDecls(cfg, prog) writeGoFiles(cfg, prog) https://github.com/rsc/c2go/blob/f6c077dd7192b90f4a7ad0a4b8f4fa5b2649899b/main.go#L62-L76 https://github.com/rsc/c2go/blob/f6c077dd7192b90f4a7ad0a4b8f...

SixSigma 12y ago
Plan9 style.

qb45 12y ago

This should be written in C and have sufficient C language coverage to understand and translate itself. Functional tests for free :)

4ad 12y ago

I already reused part of this a few months ago to automatically refactor a large body of C code (around 10k LOC)[1]. Some time ago, Go converted (by hand) the various Plan 9 linkers used into a library, and I had to do the same for the arm64 linker. I'm not sure if I saved time or not (I think I did) for the effective work, but I definitely saved time (and stress) overall, as the conversion was without bugs. I know for sure that I would have introduced a lot of bugs if I did this manually. Tools that understand source code are great, and I'm really excited about the new Go compilers. [1] https://code.google.com/p/rbits/source/browse/7lfix/main.go https://code.google.com/p/rbits/source/browse/7lfix/main.go