21 ms·
C as an Intermediate Language (2012)
- jjnoakes 10y agoPortability and interoperability are key in industry. I think all of the new languages are amazing these days. I am learning Rust, Clojure, Haskell, and many more. Buy I can not use any of them at work. I'm in a position to influence teams of smart programmers, but the only language that I would be able to use would be one that fits in to the existing infrastructure, which is natively compiled shared libraries on esoteric unix platforms. So no LLVM. No JVM. No Haskell. I could possibly get away with a lisp or scheme that compiled portable C if I wanted. But that doesn't excite me as much. I would kill for a Rust to human-readable C++ transpiler. I think it could be used immediately by many. I may have to write one.
- loup-vaillant 10y agoHaskell used to have a C back-end. That philosophy could in principle be extended to anything.
- tluyben2 10y agoThe problem is that usually not very human readable C comes out of it; to use something instead of your company's language of choice, unless you are the CTO or something else high up the ladder, would need to compile to the company's language and do it in a human readable way. Some Javascript 'transpilers' do that (I think Coffeescript?) but usually it really acts as a compiler and the idea is that no-one should need to read / use the source code that is emitted besides for running it.
- jjnoakes 10y agoThis is exactly it. This is why I think rust is a winner. I bet most rust code could be compiled to c++ very cleanly. You get rust's compiler with all of the safety checks, and you get the platform support of C++. It would be a dream.
- loup-vaillant 10y ago> The problem is that usually not very human readable C comes out of it The x86 assembly or LLVM intermediate representation that comes out of most compiler is not very human readable, but I don't see anyone having any problem with it whatsoever. Readability of generated code was never the point of C using C as a back-end. The point is portability and tooling. By the way /u/jjnoakes can suggest fancy languages and compilers, if only they can generate native code on obscure platforms (I guess, something other than x86 and ARM). Generating C code (even utterly unreadable C code) would solve his problem.
- jjnoakes 10y agoRemember, it isn't only the architecture. It is also the calling conventions and object/shared library/executable file format. All things that require work if I cross compile, but all things I get for free if I generate C or C++. I would prefer readable C or C++ as much as possible because it gives me debugability if the code generator goes wrong and it gives me an out if I want to move away from the language some day.
- loup-vaillant 10y agoWell, there aren't many solution out of this hole. If you need another language, you might have to write (and control) the compiler yourself. As in, if the code generator (or anything else) goes wrong, you can correct it. <sarcasm> Now this might require a little bit more work. </sarcasm>
- jjnoakes 10y agoThe solution I mentioned in my original post gets me out if this issue, as does the one in the article that this hn post is all about.
- qznc 10y agoI believe porting LLVM to esoteric Unix platforms would be easier than a C backend. Less fun though.
- jjnoakes 10y agoYou do the work once for C or C++ and you can use rust anywhere forever. Or you port it every time you need a new platform.
- vmorgulis 10y agoPorting nothing is better.
- jjnoakes 10y agoSure. Port nothing and keep writing in suboptimal languages. It is what we do now. But does it hurt to strive for better? I don't think it does.
- vmorgulis 10y agoYes it is. For me, C as an intermediate vs LLVM language is a "worse is better" or "good enough" issue. LLVM is far more powerful but more complex. C is simpler and already works everywhere.
- kibwen 10y agoNote that Rust code can interoperate with other languages by exposing a C interface and having the host language call it as though it were C. There's no runtime to start up or anything, so to the host platform there's no discernible difference (though you may need some machinery in your build system). This is how Rust-written libraries have been integrated into Firefox (which is obviously still integrated into Firefox), and there are companies that have been using Rust to extend Ruby for years now.
- jjnoakes 10y agoYep. I understand and this is why I could integrate rust right now into my products, module by module or object file by object file. I just need the compiler to run on my systems. Since it is written in rust, I either need it ported to my systems (too much work) or I need my rust code transpiled to C or C++ so I can build it myself (seems doable).
- pcwalton 10y agoIt's not doable. How do you deal with UB from signed overflow? How do you get proper debug info? How do you set the proper EH personality? There are no shortcuts to a proper implementation of Rust. Implement an LLVM backend for your architecture. It doesn't make sense for Rust to maintain a backend that is doomed to be forever broken in various ways.
- jjnoakes 10y agoI answered your ub question elsewhere. Exceptions and debug info come from C++. I don't see the issues there yet, but perhaps I just don't have the right knowledge yet. Rust won't have to maintain the backend. Who said they would? They might choose to if it becomes useful - it would give them things they dint have now (C++ interoperability, and new platform support for almost free), but no one here asked them to maintain anything. I am also not sure what you mean by shortcuts or broken forever in various ways.
- 10y ago
- MustardTiger 10y ago>I'm in a position to influence teams of smart programmers, but the only language that I would be able to use would be one that fits in to the existing infrastructure, which is natively compiled shared libraries on esoteric unix platforms. So, how does that prevent you from using natively compiled languages like haskell?
- jjnoakes 10y agoThe Haskell compiler doesn't run on my systems, and it does not generate human readable C or C++ from my Haskell code.
- MustardTiger 10y agoWhat systems are those? And why do you need "human readable C"? You said you needed it to work in a natively compiled shared library environment. You do not need to compile to C for that. But if you really want to do that for some reason, you can do it.
- jjnoakes 10y agoI'd prefer it to be human readable because it helps in debugability if the code generator is broken (or we misunderstand some subtle semantics and want to look closer), and if I ever want to move away from the language in the future. Ignoring that, if I don't compile to C or C++, then I'm not sure how easy it will be to use. Are you suggesting Haskell can generate object files, shared objects, and binaries for my systems without compiling to C or C++ first? How would it get the file formats and calling conventions right? Can I specify them somehow to Haskell (doubtful)?
- MustardTiger 10y ago>Are you suggesting Haskell can generate object files, shared objects, and binaries for my systems without compiling to C or C++ first? Of course, that's what ghc does by default. >How would it get the file formats and calling conventions right? The same way every other compiler does. What systems are we talking about?
- jnbiche 10y agoYour platform must be pretty esoteric indeed to not be an LLVM target. What are you targeting? Also, Rust is interoperable to the exact same extent C is. Making shared libraries exposing a C ABI is a snap.
- jjnoakes 10y agoRust is interoperable. That's why I could use it. I just need the compiler to generate code I can build with my C or C++ toolchain, since there is no rust toolchain for my platforms.
- sn9 10y agoHave you looked at OCaml? [0] [0] https://en.wikipedia.org/wiki/OCaml#Features https://en.wikipedia.org/wiki/OCaml#Features
- jjnoakes 10y agoI am familiar with ocaml at a high level. I have written a few toy interpreters in it. The c code it generates is basically the ocaml bytecode interpreter and the bytecode of my program. Not what I was thinking of.
- sn9 10y agoIt also has a native code compiler and a foreign function interface.
- jjnoakes 10y agoIt can generate portable and human readable C? Can you link to some information about that?
- loup-vaillant 10y agoNothing of the sort. But on the platforms it compiles to, Ocaml can generate object files that look like they came from C. I use this feature right now to integrate a compiler in an otherwise C++ project. Here's a link to the FFI documentation: http://caml.inria.fr/pub/docs/manual-ocaml/intfc.html http://caml.inria.fr/pub/docs/manual-ocaml/intfc.html But I run on Debian stable on x86-64. Not exactly an exotic platform.
- jjnoakes 10y agoIs this not the bytecode embedded in C which I mentioned above?
- loup-vaillant 10y ago
- ilaksh 10y agoIf you guys are interested in languages that compile to C, the best programming language I know of compiles to C and it is called Nim.
- felixangell1024 10y agoEDIT0: It appears I was wrong, for some reason I was under the impression DMD produced C code. Ignore this comment! :) Another language that is widely used by hobbyists and in production is D which, as far as I know, produces C code too. EDIT: I should say there are a few D compilers, DMD, LDC, and some other one by GNU(?). I think that DMD is the main compiler, but I could be wrong. LDC generates LLVM code.
- dom96 10y agoAre you sure? IIRC the main D compiler doesn't generate C.
- gmfawcett 10y agoThere are three compilers for D (DMD, GDC, LDC), but none of them uses C as an intermediary language.
- deleted 10y ago[deleted]
- bjz_ 10y agoWhy on earth would LDC use C when it has access to LLVM IR? Edit: A cursory look seems to suggest they do translate to IR: https://github.com/ldc-developers/ldc/tree/master/ir https://github.com/ldc-developers/ldc/tree/master/ir
- keithnz 10y agoreally wish this was more common, as language choices on embedded platforms are often VERY limited. ie, C. Though on embedded systems you'd also want more constraints for the backend, like do not use dynamic memory, perhaps being able to specify the output code is MISRA compliant.
- felixangell1024 10y agoIt's also insanely easy to generate code for. In addition to this, you don't have to build an entire toolchain of LLVM stuff to get it to work on your system -- LLVM is a nightmare to setup on Windows.
- dom96 10y agoIndeed. C is brilliant as an intermediate language. One of the greatest things about it is that once you generate C code it's not too much of a stretch to also generate C++/Objective C specific code and take advantage of the libraries written in those languages. Despite this, there are many people out there who view languages that compile to C as being inferior. I still don't understand it.
- bjz_ 10y agoC might be decent for the short term, but if your language is at the stage of trying to be adopted, I'd rather see some IR output with a C ffi. You have to take into account what semantics you are saddled with as you go through the translation process through C, for example all that damn undefined behavior...
- yokohummer7 10y agoThere are many things that are present in assembly but lacking in C. For example, how do you detect integer overflow in C without resorting to compiler extensions?
- jjnoakes 10y agoYou stub out the few architecture specific routines you need and let someone write them for their architecture in asm if it doesn't exist already. A minimal porting effort for the best of all worlds.
- pcwalton 10y agoThe best of all worlds, except that you lose your optimizations.
- jjnoakes 10y agoI would like to see the benchmarks before I agreed to that. You might not get the best optimization possible if you have to leave it up to the C compiler with inline assember... But that mix worked well for C and C++. And rust can still optimize things like deciding which integer optimizations need the overflow check, and only add it where static analysis failed.
- deleted 10y ago[deleted]
- neopallium 10y agoHere is a list of compilers that can generate C code for different languages: https://github.com/dbohdan/compilers-targeting-c https://github.com/dbohdan/compilers-targeting-c
- Artlav 10y agoThis begs the question of what would be better - to compile your favorite language to C and use something like Emscripten to make JS, or to compile your code directly to JS? So far as my experiments went, a 3000 native score Dhrystone test compiles to C at 1300, and that C compiles to 850 worth of JS. Which suggests that a direct-to-JS compiler might be worth the efforts...
- gokr 10y agoYou can play around with Nim for getting some numbers on this - since it can compile to C, C++ or js.
- kragen 10y agoAs far as I can tell, on all the axes that LLVM excels as an intermediate language (ease of getting started, debugging support, many backends, optimization, flexibility), C is even better. C is easier to get started with, has easier debugging support, has more backends, better optimization, and more flexibility. To take a simple example, I have here a 2428-line C program generated by compiling Linus Åkesson's Game of Life in BF (http://www.linusakesson.net/programming/brainfuck/ http://www.linusakesson.net/programming/brainfuck/) into C using Daniel B. Cristofani's dbf2c.b (http://www.hevanet.com/cristofd/brainfuck/dbf2c.b http://www.hevanet.com/cristofd/brainfuck/dbf2c.b), which is a BF compiler written in BF. Compiling these 2428 lines of C to machine code using tcc 0.9.25 takes 20ms on my 1.6GHz Atom netbook. Most of this is about 16ms of tcc overhead (startup and shutdown time); the rest is compiling several hundred thousand lines of C per second with tcc. You should get several million lines of C per second with tcc on a modern machine. This isn't optimized code, about equivalent to gcc -O0, typically about 3×–5× slower than optimized code. But that's enormously better than interpretation overhead. (Using decent optimization levels with GCC makes it take several seconds to compile, because GCC's optimizer doesn't deal well with enormous functions.) dbf2c.b, the C-generating BF compiler, is 892 bytes of BF code when stripped. Now, I'm not saying you should write your compilers in deliberately obfuscated programming languages in as few bytes as possible; I'm saying that the fact that this is even possible at all should give you really good feelings about how easy it is to compile things to C.
- pcwalton 10y agoWhat? How does C have "better optimization" than LLVM? As far as I can tell, this seems clearly false: you can't do custom alias sets, for instance, in C, whereas you can in LLVM. None of your examples mention optimization at all (and BF is also totally unrepresentative of anything in the real world). Also, C may have easier debugging support, but it's significantly worse than having exact control over the contents of your DWARF DIEs. Making it easy to have a bad debugging experience isn't a plus as I see it.
- kragen 10y agoC has better optimization than LLVM because if you generate LLVM IR, you can't compile it with icc or Visual Studio. (Or GCC, but LLVM probably already equals or beats GCC at optimization already.) You're probably right that LLVM also has better optimization than C because of things like the possibility of doing better aliasing. I agree that BF is unrepresentative of anything in the real world. Thank goodness. I can certainly see how debugging support could be better if you can go beyond the support C gives you.
- FraaJad 10y agoA few languages that generate C that are currently popular are: * nim * Vala (GObject backend) * Purescript (technically it has a C++ backend, but it is worth mentioning here for the very clean C++ it produces!)
- cokernel_hacker 10y agoProfessional compiler engineer here, C is a mediocre intermediate language. Let's start with an excellent quote from Wittgenstein. "The limits of my language mean the limits of my world." Using C as your intermediate language means that your expressiveness is limited to valid C programs. This is workable but only if your language can be mapped to C in _useful_ ways. For example, let's say your language has behavior similar to scheme's tail-call. How would you get this behavior from a C compiler? You will never be able to make this reliably across optimization levels, etc. Guaranteed tail-calls are the tip of the iceberg, there are a lot more features which cannot be reasonably mapped onto C. Real compiler IRs increase your expressivity beyond what the C language designers decided was important.
- pcwalton 10y agoExactly, and this is what most people who work on compilers discovered long ago. It's disappointing to see "just compile to C!" still so popular on HN. I would cite GC as the most prominent example of a feature that is incompatible with compiling to C. It is impossible to write a competitive high performance tracing GC on top of C. Mandatory register spills at safe points and conservative scanning have huge downsides.
- ndesaulniers 10y ago> It's disappointing to see "just compile to C!" still so popular on HN. Posts on C get blindly upvoted on HN and proggit. I guarantee it. Me thinks it's like the history channel for programmers; learning about their forerunners and how they had to bang rocks together to make fire. It's also how I've gotten all of my karma. Unfortunately, the abyss has also looked into me, as I now write C code for a living...
- groovy2shoes 10y agoUsing C as an intermediary language can still be useful at times. For example, when writing a bootstrap compiler, you can target C and magically support virtually every platform. It's not so great for production compilers for numerous reasons, including performance (poor unless your languages maps cleanly to C) and debugging experience (debugging object code is not fun, and if your object is C, you get stuck with whatever debugging information your C compiler outputs; you don't have any say in the matter). For bootstrapping compilers, I only see three sane options, really: 1. Write an interpreter to host the compiler (probably in C). 2. Write a cross-compiler. 3. Write an X-to-C compiler. I insist on C here not because of performance, but because it's ubiquitous.
- vmorgulis 10y agoIf the target compilers are GCC and clang, C++ is even a better intermediate language.