18 ms·
This is interesting and pretty awesome. I can imagine having a lot of fun making a toy compiler that emits LLVM IR directly. Practically speaking, is there a g
by kwantam 13y ago
This is interesting and pretty awesome. I can imagine having a lot of fun making a toy compiler that emits LLVM IR directly.
Practically speaking, is there a good reason to do this in non-toy software rather than the more-or-less standard practice of using C as an IR? That gives you, in addition to compilation via LLVM, the ability (in principle) to use gcc, icc, et al. (In practice, in many cases people end up using compiler-specific extensions in the generated C, which does impose limitations on portability to other compilers, but that's certainly avoidable.)
I suppose if you are dynamically generating things that need to be compiled as quickly as possible, eliminating an unnecessary compilation step could be a significant advantage. It doesn't seem like such a constraint would apply in most cases, though.
- pbsdp 13y agoC is a heavyweight intermediary language which brings with it a /lot/ of baggage. There's no valid direct technical reason other than financial resource constraints to justify compiling to C (... or JS, for that matter) instead of a more suitable and expressive IR/bytecode. It's impossible to implement a variety of useful abstractions in portable C -- for example, function trampolines that do not permute register state and do not appear in the callstack after execution.
- maaku 13y agoOh come now, there are good reasons to compile to JS if you are doing things in a browser.
- pbsdp 13y agoNot direct technical reasons, no. The only indirect technical justification is in maintaining compatibility with existing browsers, but that falls over pretty fast when you're a browser maker (Mozilla) and refusing to work with another browser maker (Google) on jumping over that compatibility hurdle.
- mistercow 13y agoCan you explain specifically what you mean by a "direct" vs. an "indirect" technical reason?
- pbsdp 13y agoDirect: Constraints/requirements directly related to implementing a maximally effecient runtime architecture. Indirect: Market constraints that drive technical limitations. When it comes to browsers, Mozilla both creates and exists within market constraints that drive indirect technical reasoning. Lashing your industry to JS for another 10 years as a bytecode target is clearly not an optimal long-term solution as compared to the current state of the art, but it might make sense as a legacy support mechanism given the indirect technical constraints.
- azakai 13y ago> compared to the current state of the art What do you consider the current state of the art?
- pbsdp 13y agoJVM, CLR, Mono AOT complation, NaCL/PNaCL. A future in which we wind up with architectural support for NaCL-style sandboxing intrinsics, in the same way we saw them be developed for VT-(x|d). Sticking us with JS for another decade and expecting us to compete against better application runtimes? No.
- mistercow 13y ago>(... or JS, for that matter) I have to disagree with that. There are lots of languages whose purpose is to produce JS. For example, it would be very silly for CoffeeScript to use LLVM IR. It would end up reïmplementing all of the things that it shares with JS (that is, most of JS) with no conceivable benefit. Maybe if/when PNaCl is everywhere, it would make sense for it to do that, but even then it's a hard sell as long as four giant companies are competing their asses off to optimize JS. But I do agree with you if the language doesn't correspond well with JS. In that case, it's probably better to target LLVM and then compile to JS via emscripten.
- gngeal 13y agoThere's no valid direct technical reason other than financial resource constraints to justify compiling to C (... or JS, for that matter) instead of a more suitable and expressive IR/bytecode. How does one do call/cc in LLVM, or delimited continuations, for that matter?
- pbsdp 13y agoThe same way you'd do it in any runtime? Adopt an adequate representation model of the machine state, and save that state.
- gngeal 13y agoWhy would I save "a representaion model" when I can simply save the process/thread state?
- pbsdp 13y agoYou can't do that in portable C as-is. Watch what happens when you try to restore thread state on a different system thread using an OS that instrisically ties system threads to user threads to thread local variables to the stack. And no, getcontext() and setcontext() aren't portable.
- gngeal 13y agoWhy would I care if it's portable or not? I've never understood the mentality of "but it's not portable". If something isn't not portable, you put #+ and #- on top of it. I'd rather go for "straight-forward and performant" than for some abstract code transformations to make it runnable on platforms that almost nobody uses to the detriment of everyone else. LLVM is the lowest common denominator. I'm really not fond of lowest common denominators. (Not to mention the fact that it's bloatware in the first place - it's about twice the size of my favourite compiler, and that's before you add the actual language you're trying to compile.)
- 13y ago
- azakai 13y ago> C is a heavyweight intermediary language which brings with it a /lot/ of baggage. There's no valid direct technical reason other than financial resource constraints to justify compiling to C (... or JS, for that matter) instead of a more suitable and expressive IR/bytecode. 1. In theory. But what is that "more suitable and expressive IR/bytecode"? I would argue such a bytecode should be portable, but LLVM IR is not that. 2. There are lots of reasons to target C. C can be compiled by many compilers, while LLVM IR can only be compiled by LLVM to things LLVM can compile to. For example, game consoles, various new embedded platforms, etc. - they all have C compilers, but LLVM might not support them (and possibly cannot support them).
- pbsdp 13y ago> 2. There are lots of reasons to target C. C can be compiled by many compilers, while LLVM IR can only be compiled by LLVM to things LLVM can compile to. For example, game consoles, various new embedded platforms, etc. - they all have C compilers, but LLVM might not support them (and possibly cannot support them). Is the point to produce something optimal for users, or produce something optimal for developers? Most successful game and application platforms trend towards the former, whereas your work on web technologies trends towards the latter.
- eliben 13y agoIt's not difficult to restrict LLVM IR to a portable subset, though ;-)
- 13y ago
- vidarh 13y agoLLVM IR is less portable than C. That's a valid direct technical reason. Generating C (or another high level language) also makes integration with tools in that language potentially trivial (depending on the semantics of the language you're compiling), and depending on your preference you may find it far easier to debug the output. As for portability, while you're right that a number of things can't be implemented in portable C, you gain the benefit that C has been ported to far more systems than LLVM - there are C compilers even for the Commodore 64. I agree with you that C is heavyweight, but LLVM is too. Personally I prefer to build code generators directly - the LLVM infrastructure may be great, but I much prefer to write compiler that are self contained and bootstrapped in the language they are intended to compile. Then again I might just be difficult.
- hga 13y agoC as a target language locks you into a lot of things, like the C calling convention. Might you prefer being able to do tail call optimization without trampolines to C's varargs? It's also an issue WRT to precise garbage collection, although that's currently about the same with LLVM, which discards the distinction between pointers and ints. But in theory you or someone might enhance LLVM someday.
- cliffbean 13y agoThe flip side of this is that using C gives you the C calling convention for free. LLVM IR doesn't abstract away all the details of the C calling convention, and requires IR producers to produce platform-specific code for it.
- pbsdp 13y agoCalling conventions aren't the hard part; it's just a bit of register/stack allocation.
- cliffbean 13y agoI have actually written code to produce LLVM IR to conform to a platform's native C ABI. It's quite a bit more work than "just a bit of register/stack allocation". For one thing, LLVM IR doesn't expose the concept of a stack pointer, so you can't just put things on the stack where they need to go. You need to put things in LLVM's abstractions in places where you happen to know the codegen will lower them to where they need to go. And it's different for every platform.
- pbsdp 13y agoIn the relative context of implementing a production runtime, that's "just a bit of register/stack allocation". If you're hobby-hacking a project, then you may as well just target something that already exists, because micro-inefficiencies that matter on the scale of a deployed production platform probably don't matter for your human-resource-constrained hobby project. If you're designing the next browser language runtime, or a language meant to be used outside of hobby deployments, there's no point in permanently hamstringing your implementation architecture to save a small amount of implementation time within what is already a large and complex problem space.
- 0x09 13y ago> I can imagine having a lot of fun making a toy compiler that emits LLVM IR directly. Practically speaking, is there a good reason to do this in non-toy software rather than the more-or-less standard practice of using C as an IR? Well what you're describing is the whole living purpose of LLVM as a compiler framework. In other words, clang, LDC, GHC, and all other LLVM front-end compilers are non-toy software that transform their respective language into IR and hand it to libLLVM* for optimization/codegen/etc. Targeting C puts a relatively large layer and potentially good deal of ambiguity in between your compiler and the machine code, rather it seems more fitting for experimental or "toy" software to use that in lieu of a real compiler or LLVM frontend.
- caf 13y agoA compiler frontend that emits LLVM IR has to worry about things that one that emits C doesn't, though - for example, target alignment requirements.
- eli_gottlieb 13y agoI can imagine having a lot of fun making a toy compiler that emits LLVM IR directly. Well, I'm still plugging away at the Deca compiler that outputs to LLVM IR directly. It's loads of fun! Practically speaking, is there a good reason to do this in non-toy software rather than the more-or-less standard practice of using C as an IR? I can tell you why I use it: * Even through the C interface, I have an actual library API that outputs the IR code directly rather than having to transform my syntax trees into C syntax trees and then output through a C parser library. * You recursively walk your syntax tree passing an LLVMInstructionBuilder* around to different instruction functions in the API. The result is that your code is a recursive tree-walk (rather than an ordered tree-walk), and the builder object serializes it all for you (including using one instruction's output as input to another) according to the natural order of evaluation in your higher-level language.
- marssaxman 13y agoI am not sure what you mean by "standard practice of using C as an IR". I was a compiler engineer for a decade or so and I have only seen C used as an IR for quick and dirty hack projects which don't care about things like debugging or compile time. The commercial compilers I have worked on each used their own intermediate representation; none used C.
- ygra 13y agoAnecdotal, but at least the Eiffel compiler (EiffelStudio, if I recall correctly) we used in university compiled to C.
- marssaxman 13y agoInteresting. How did they handle debugging?
- ygra 13y agoApparently they include an interpreter as well which probably could be used for debugging. And hopefully interpreter behaviour matches compiled behaviour, then ;-)
- duaneb 13y agoIR is a superset of C without the massive overhead, you really need a good portability reason to use C instead. And if llvm doesn't support it, chances are you can't write very portable c, either.
- pash 13y agoLennart Augustsson gave a talk [0] at last year's Haskell Exchange showing how to embed a DSL in Haskell and then use Haskell's LLVM bindings to generate LLVM IR for it. If you need a blazingly fast DSL, give it a try! 0. http://skillsmatter.com/podcast/home/making-edsls-fly http://skillsmatter.com/podcast/home/making-edsls-fly