4 ms·

Why C but not llvm? Structured code generation is always better than string-based one.

by densh 13y ago

pat_punnu 13y ago
Can you show a proof for that claim? I believe it to be nonsense.
- bratsche 13y ago
  I'm not sure if this is what the previous poster was talking about, but clang has some APIs that let you get access to the AST pretty easily. It's been a really long time since I've looked at it, and at the time I don't think it was exposed as a library API for general consumption. But for example: https://github.com/bratsche/clang/blob/gtkrewriter/tools/clang-cc/RewriteGtk.cpp https://github.com/bratsche/clang/blob/gtkrewriter/tools/cla...
  - pat_punnu 13y ago
    What is his definition of better? It sounds like he thinks it's entirely objective, so he should be able to express it clearly and logically. Does he think that it's technically more powerful? Again he should be able to prove that if that's the case. Otherwise he's just giving a shitty opinion, and should say that. I think the claim is nonsense because with inline assembler there is nothing that you cannot express in C that you can with LLVM. So the decision between the two is opinion.
    - bratsche 13y ago
      I don't know what his definition of anything is. I happen to agree with you. The example I pasted above was just from my only experience working with a C AST. In that case I wanted to input C code and then write out a transformed version of it.
    - coldtea 13y ago
      >Does he think that it's technically more powerful? Again he should be able to prove that if that's the case. Otherwise he's just giving a shitty opinion, and should say that. It's not like it's some controversial opinion what he said -- it's both self evident and common place. It's you who offers the more controversial opinion (and in a rude way, to top). >I think the claim is nonsense because with inline assembler there is nothing that you cannot express in C that you can with LLVM. So the decision between the two is opinion. It's not about "expression", and nobody argued that you can express more in LLVM. This is missing the point by miles! It's about having more structure and less of an ad-hoc pipeline, which helps with better tooling, error prevention, etc. (Not only what you wrote is wrong, but even if the original argument was about expression, your opinion would still be wrong. Two things offering equivalent expressive power, does not mean that they are just as good to use in practice at all. Might as well ask "why invent new languages, when assembly can express everything"). The only benefit to using C for something like this is portability, which is something else altogether.
      - vidarh 13y ago
        > It's not like it's some controversial opinion what he said -- it's both self evident and common place. As someone who has written more than one compiler, I don't see how it is self-evident at all. It's also not at all that common-place compared to generating C or asm output textually. > It's about having more structure and less of an ad-hoc pipeline, which helps with better tooling, error prevention, etc. Those provide some benefits, sure. At the cost of massive amounts of complexity in the case of LLVM. > The only benefit to using C for something like this is portability, which is something else altogether. Now it is you who are wrong. Other people have already pointed out, for example, that C provides an easy-to-read intermediate format, and is simple to generate, as other benefits. Not having to deal with a massive C++ codebase is another. You may disagree that these other benefits are worth it, but for me at least they are (just taking a break from a compiler that generates textual asm because I find even that preferable to dealing with LLVM).
        coldtea 13y ago
        Sure, I agree about this: "C provides an easy-to-read intermediate format, and is simple to generate, as other benefits. Not having to deal with a massive C++ codebase is another.". So portability and less dependencies, plus easier.
- coldtea 13y ago
  Nonsense?! Outputting text to be interpreted as code is far more low level and error prone than targeting an AST via an API like LLVMs. And you loose a lot of high quality tooling that you could take advantage of.
  - pat_punnu 13y ago
    "more ... error prone" Can you prove that? If not, it's just a baseless opinion.
    - abraininavat 13y ago
      Can you prove that? If not, it's just a baseless opinion. You are wrong here. Unproved statements are not necessarily baseless opinions. They may be well supported opinions. I think you meant to ask him, "Can you support that?"
    - coldtea 13y ago
      What's with this childish "can you prove this, can you prove that" thing? Are you 12 year old? Not everything can (or should) be proved of the drop of a hat in a discussion list -- that doesn't make everything without a formal proof "baseless" opinion. If you cannot see the self-evidentness (sic) of a STRUCTURED API to produce an AST makes it easier to avoid mistakes compared to spitting out text to compile as a C program, then I'm not sure any proof would help anyway. It's like asking me to prove why using an XML processor to crete and save a DOM tree would produce more error free results than manually compiling tags as strings. Or why parsing a JSON file and working on the nodes is less error prone than using regular expressions to extract values from the JSON as a big string.
      - pat_punnu 13y ago
        We work in a technical field. We call ourselves engineers. The academic version is computer SCIENCE. Unsupportable opinions and opinions presented as facts are inexcusable.
        13y ago
mjn 13y ago
If you generate C, you can get a C AST from the result easily, which is a structured format...
rayiner 13y ago
Generating C is quicker to write and easier to debug (can more easily read intermediate output) and doesn't require accessing a C++ api.
- rdmckenzie 13y ago
  I can't stress the C++ api part of this enough. At the moment I'm building a Pascal compiler in Clojure for course work and when the time came to do byte code generation the first thing I looked to was the LLVM infrastructure both for ease of use and because it would trivially provide me with the ability to target non-x86 platforms. The reality of the matter is that interacting with non-java linked libraries is a real pain from every JVM language know of. Two years ago this was posted here [https://github.com/jasonjckn/llvm-clojure-bindings https://github.com/jasonjckn/llvm-clojure-bindings], but since then LLVM has gone through two major restructurings so it didn't work out of the box and I estimated that it would take less effort to build my own naive infrastructure than to patch this one & integrate it. As a result I'm generating code in terms of lists of newline terminated assembly statement strings that I can just print or write to a file when I'm done. While I agree that C is a sub-optimal output format in that you have to compile the output, it is also the clear lingua franca for systems programming and assembly generation these days. Generating C gives you interesting options like linking to other C codebases or your own C code the same way that cljs gives you the option of interacting with "native" javascript libraries as well as clojurescript toolkits.
  - pjmlp 13y ago
    Why not use the Java LLVM bindings? They are usually in sync with the LLVM release.
  - jared314 13y ago
    Have you looked at mjolnir[0]? It was part of a presentation at Clojure West this year. I'm not sure if the video is out yet, but the slides are[1]. [0] https://github.com/halgari/mjolnir https://github.com/halgari/mjolnir [1] https://github.com/strangeloop/clojurewest2013/tree/master/slides https://github.com/strangeloop/clojurewest2013/tree/master/s...
saejox 13y ago
Generated C can be copy pasted and readable by humans. Debugging IR code is a terrible experience.
swannodette 13y ago
People have made this argument for ClojureScript as well. In the case of ClojureScript the benefits are considerably less clear as we rely on the ability of Google Closure to do fairly amazing optimizations on the generated JavaScript source. That said I'd love to see a serious Clojure-in-Clojure targeting LLVM.
cliffbean 13y ago
Similarly, structured control flow (C's if, for, while, switch etc.) is better than unstructured (LLVM IR just has the equivalent of gotos (yes, LLVM IR has switch too, but it can go anywhere, so it still isn't inherently structured)). So it goes both ways. Some other advantages of C over LLVM IR: ABI compatibility with C automatically. In LLVM IR, being ABI-compatible with C is often a considerable headache and you have to do different things on each platform. You can use C libraries by #including their header files (the way many C libraries were designed to be used), instead of hardwiring all that information (macro expansions, enum values, typedefs, inline functions, struct definitions, etc.) from their header files in your code generator. You have the option of switching C compilers; you're not as locked into a single backend. With a bit of care, you can make your output much more readable. LLVM IR demands either SSA form (most front-ends don't want to do this) or herds of allocas, loads, and stores everywhere. In C, you just say "int x;" to declare an int, and just "x" to refer to it. You can use C features like bitfields, designated initializers, short-circuit operators (&&, ||), compound assignment (+=, -=, etc.), and so on. You can do all these things in LLVM IR, but you have to lower them yourself.
masklinn 13y ago
Generating C is much simpler than LLVM IR (and you can do structured code generation for C: generate and serialize C AST); it's infinitely easier to read and debug; it allows switching compiler; it provides C stdlib to piggyback on with very little overhead. It's a fine strategy to start with, there are more important things than fucking around with LLVM IR at this point, they can switch to LLVM or a native generator if they get the thing effective and off-the ground.

Why C but not llvm? Structured code generation is always better than string-based one.

by densh 13y ago

Why C but not llvm? Structured code generation is always better than string-based one.

pat_punnu 13y ago

Can you show a proof for that claim? I believe it to be nonsense.

bratsche 13y ago
I'm not sure if this is what the previous poster was talking about, but clang has some APIs that let you get access to the AST pretty easily. It's been a really long time since I've looked at it, and at the time I don't think it was exposed as a library API for general consumption. But for example: https://github.com/bratsche/clang/blob/gtkrewriter/tools/clang-cc/RewriteGtk.cpp https://github.com/bratsche/clang/blob/gtkrewriter/tools/cla...
- pat_punnu 13y ago
  What is his definition of better? It sounds like he thinks it's entirely objective, so he should be able to express it clearly and logically. Does he think that it's technically more powerful? Again he should be able to prove that if that's the case. Otherwise he's just giving a shitty opinion, and should say that. I think the claim is nonsense because with inline assembler there is nothing that you cannot express in C that you can with LLVM. So the decision between the two is opinion.
  - bratsche 13y ago
    I don't know what his definition of anything is. I happen to agree with you. The example I pasted above was just from my only experience working with a C AST. In that case I wanted to input C code and then write out a transformed version of it.
  - coldtea 13y ago
    >Does he think that it's technically more powerful? Again he should be able to prove that if that's the case. Otherwise he's just giving a shitty opinion, and should say that. It's not like it's some controversial opinion what he said -- it's both self evident and common place. It's you who offers the more controversial opinion (and in a rude way, to top). >I think the claim is nonsense because with inline assembler there is nothing that you cannot express in C that you can with LLVM. So the decision between the two is opinion. It's not about "expression", and nobody argued that you can express more in LLVM. This is missing the point by miles! It's about having more structure and less of an ad-hoc pipeline, which helps with better tooling, error prevention, etc. (Not only what you wrote is wrong, but even if the original argument was about expression, your opinion would still be wrong. Two things offering equivalent expressive power, does not mean that they are just as good to use in practice at all. Might as well ask "why invent new languages, when assembly can express everything"). The only benefit to using C for something like this is portability, which is something else altogether.
    - vidarh 13y ago
      > It's not like it's some controversial opinion what he said -- it's both self evident and common place. As someone who has written more than one compiler, I don't see how it is self-evident at all. It's also not at all that common-place compared to generating C or asm output textually. > It's about having more structure and less of an ad-hoc pipeline, which helps with better tooling, error prevention, etc. Those provide some benefits, sure. At the cost of massive amounts of complexity in the case of LLVM. > The only benefit to using C for something like this is portability, which is something else altogether. Now it is you who are wrong. Other people have already pointed out, for example, that C provides an easy-to-read intermediate format, and is simple to generate, as other benefits. Not having to deal with a massive C++ codebase is another. You may disagree that these other benefits are worth it, but for me at least they are (just taking a break from a compiler that generates textual asm because I find even that preferable to dealing with LLVM).
      - coldtea 13y ago
        Sure, I agree about this: "C provides an easy-to-read intermediate format, and is simple to generate, as other benefits. Not having to deal with a massive C++ codebase is another.". So portability and less dependencies, plus easier.
coldtea 13y ago
Nonsense?! Outputting text to be interpreted as code is far more low level and error prone than targeting an AST via an API like LLVMs. And you loose a lot of high quality tooling that you could take advantage of.
- pat_punnu 13y ago
  "more ... error prone" Can you prove that? If not, it's just a baseless opinion.
  - abraininavat 13y ago
    Can you prove that? If not, it's just a baseless opinion. You are wrong here. Unproved statements are not necessarily baseless opinions. They may be well supported opinions. I think you meant to ask him, "Can you support that?"
  - coldtea 13y ago
    What's with this childish "can you prove this, can you prove that" thing? Are you 12 year old? Not everything can (or should) be proved of the drop of a hat in a discussion list -- that doesn't make everything without a formal proof "baseless" opinion. If you cannot see the self-evidentness (sic) of a STRUCTURED API to produce an AST makes it easier to avoid mistakes compared to spitting out text to compile as a C program, then I'm not sure any proof would help anyway. It's like asking me to prove why using an XML processor to crete and save a DOM tree would produce more error free results than manually compiling tags as strings. Or why parsing a JSON file and working on the nodes is less error prone than using regular expressions to extract values from the JSON as a big string.
    - pat_punnu 13y ago
      We work in a technical field. We call ourselves engineers. The academic version is computer SCIENCE. Unsupportable opinions and opinions presented as facts are inexcusable.
      - 13y ago

mjn 13y ago

If you generate C, you can get a C AST from the result easily, which is a structured format...

rayiner 13y ago

Generating C is quicker to write and easier to debug (can more easily read intermediate output) and doesn't require accessing a C++ api.

rdmckenzie 13y ago
I can't stress the C++ api part of this enough. At the moment I'm building a Pascal compiler in Clojure for course work and when the time came to do byte code generation the first thing I looked to was the LLVM infrastructure both for ease of use and because it would trivially provide me with the ability to target non-x86 platforms. The reality of the matter is that interacting with non-java linked libraries is a real pain from every JVM language know of. Two years ago this was posted here [https://github.com/jasonjckn/llvm-clojure-bindings https://github.com/jasonjckn/llvm-clojure-bindings], but since then LLVM has gone through two major restructurings so it didn't work out of the box and I estimated that it would take less effort to build my own naive infrastructure than to patch this one & integrate it. As a result I'm generating code in terms of lists of newline terminated assembly statement strings that I can just print or write to a file when I'm done. While I agree that C is a sub-optimal output format in that you have to compile the output, it is also the clear lingua franca for systems programming and assembly generation these days. Generating C gives you interesting options like linking to other C codebases or your own C code the same way that cljs gives you the option of interacting with "native" javascript libraries as well as clojurescript toolkits.
- pjmlp 13y ago
  Why not use the Java LLVM bindings? They are usually in sync with the LLVM release.
- jared314 13y ago
  Have you looked at mjolnir[0]? It was part of a presentation at Clojure West this year. I'm not sure if the video is out yet, but the slides are[1]. [0] https://github.com/halgari/mjolnir https://github.com/halgari/mjolnir [1] https://github.com/strangeloop/clojurewest2013/tree/master/slides https://github.com/strangeloop/clojurewest2013/tree/master/s...

saejox 13y ago

Generated C can be copy pasted and readable by humans. Debugging IR code is a terrible experience.

swannodette 13y ago

People have made this argument for ClojureScript as well. In the case of ClojureScript the benefits are considerably less clear as we rely on the ability of Google Closure to do fairly amazing optimizations on the generated JavaScript source. That said I'd love to see a serious Clojure-in-Clojure targeting LLVM.

cliffbean 13y ago

Similarly, structured control flow (C's if, for, while, switch etc.) is better than unstructured (LLVM IR just has the equivalent of gotos (yes, LLVM IR has switch too, but it can go anywhere, so it still isn't inherently structured)). So it goes both ways. Some other advantages of C over LLVM IR: ABI compatibility with C automatically. In LLVM IR, being ABI-compatible with C is often a considerable headache and you have to do different things on each platform. You can use C libraries by #including their header files (the way many C libraries were designed to be used), instead of hardwiring all that information (macro expansions, enum values, typedefs, inline functions, struct definitions, etc.) from their header files in your code generator. You have the option of switching C compilers; you're not as locked into a single backend. With a bit of care, you can make your output much more readable. LLVM IR demands either SSA form (most front-ends don't want to do this) or herds of allocas, loads, and stores everywhere. In C, you just say "int x;" to declare an int, and just "x" to refer to it. You can use C features like bitfields, designated initializers, short-circuit operators (&&, ||), compound assignment (+=, -=, etc.), and so on. You can do all these things in LLVM IR, but you have to lower them yourself.

masklinn 13y ago

Generating C is much simpler than LLVM IR (and you can do structured code generation for C: generate and serialize C AST); it's infinitely easier to read and debug; it allows switching compiler; it provides C stdlib to piggyback on with very little overhead. It's a fine strategy to start with, there are more important things than fucking around with LLVM IR at this point, they can switch to LLVM or a native generator if they get the thing effective and off-the ground.