5 ms·

I'm still learning how to build a language, and compiling to c seems like the easiest thing to do, but I'm curious if it's more difficult or not. If my languag

by jokoon 9y ago

I'm still learning how to build a language, and compiling to c seems like the easiest thing to do, but I'm curious if it's more difficult or not. If my language is close to c (functions, scope, variable, types), can I take advantage of it so it's less work in my compiler to let the c compiler catch errors, or must I rewrite a full parser? All I want to do is add pythonic indenting, range loops, maps, and geometric types with their operators (a little similar to shader languages).

tyingq 9y ago
Sounds somewhat like nim. https://nim-lang.org https://nim-lang.org
qznc 9y ago
Using LLVM is a little bit harder but gives you advantages like better debugging. Long term that pays off. If you only play around, using C is fine.
- MaxBarraclough 9y ago
  Vala seems to be doing ok targeting C. I'd hardly say it's just for toys.
  - humanrebar 9y ago
    How does Vala handle debugging info and compiler diagnostics? Do the end users have to analyze the emitted C when something goes wrong?
    - MaxBarraclough 9y ago
      Annoyingly Vala is extremely shy about the generated C, but I believe it uses #pragmas to map the generated C back to the Vala source. I've only dabbled in Vala, but apparently its "-g" flag lets you debug with GDB. https://wiki.gnome.org/Projects/Vala/Tutorial#Debugging https://wiki.gnome.org/Projects/Vala/Tutorial#Debugging
  - qznc 9y ago
    There certainly is a grey area. For example, there are esoteric architectures which LLVM does not support but a C compiler exists. Emitting C is probably easier than developing a few LLVM backends. Still, my simple opinion is: If you are serious, use LLVM. It is not that much harder than outputting C and has clear advantages like debug information.
    - MaxBarraclough 9y ago
      As I mention below, Vala debugging isn't the catastrophe you think: https://wiki.gnome.org/Projects/Vala/Tutorial#Debugging https://wiki.gnome.org/Projects/Vala/Tutorial#Debugging I believe it's based on #pragmas that tell the C compiler (and/or GDB, presumably) which line of which file to blame. Serious work has been done in Vala. It's not fair to dismiss the language as a quaint toy.
      - qznc 9y ago
        Preprocessor directives are not powerful enough [0], imho. [0] https://news.ycombinator.com/item?id=16197856 https://news.ycombinator.com/item?id=16197856
        MaxBarraclough 9y ago
        That isn't an attribute of using C as an IR, it's an attribute of optimising compilers. The solution is to disable optimisation for debug builds, not to avoid C. From the gnome.org page: > The -g command line option tells the Vala compiler to include Vala source code line information in the compiled binary
  - mirekrusin 9y ago
    Nim does it as well, but in case of Vala it's a bit different case as it emits glib based code with GObject and friends - basically whole OO system with containers, event loop and what not. The idea of Vala is that if you're using glib then using Vala will generate glib code as if you'd write it by hand, but much easier. The rationale is much stronger to emit C as you can mix it with existing code etc.
- dom96 9y ago
  I've never had any trouble debugging Nim, which compiles to C. The Nim compiler adds #line directives so you can just use gdb/lldb[1]. Can you elaborate on what debugging advantages LLVM brings? 1 - https://nim-lang.org/blog/2017/10/02/documenting-profiling-and-debugging-nim-code.html#using-gdb-lldb https://nim-lang.org/blog/2017/10/02/documenting-profiling-a...
  - qznc 9y ago
    Does Nim optimize anything on the way to C? That usually destroys useful information. I can give you an example from C compilers. Consider a simple loop: int a[] = ...; for (int i=0; i<max; i++) { print(a[i]); } Something fishy is printed, so I run it in my debugger, break at some point, and print "i". Gdb can usually do that. Now think about the optimizations a compiler probably did: While the index is incremented by 1, we actually step 4 bytes further due to the int array, so the compiler does i+=4 instead to avoid the multiply by 4 instruction. While i starts at 0, we can instead use the "a" pointer and avoid to add an offset at every loop iteration. The code might actually look like this: int *a = ... int end = a + max*4; for (char* x=a; x < end; x+=4) { print(x); } How can gdb print "i"? The trick is DWARF [0], basically the compiler inserts the information: If you want to print i here, then compute it by i=(x-a)/4. tldr: #line directives are not enough [0] https://en.wikipedia.org/wiki/DWARF https://en.wikipedia.org/wiki/DWARF
exDM69 9y ago
I would say that emiting LLVM IR is easier than C. I have done a few toy compiler projects with LLVM and it is very nice. I've never done a compiler that emits C, but I can't imagine it being any easier. At least you'd have to write more boilerplate code. You also can't create JIT compiled REPL using C as easily.
- RX14 9y ago
  The thing about llvm ir is you get a code builder "for free" from llvm, where you're just calling functions to build your ir. However, when emitting C you have to build that yourself and take care to always emit valid code. Not sure how much difference that actually makes though.
  - lucozade 9y ago
    I'd actually argue the other way. If you emit invalid C code, you'll at least be told by the compiler (though the error might be pretty obscure). LLVM tends to assume that you are emitting valid IR and panics when you don't. BTW this isn't a criticism of LLVM (far from it). It's expected to be used as a backend so any invalid IR is, pretty much by definition, a programmer error. At the quick and dirty level, IMO C wins here as you can just splat the C you're building to a file and inspect it. With LLVM you need to get through the hurdle of building valid IR before you can serialise it. In the longer term, I would argue that LLVM is better as you have way more control over what you're generating. Once your lowering works then the issue above is irrelevant.
    - jcranmer 9y ago
      > I'd actually argue the other way. If you emit invalid C code, you'll at least be told by the compiler (though the error might be pretty obscure). LLVM tends to assume that you are emitting valid IR and panics when you don't. LLVM does have a verifier pass which should catch every invalid IR. However, optimization passes aren't necessarily robust against weird types (I don't expect <13 x i3> to go over very well), and the backends are liable to flip out if the IR isn't in the subset they expect (e.g., weird types).
lucozade 9y ago
As described, you’ll almost definitely need to write a parser. However, if your type system is very similar to C’s you may be able to just lower from the AST to C and let it worry about semantic analysis and code generation. The biggest loss is that a lot of error and debug info will be relative to the C rather than your language. This means the user will be exposed to the internals. Probably the biggest win, is that it’s almost trivial to integrate C, and potentially C++ libraries into your language. This has been hugely beneficial to Nim, for example. They have written a full front end though so they don’t have the error issue above. Like most things it’s a trade off.
- humanrebar 9y ago
  Not mentioned in that list of downsides is that you need to at least embrace the calling conventions, threading models, and memory models that C supports, at least for the flavors you target. > The biggest loss is that a lot of error and debug info will be relative to the C rather than your language. This means the user will be exposed to the internals. In other words, sometimes the end user has to debug a bunch of code they didn't write. I wouldn't underestimate how much a pain this can be. > Probably the biggest win, is that it’s almost trivial to integrate C, and potentially C++ libraries into your language. Again, subject to adhering to threading models, etc. If you're writing a language as a hobby, this may not be a big concern, but at scale, someone will be using memcpy or memory barriers or thread-local memory somewhere. It is certainly possible to support those use cases, but I'd be careful about overstating how trivial that is.
dtech 9y ago
> let the c compiler catch errors, or must I rewrite a full parser? In the long run you always want to do it specifically for your language, because otherwise giving usable error messages and debugging information becomes impossible.
dfox 9y ago
You can implemement reasonably C-like languages (eg. Objective C and various C-derived DSLs in many Lisp implementations) by a simple text transforming preprocessor and get reasonable error messages from the C backend (#file is your friend for achieving that). But for anything more complex you are better off implementing complete parser with your own error handling and reporting. Edit: one of the clearest signs of language implemented in this way is use of '@' character as part of syntax extensions as it is the only printable ascii character that has no syntactic meaning in C (the other such "unused character" is '$', but its meaning is implementation-defined, for example gcc has option that causes it to be accepted as part of identifiers, which is quite obviously the default on VMS)

by jokoon 9y ago

tyingq 9y ago

Sounds somewhat like nim. https://nim-lang.org https://nim-lang.org

qznc 9y ago

Using LLVM is a little bit harder but gives you advantages like better debugging. Long term that pays off. If you only play around, using C is fine.

MaxBarraclough 9y ago
Vala seems to be doing ok targeting C. I'd hardly say it's just for toys.
- humanrebar 9y ago
  How does Vala handle debugging info and compiler diagnostics? Do the end users have to analyze the emitted C when something goes wrong?
  - MaxBarraclough 9y ago
    Annoyingly Vala is extremely shy about the generated C, but I believe it uses #pragmas to map the generated C back to the Vala source. I've only dabbled in Vala, but apparently its "-g" flag lets you debug with GDB. https://wiki.gnome.org/Projects/Vala/Tutorial#Debugging https://wiki.gnome.org/Projects/Vala/Tutorial#Debugging
- qznc 9y ago
  There certainly is a grey area. For example, there are esoteric architectures which LLVM does not support but a C compiler exists. Emitting C is probably easier than developing a few LLVM backends. Still, my simple opinion is: If you are serious, use LLVM. It is not that much harder than outputting C and has clear advantages like debug information.
  - MaxBarraclough 9y ago
    As I mention below, Vala debugging isn't the catastrophe you think: https://wiki.gnome.org/Projects/Vala/Tutorial#Debugging https://wiki.gnome.org/Projects/Vala/Tutorial#Debugging I believe it's based on #pragmas that tell the C compiler (and/or GDB, presumably) which line of which file to blame. Serious work has been done in Vala. It's not fair to dismiss the language as a quaint toy.
    - qznc 9y ago
      Preprocessor directives are not powerful enough [0], imho. [0] https://news.ycombinator.com/item?id=16197856 https://news.ycombinator.com/item?id=16197856
      - MaxBarraclough 9y ago
        That isn't an attribute of using C as an IR, it's an attribute of optimising compilers. The solution is to disable optimisation for debug builds, not to avoid C. From the gnome.org page: > The -g command line option tells the Vala compiler to include Vala source code line information in the compiled binary
- mirekrusin 9y ago
  Nim does it as well, but in case of Vala it's a bit different case as it emits glib based code with GObject and friends - basically whole OO system with containers, event loop and what not. The idea of Vala is that if you're using glib then using Vala will generate glib code as if you'd write it by hand, but much easier. The rationale is much stronger to emit C as you can mix it with existing code etc.
dom96 9y ago
I've never had any trouble debugging Nim, which compiles to C. The Nim compiler adds #line directives so you can just use gdb/lldb[1]. Can you elaborate on what debugging advantages LLVM brings? 1 - https://nim-lang.org/blog/2017/10/02/documenting-profiling-and-debugging-nim-code.html#using-gdb-lldb https://nim-lang.org/blog/2017/10/02/documenting-profiling-a...
- qznc 9y ago
  Does Nim optimize anything on the way to C? That usually destroys useful information. I can give you an example from C compilers. Consider a simple loop: int a[] = ...; for (int i=0; i<max; i++) { print(a[i]); } Something fishy is printed, so I run it in my debugger, break at some point, and print "i". Gdb can usually do that. Now think about the optimizations a compiler probably did: While the index is incremented by 1, we actually step 4 bytes further due to the int array, so the compiler does i+=4 instead to avoid the multiply by 4 instruction. While i starts at 0, we can instead use the "a" pointer and avoid to add an offset at every loop iteration. The code might actually look like this: int *a = ... int end = a + max*4; for (char* x=a; x < end; x+=4) { print(x); } How can gdb print "i"? The trick is DWARF [0], basically the compiler inserts the information: If you want to print i here, then compute it by i=(x-a)/4. tldr: #line directives are not enough [0] https://en.wikipedia.org/wiki/DWARF https://en.wikipedia.org/wiki/DWARF

exDM69 9y ago

I would say that emiting LLVM IR is easier than C. I have done a few toy compiler projects with LLVM and it is very nice. I've never done a compiler that emits C, but I can't imagine it being any easier. At least you'd have to write more boilerplate code. You also can't create JIT compiled REPL using C as easily.

RX14 9y ago
The thing about llvm ir is you get a code builder "for free" from llvm, where you're just calling functions to build your ir. However, when emitting C you have to build that yourself and take care to always emit valid code. Not sure how much difference that actually makes though.
- lucozade 9y ago
  I'd actually argue the other way. If you emit invalid C code, you'll at least be told by the compiler (though the error might be pretty obscure). LLVM tends to assume that you are emitting valid IR and panics when you don't. BTW this isn't a criticism of LLVM (far from it). It's expected to be used as a backend so any invalid IR is, pretty much by definition, a programmer error. At the quick and dirty level, IMO C wins here as you can just splat the C you're building to a file and inspect it. With LLVM you need to get through the hurdle of building valid IR before you can serialise it. In the longer term, I would argue that LLVM is better as you have way more control over what you're generating. Once your lowering works then the issue above is irrelevant.
  - jcranmer 9y ago
    > I'd actually argue the other way. If you emit invalid C code, you'll at least be told by the compiler (though the error might be pretty obscure). LLVM tends to assume that you are emitting valid IR and panics when you don't. LLVM does have a verifier pass which should catch every invalid IR. However, optimization passes aren't necessarily robust against weird types (I don't expect <13 x i3> to go over very well), and the backends are liable to flip out if the IR isn't in the subset they expect (e.g., weird types).

lucozade 9y ago

As described, you’ll almost definitely need to write a parser. However, if your type system is very similar to C’s you may be able to just lower from the AST to C and let it worry about semantic analysis and code generation. The biggest loss is that a lot of error and debug info will be relative to the C rather than your language. This means the user will be exposed to the internals. Probably the biggest win, is that it’s almost trivial to integrate C, and potentially C++ libraries into your language. This has been hugely beneficial to Nim, for example. They have written a full front end though so they don’t have the error issue above. Like most things it’s a trade off.

humanrebar 9y ago
Not mentioned in that list of downsides is that you need to at least embrace the calling conventions, threading models, and memory models that C supports, at least for the flavors you target. > The biggest loss is that a lot of error and debug info will be relative to the C rather than your language. This means the user will be exposed to the internals. In other words, sometimes the end user has to debug a bunch of code they didn't write. I wouldn't underestimate how much a pain this can be. > Probably the biggest win, is that it’s almost trivial to integrate C, and potentially C++ libraries into your language. Again, subject to adhering to threading models, etc. If you're writing a language as a hobby, this may not be a big concern, but at scale, someone will be using memcpy or memory barriers or thread-local memory somewhere. It is certainly possible to support those use cases, but I'd be careful about overstating how trivial that is.

dtech 9y ago

> let the c compiler catch errors, or must I rewrite a full parser? In the long run you always want to do it specifically for your language, because otherwise giving usable error messages and debugging information becomes impossible.

dfox 9y ago

You can implemement reasonably C-like languages (eg. Objective C and various C-derived DSLs in many Lisp implementations) by a simple text transforming preprocessor and get reasonable error messages from the C backend (#file is your friend for achieving that). But for anything more complex you are better off implementing complete parser with your own error handling and reporting. Edit: one of the clearest signs of language implemented in this way is use of '@' character as part of syntax extensions as it is the only printable ascii character that has no syntactic meaning in C (the other such "unused character" is '$', but its meaning is implementation-defined, for example gcc has option that causes it to be accepted as part of identifiers, which is quite obviously the default on VMS)