7 ms·

Show HN: My own C compiler (just for fun)

175 points by rcorcs 10y ago

pmiller2 10y ago
I will always upvote something like this, but I am wondering: why write it in C++? I didn't really read the whole thing, but I skimmed a bit, and didn't see a lot of advantage taken of the higher-level facilities of C++. It seems like you've picked a route that gives you all the disadvantages of using C, while simultaneously destroying the possibility of it ever being self-hosting. Still, a fun project, and you have my upvote. :)
- codewritinfool 10y ago
  I was wondering the same thing... ?? Oh, and you get an upvote from me too.
- unsignedqword 10y ago
  Not everyone wants to write the most idiomatic C++, so writing variably C-ish code in C++ isn't really all that uncommon nowadays. If you're a C programmer but you notice C++ has a convenient feature (or features) that you might find useful in a pinch, hell, maybe it's worth it to bite the bullet (although that's not to say that using C++ in lieu of C can have its drawbacks, though)
  - pmiller2 10y ago
    I would completely agree with you if this weren't a C compiler project. In this case, the advantage of having a self-hosting compiler is huge! Among other things, you can use the compiler source itself to test the compiler, which is pretty cool. I have worked on projects that were essentially "C with classes," and it was fine. I'm not saying every C++ project needs to go all out using every feature from C++ 11, but I didn't even see much beyond use of the string class here. It seems not worth giving up self-hosting to me.
    - mac01021 10y ago
      > the advantage of having a self-hosting compiler is huge! I agree that it's kind of neat to compile your compiler with itself, but it's not clear to me that this constitutes a significant productivity boost. There are countless other C programs that have already been written and can be used as test cases. So I'm left wondering what the huge advantage that self-hosting provides is. I am interested so, if you have more to say on the topic, I will certainly read it.
      - userbinator 10y ago
        If we're talking about toy compilers and learning, then productivity isn't really in consideration. Having a self-hosting compiler for a non-trivial language means you've achieved the "bootstrap" stage, which is a significant milestone and really gives you a practical understanding of what's involved in developing a programming language. A compiler that is powerful enough to compile itself is on the way to exiting the "toy"/"theoretical" realm that most people who study compilers don't ever go beyond.
    - apk17 10y ago
      You could just start to replace C++ things by plain C until it is self-hosting.
- userbinator 10y ago
  while simultaneously destroying the possibility of it ever being self-hosting. ...at least until it begins turning into a C++ compiler ;-) That makes me wonder, given that there seem to be quite a lot of "toy" C/C-subset compilers being written these days, just how much harder C++ would be. For a long time it was commonly thought that even the simplest C compiler would be extremely complex and difficult, but that notion appears to have been somewhat defeated, so the next logical step in that direction is C++ or maybe C++-to-C (like Cfront) compilers. The closest I'm aware of would be the C++-subset used in TempleOS, which is self-hosting.
  - apk17 10y ago
    Horribly. C++ has so many features, and interactions between them, that you really don't want to be dragged into that as a toy project. I'd wager that the C++ language spec is longer than the source of a functional C compiler (both without the standard libraries).
    - userbinator 10y ago
      Maybe the interactions aren't actually the issue because the features are quite orthogonal and independent of each other, and the complexity is really the natural result of the interactions between them. If I wanted to write a C++ compiler I'd probably start by adding features in this order: - Member functions in structures; constructors; destructors - Single inheritance - Virtual functions - Exceptions - Multiple inheritance - Virtual inheritance - Templates There's definitely more I haven't listed, but the first few don't seem too difficult...
- reacweb 10y ago
  Having a self hosting compiler gives a very handy tool to assess the gains of a speed optimisation: if the speed improvement of the compiler compensate the more complex compilation algorithm, then it is justified.
m1n1 10y ago
Where does one go for a fully representative sample of all kinds of valid C code to test a project like this? Or -- given a formal grammar, has someone produced a tool that generates representative code?
- robertelder 10y ago
  For specific test cases, you could use gcc's test cases: https://github.com/gcc-mirror/gcc/tree/master/gcc/testsuite https://github.com/gcc-mirror/gcc/tree/master/gcc/testsuite Not sure about the latter, but you could probably accomplish something similar without too much work using a fuzzer.
- moosingin3space 10y ago
  There's csmith: https://embed.cs.utah.edu/csmith/ https://embed.cs.utah.edu/csmith/
  - nickpsecurity 10y ago
    I give +1 to Csmith recommendation. They used it to tear up all kinds of compilers with practical bugs found as a result. They even found a few in CompCert despite formal proofs due to specification errors. The middle-end came out flawless, though. So, such results speak for themselves. I think Csmith has to be the baseline. Not sure if it's easy to tune for the equivalent of basic, acceptance tests, though. If not, then it could be nice to have a long list of tests that each correspond to specific, C-language features to help a compiler writer gradually implement one.
    - zzzcpan 10y ago
      For gradual implementation I would start from a test suite in tcc [1]. Eventually adopting the one from gcc and csmith. [1] http://bellard.org/tcc/ http://bellard.org/tcc/
      - nickpsecurity 10y ago
        Forgot about that one. Thank you! I remember it being small and fast but didn't know it had memory and bounds checking. That's a great combination.
sheepleherd 10y ago
the computer science / computer programming problem I'd like to see solved is, keeping projects "fresh" and open/accessible enough that people like this could feel like they were learning in an unencumbered way, and at the same time contributing something useful to an existing project, while at the same time pushing the capabilities of what available open source projects can provide. "Reinventing the wheel" projects absolutely litter public source nodes; believe me, I know why people do it; but my dream is the dream of software that most of us have given up on, code reuse, "reentrancy", shared libraries, etc. Maybe something like a "wikipedia of source code". I'm not discounting the benefit of doing a project to learn about it; what I'm saying is, too bad it's not code that will be useful for anything else without a lot more work; and too bad work is going into something that is not reuseful-able.
- marvy 10y ago
  more details?
  - sheepleherd 10y ago
    more detailed question please? I'm happy to discuss but I'm not sure whether you are looking for bottom up nitty gritty details or more top down grounded philosophizing.
    - marvy 10y ago
      Looking at the rest of the thread helped. Thanks!
- adrianN 10y ago
  Any nontrivial project requires lots of time just to understand its design. Even minimal contributions will likely require comparable amounts of work to completing a toy project. And as any professional programmer knows, reading other people's code is a lot less fun than writing your own.
  - sheepleherd 10y ago
melling 10y ago
There are several other C compilers on Github. I list a few of them here. https://github.com/melling/ComputerLanguages/blob/master/compilers.org https://github.com/melling/ComputerLanguages/blob/master/com... They probably deserve their own section.
- andrewchambers 10y ago
  could you add my own self hosting C compiler: https://github.com/andrewchambers/c https://github.com/andrewchambers/c
  - melling 10y ago
    Added it to the Compiler section: https://github.com/melling/ComputerLanguages/blob/master/compilers.org#compilers https://github.com/melling/ComputerLanguages/blob/master/com...
- pkd 10y ago
  Unrelated to the thread, but I opened a PR on your repo the last time you posted about it here. Right now it is still open :/
  - melling 10y ago
    I'll have to fix it after my vacation. There's now a conflict. I didn't see it because I'm subscribed to too many github repos.
rcorcs 10y ago
I know there are hundreds of C compilers and that I am just reinventing the wheel. But my main goal is just learning. I have already written another compiler before, but it was just for a toy language. For this reason, I would like to build a compiler for a "real-life" language, and C is an important but yet small language. I want to master the whole process of a fully working compiler for a "real-life" language, and afterwards continue to build on top of this knowledge, since I have been doing research on automatic parallelisation, and I am interested in optimising compilers in general. Even if this project doesn't turn out to be useful for a lot of people, I hope at least to inspire a few people to tackle big problems.
- d99kris 10y ago
  The code base looks nice and easy to navigate. I didn't see any license mentioned - I'd suggest stating one clearly for those considering to build something on top of it, or to contribute.
- warmwaffles 10y ago
  kudos to you, this is something I don't think I could ever accomplish but I know it's not easy. Keep learning and growing.
- userbinator 10y ago
  I'm curious to know your reasoning behind using a generated parser instead of writing your own. This might be the first "just for fun" C compiler I've seen on HN or elsewhere that doesn't use some variant of recursive-descent/precedence, which is employed even by the "real" compilers like GCC, Clang, ICC, and MSVC. If you are doing this for learning then I'd recommend studying C4's parser (https://news.ycombinator.com/item?id=8558822 https://news.ycombinator.com/item?id=8558822 ) and this article: https://www.engr.mun.ca/~theo/Misc/exp_parsing.htm https://www.engr.mun.ca/~theo/Misc/exp_parsing.htm The precedence-climbing algorithm is so amazingly simple and concise that I think anyone playing with compilers should implement a parser using it at least once, just to experience its astounding elegance. IMHO seeing an entire programming language's AST parsed almost entirely using a single recursive function with very little code and a table concisely describing its grammar can be quite exhilarating.
cyphar 10y ago
Can you please add a free software license to your project, as it's currently proprietary? I'd recommend GPL, but go with whatever you want. Contrary to popular opinion, copyright applies automatically to all of your works (unless you are an employee of the US Government). Due to the draconian nature of copyright laws, people have very few practical freedoms with your work unless you use a free software license (GPL, MIT, Apache, etc). EDIT: Actually, looking through your GitHub profile it looks like most of your projects don't have free software licenses. Can you please rectify this, as it's clear you're doing cool work but it's not free software (or even "open source").

175 points by rcorcs 10y ago

I will always upvote something like this, but I am wondering: why write it in C++? I didn't really read the whole thing, but I skimmed a bit, and didn't see a lot of advantage taken of the higher-level facilities of C++. It seems like you've picked a route that gives you all the disadvantages of using C, while simultaneously destroying the possibility of it ever being self-hosting. Still, a fun project, and you have my upvote. :)

codewritinfool 10y ago
I was wondering the same thing... ?? Oh, and you get an upvote from me too.
unsignedqword 10y ago
Not everyone wants to write the most idiomatic C++, so writing variably C-ish code in C++ isn't really all that uncommon nowadays. If you're a C programmer but you notice C++ has a convenient feature (or features) that you might find useful in a pinch, hell, maybe it's worth it to bite the bullet (although that's not to say that using C++ in lieu of C can have its drawbacks, though)
- pmiller2 10y ago
  I would completely agree with you if this weren't a C compiler project. In this case, the advantage of having a self-hosting compiler is huge! Among other things, you can use the compiler source itself to test the compiler, which is pretty cool. I have worked on projects that were essentially "C with classes," and it was fine. I'm not saying every C++ project needs to go all out using every feature from C++ 11, but I didn't even see much beyond use of the string class here. It seems not worth giving up self-hosting to me.
  - mac01021 10y ago
    > the advantage of having a self-hosting compiler is huge! I agree that it's kind of neat to compile your compiler with itself, but it's not clear to me that this constitutes a significant productivity boost. There are countless other C programs that have already been written and can be used as test cases. So I'm left wondering what the huge advantage that self-hosting provides is. I am interested so, if you have more to say on the topic, I will certainly read it.
    - userbinator 10y ago
      If we're talking about toy compilers and learning, then productivity isn't really in consideration. Having a self-hosting compiler for a non-trivial language means you've achieved the "bootstrap" stage, which is a significant milestone and really gives you a practical understanding of what's involved in developing a programming language. A compiler that is powerful enough to compile itself is on the way to exiting the "toy"/"theoretical" realm that most people who study compilers don't ever go beyond.
  - apk17 10y ago
    You could just start to replace C++ things by plain C until it is self-hosting.
userbinator 10y ago
while simultaneously destroying the possibility of it ever being self-hosting. ...at least until it begins turning into a C++ compiler ;-) That makes me wonder, given that there seem to be quite a lot of "toy" C/C-subset compilers being written these days, just how much harder C++ would be. For a long time it was commonly thought that even the simplest C compiler would be extremely complex and difficult, but that notion appears to have been somewhat defeated, so the next logical step in that direction is C++ or maybe C++-to-C (like Cfront) compilers. The closest I'm aware of would be the C++-subset used in TempleOS, which is self-hosting.
- apk17 10y ago
  Horribly. C++ has so many features, and interactions between them, that you really don't want to be dragged into that as a toy project. I'd wager that the C++ language spec is longer than the source of a functional C compiler (both without the standard libraries).
  - userbinator 10y ago
    Maybe the interactions aren't actually the issue because the features are quite orthogonal and independent of each other, and the complexity is really the natural result of the interactions between them. If I wanted to write a C++ compiler I'd probably start by adding features in this order: - Member functions in structures; constructors; destructors - Single inheritance - Virtual functions - Exceptions - Multiple inheritance - Virtual inheritance - Templates There's definitely more I haven't listed, but the first few don't seem too difficult...
reacweb 10y ago
Having a self hosting compiler gives a very handy tool to assess the gains of a speed optimisation: if the speed improvement of the compiler compensate the more complex compilation algorithm, then it is justified.

m1n1 10y ago

Where does one go for a fully representative sample of all kinds of valid C code to test a project like this? Or -- given a formal grammar, has someone produced a tool that generates representative code?

robertelder 10y ago
For specific test cases, you could use gcc's test cases: https://github.com/gcc-mirror/gcc/tree/master/gcc/testsuite https://github.com/gcc-mirror/gcc/tree/master/gcc/testsuite Not sure about the latter, but you could probably accomplish something similar without too much work using a fuzzer.
moosingin3space 10y ago
There's csmith: https://embed.cs.utah.edu/csmith/ https://embed.cs.utah.edu/csmith/
- nickpsecurity 10y ago
  I give +1 to Csmith recommendation. They used it to tear up all kinds of compilers with practical bugs found as a result. They even found a few in CompCert despite formal proofs due to specification errors. The middle-end came out flawless, though. So, such results speak for themselves. I think Csmith has to be the baseline. Not sure if it's easy to tune for the equivalent of basic, acceptance tests, though. If not, then it could be nice to have a long list of tests that each correspond to specific, C-language features to help a compiler writer gradually implement one.
  - zzzcpan 10y ago
    For gradual implementation I would start from a test suite in tcc [1]. Eventually adopting the one from gcc and csmith. [1] http://bellard.org/tcc/ http://bellard.org/tcc/
    - nickpsecurity 10y ago
      Forgot about that one. Thank you! I remember it being small and fast but didn't know it had memory and bounds checking. That's a great combination.

sheepleherd 10y ago

the computer science / computer programming problem I'd like to see solved is, keeping projects "fresh" and open/accessible enough that people like this could feel like they were learning in an unencumbered way, and at the same time contributing something useful to an existing project, while at the same time pushing the capabilities of what available open source projects can provide. "Reinventing the wheel" projects absolutely litter public source nodes; believe me, I know why people do it; but my dream is the dream of software that most of us have given up on, code reuse, "reentrancy", shared libraries, etc. Maybe something like a "wikipedia of source code". I'm not discounting the benefit of doing a project to learn about it; what I'm saying is, too bad it's not code that will be useful for anything else without a lot more work; and too bad work is going into something that is not reuseful-able.

marvy 10y ago
more details?
- sheepleherd 10y ago
  more detailed question please? I'm happy to discuss but I'm not sure whether you are looking for bottom up nitty gritty details or more top down grounded philosophizing.
  - marvy 10y ago
    Looking at the rest of the thread helped. Thanks!
adrianN 10y ago
Any nontrivial project requires lots of time just to understand its design. Even minimal contributions will likely require comparable amounts of work to completing a toy project. And as any professional programmer knows, reading other people's code is a lot less fun than writing your own.
- sheepleherd 10y ago

melling 10y ago

There are several other C compilers on Github. I list a few of them here. https://github.com/melling/ComputerLanguages/blob/master/compilers.org https://github.com/melling/ComputerLanguages/blob/master/com... They probably deserve their own section.

andrewchambers 10y ago
could you add my own self hosting C compiler: https://github.com/andrewchambers/c https://github.com/andrewchambers/c
- melling 10y ago
  Added it to the Compiler section: https://github.com/melling/ComputerLanguages/blob/master/compilers.org#compilers https://github.com/melling/ComputerLanguages/blob/master/com...
pkd 10y ago
Unrelated to the thread, but I opened a PR on your repo the last time you posted about it here. Right now it is still open :/
- melling 10y ago
  I'll have to fix it after my vacation. There's now a conflict. I didn't see it because I'm subscribed to too many github repos.

rcorcs 10y ago

I know there are hundreds of C compilers and that I am just reinventing the wheel. But my main goal is just learning. I have already written another compiler before, but it was just for a toy language. For this reason, I would like to build a compiler for a "real-life" language, and C is an important but yet small language. I want to master the whole process of a fully working compiler for a "real-life" language, and afterwards continue to build on top of this knowledge, since I have been doing research on automatic parallelisation, and I am interested in optimising compilers in general. Even if this project doesn't turn out to be useful for a lot of people, I hope at least to inspire a few people to tackle big problems.

d99kris 10y ago
The code base looks nice and easy to navigate. I didn't see any license mentioned - I'd suggest stating one clearly for those considering to build something on top of it, or to contribute.
warmwaffles 10y ago
kudos to you, this is something I don't think I could ever accomplish but I know it's not easy. Keep learning and growing.
userbinator 10y ago
I'm curious to know your reasoning behind using a generated parser instead of writing your own. This might be the first "just for fun" C compiler I've seen on HN or elsewhere that doesn't use some variant of recursive-descent/precedence, which is employed even by the "real" compilers like GCC, Clang, ICC, and MSVC. If you are doing this for learning then I'd recommend studying C4's parser (https://news.ycombinator.com/item?id=8558822 https://news.ycombinator.com/item?id=8558822 ) and this article: https://www.engr.mun.ca/~theo/Misc/exp_parsing.htm https://www.engr.mun.ca/~theo/Misc/exp_parsing.htm The precedence-climbing algorithm is so amazingly simple and concise that I think anyone playing with compilers should implement a parser using it at least once, just to experience its astounding elegance. IMHO seeing an entire programming language's AST parsed almost entirely using a single recursive function with very little code and a table concisely describing its grammar can be quite exhilarating.

cyphar 10y ago

Can you please add a free software license to your project, as it's currently proprietary? I'd recommend GPL, but go with whatever you want. Contrary to popular opinion, copyright applies automatically to all of your works (unless you are an employee of the US Government). Due to the draconian nature of copyright laws, people have very few practical freedoms with your work unless you use a free software license (GPL, MIT, Apache, etc). EDIT: Actually, looking through your GitHub profile it looks like most of your projects don't have free software licenses. Can you please rectify this, as it's clear you're doing cool work but it's not free software (or even "open source").