6 ms·

RECC, The Robert Elder Compiler Collection

85 points by ScottWRobinson 11y ago

robertelder 11y ago
Hi, I'm the author and I would be interested in hearing feedback on what you'd like to see next? Directions I can go in: - Fully client side C compiler and IDE - Full access to parse tree, static analysis with rich html annotations for errors - For example, if a function call has arguments that don't match the prototype, show a detailed graphic demonstrating that the arguments don't fit into the function - Write a proper shell for the kernel (right now it responds to single letter commands 'q' 's' 'p'...) - CPU Emulators for more languages - Make the compiler produce faster code (it is currently anti-optimized for code simplicity) - Implement currently unsupported C features (float, double, union, goto, K&R C style functions, ...) - Implement more of the standard library - Work toward having better static analysis than GCC or CLANG (but only for C89) - Write blog posts (about what?) - Work toward compiling the Linux Kernel - Create learning tools to help visualize the OS and program stack as the program executes (In addition to showing stack values, provide type annotations via background colours/graphics eg. (0x10000 <= unsigned int)) - Static analysis for MISRA C (at least the rules that aren't equivalent to the halting problem) - Different CPU word sizes - Advanced in browser profiling - A visual tool/animation that shows you every single step of how the C preprocessor works (for complicated cases like recursive variadic macro functions) - Thousands of other ideas...
- tlarkworthy 11y ago
  Awesome project! The simple CPU target for a complex language excites me. I am looking for a more interesting target for this: https://github.com/tomlarkworthy/firefrack https://github.com/tomlarkworthy/firefrack So my vote would be on getting as much 3rd party stuff compiling as possible with the architecture you have. Linux would be amazing. This guy did some cool work on compiling linux for 8 bit micros. Might be inspiring if you have not seen it: http://dmitry.gr/index.php?proj=07.+Linux+on+8bit&r=05.Projects http://dmitry.gr/index.php?proj=07.+Linux+on+8bit&r=05.Proje...
  - robertelder 11y ago
    Yeah, I actually read that guys blog post to see how practical it would be to run something like linux on a smaller word sized CPU. Since the CPU emulation code is only 325 lines of C, it wouldn't be a lot of work to port it to emulate the 32 bit instructions on a very small word size.
- tptacek 11y ago
  512 registers! Did you do this so you could punt on register allocation?
  - robertelder 11y ago
    In the previous version of the spec, I had a large number of bits that were unused, but once I thought about it I realized that any bits that go unused tautologically guarantee less than optimal performance. I played with the instructions a bit, and it worked out well to allow addressing of registers with 9 bits. Right now, the compiler mainly works as a stack machine and doesn't use more than 10 registers at a time (usually just one or two). In the future, I think it would make sense to make the number of registers big enough that it would fit inside the lowest level CPU cache, since this CPU is mainly designed to be emulated. Storing temporary data in registers instead of the stack has the advantage of not needing to push and pop all the time.
    - tptacek 11y ago
      I like the idea of visualizing the parse tree, and another fun place to go would be to do a real IR, bring things into SSA form, show phi function placement, and then do register allocation and optimization.
      - rayiner 11y ago
        I have in the past used xdot files to good effect for visualizing phi function placement and interference graphs. What would be great is a way to make the result interactive (e.g. drill down from basic blocks to instructions, selectively get more or less detail on voluminous data like live ranges). Graphviz unfortunately isn't suitable for that.
        dman 11y ago
        Would you pay for such a tool?
        11y ago
- ticviking 11y ago
  I haven't been able to play with this much yet, but it feels an accessible way to play with low level computer ideas. I feel like this could hit a sweet spot between "toy computer that doesn't do anything fun", and "fully modern computer that is too complicated to understand the entire stack on" for code camps and teenagers. I'm definitely following with interest, and will be seeing if I can make some fun activities using this for our upcoming code camp. I would love to know what contributions you would like from the community.
  - robertelder 11y ago
    I'm glad to hear that because, although I would like to take this project somewhere where it would be used for real engineering applications, I think it would only currently have practical applications in teaching. I would love to work on this project full time, but unless I could derive income from a project like this, is just isn't practical. In terms of contributions from the community, I think the two most valuable things any of you could do at this point would be: 1) Tell other people and share + like etc on Facebook Twitter etc. 2) Reach out to me and provide feedback on why this project interests you, and where you think it could go. You can reach me at 'robert at robertelder.org'
- tinco 11y ago
  It all looks great. Could you perhaps license your test suite and maybe your libc stuff MIT? I understand if you want to keep your kernel and compiler under some license that still allows for commercial control (like GPL or some dual license scheme) but there's not much commercial value in the test suite and I think it could be of value to others :)
  - robertelder 11y ago
    What do you think about Apache 2.0?
    - tinco 11y ago
      I myself like to keep the world neatly divided in GPL and MIT, but Apache 2.0 looks permissive enough :) What makes you prefer Apache 2.0 over MIT?
      - robertelder 11y ago
        The Apache license explicitly mentions patents. I think that for a project to be successful it should be painless for big companies to use it. Lawyers and IP people at large companies generally have a much more serious tone than programmers when it comes to considering which licenses they can include in their stack.
- zvrba 11y ago
  Related to your project: At my university, we learned basics of assembler programming in an IDE made by one of the professors. It was written in Xaw/Athena widget se. Now I see somebody has ported it to javascript: http://ivanzuzak.info/FRISCjs/webapp/ http://ivanzuzak.info/FRISCjs/webapp/
tptacek 11y ago
This is wonderful.
Zardoz84 11y ago
mmm... I'm thinking about of implementing it (or write a wrapper) on the Trillek virtual computer. The virtual computer architecture is like a S100 bus computer of 32 bits. Any CPU of 32 bits could be plugged on it, and One Page CPU it's far more simple that our TR3200 cpu. To put it, I only need to implement a class that follow this interface : https://github.com/trillek-team/trillek-vcomputer-module/blob/develop/include/cpu.hpp https://github.com/trillek-team/trillek-vcomputer-module/blo... and access ROM/RAM and memory mapped devices using vcomp class read/write funtions.
tinco 11y ago
Great project Rob! I'm writing a C compiler in Haskell at the moment, and I'm totally going to borrow your test suite and libc, it looks very neat :) edit: Oh jeez, I just saw your license file. I'll have to wait :)
- robertelder 11y ago
  I figured that the license doesn't really matter for now, since I don't expect anyone to actually use this for production code (there are still quite a few bugs, and it is not very user friendly). A couple people have brought up the licensing, so I may give that some attention. I was considering Apache 2.0. Any opinions? If you want, you can subscribe your email on http://recc.robertelder.org/ http://recc.robertelder.org/ and I'll update you when the license has changed.
  - david-given 11y ago
    Right now the license forbids anyone from looking at the code, which is publicly available on github, so everyone who has ever looked at your github page --- I'm guessing, every commenter here --- is violating it. That's not good. I'd strongly recommend that you change it ASAP. The main two options are: - The 'I just don't care' license: BSD 2-clause. This allows anyone to do anything with it, other than claim they wrote it. It's the most open of the licenses. http://opensource.org/licenses/BSD-2-Clause http://opensource.org/licenses/BSD-2-Clause - The 'One true way' license: GPLv3. This is the classic copyleft license. It requires any modifications to the program to also be GPLv3'd. This requires quite a lot more thought, because if your compiler runtime is also GPLv3'd it means your compiler can't be used to produce distributable non GPLv3'd binaries, which is bad; so you'll have to license your runtime differently. GPLv3 also makes it really hard to use your program commercially (you may consider this a positive or negative quality). http://opensource.org/licenses/gpl-3.0.html http://opensource.org/licenses/gpl-3.0.html Most other licenses are basically just variations on the them. Also, writing your own license, particularly a more restrictive one ('this software can only by used for education', say) will basically kill any use of your program. And it'll probably be invalid, too. Writing licenses is hard. ... A cautionary tale follows: there is a C compiler called vbcc (http://www.compilers.de/vbcc.html http://www.compilers.de/vbcc.html). It's incredibly small, fast, and produces great code. It's easy to port. But it's released under a look-but-don't-touch license which forbids distribution of modified archives. This means that if I, say, modify it to include a compiler backend for Infocom's Z-machine, which I did (http://cowlark.com/vbcc-z-compiler http://cowlark.com/vbcc-z-compiler), then I cannot distribute a version of vbcc which contains that backend! Instead I have to distribute the unmodified archive and a patch. This makes it way too hard to develop for, and additionally means that no software distributor will touch it. When vbcc's author finally gets hit by a bus, vbcc will die. Which is a shame; it's really nice. Please don't do that!

Hi, I'm the author and I would be interested in hearing feedback on what you'd like to see next? Directions I can go in: - Fully client side C compiler and IDE - Full access to parse tree, static analysis with rich html annotations for errors - For example, if a function call has arguments that don't match the prototype, show a detailed graphic demonstrating that the arguments don't fit into the function - Write a proper shell for the kernel (right now it responds to single letter commands 'q' 's' 'p'...) - CPU Emulators for more languages - Make the compiler produce faster code (it is currently anti-optimized for code simplicity) - Implement currently unsupported C features (float, double, union, goto, K&R C style functions, ...) - Implement more of the standard library - Work toward having better static analysis than GCC or CLANG (but only for C89) - Write blog posts (about what?) - Work toward compiling the Linux Kernel - Create learning tools to help visualize the OS and program stack as the program executes (In addition to showing stack values, provide type annotations via background colours/graphics eg. (0x10000 <= unsigned int)) - Static analysis for MISRA C (at least the rules that aren't equivalent to the halting problem) - Different CPU word sizes - Advanced in browser profiling - A visual tool/animation that shows you every single step of how the C preprocessor works (for complicated cases like recursive variadic macro functions) - Thousands of other ideas...

tlarkworthy 11y ago
Awesome project! The simple CPU target for a complex language excites me. I am looking for a more interesting target for this: https://github.com/tomlarkworthy/firefrack https://github.com/tomlarkworthy/firefrack So my vote would be on getting as much 3rd party stuff compiling as possible with the architecture you have. Linux would be amazing. This guy did some cool work on compiling linux for 8 bit micros. Might be inspiring if you have not seen it: http://dmitry.gr/index.php?proj=07.+Linux+on+8bit&r=05.Projects http://dmitry.gr/index.php?proj=07.+Linux+on+8bit&r=05.Proje...
- robertelder 11y ago
  Yeah, I actually read that guys blog post to see how practical it would be to run something like linux on a smaller word sized CPU. Since the CPU emulation code is only 325 lines of C, it wouldn't be a lot of work to port it to emulate the 32 bit instructions on a very small word size.
tptacek 11y ago
512 registers! Did you do this so you could punt on register allocation?
- robertelder 11y ago
  In the previous version of the spec, I had a large number of bits that were unused, but once I thought about it I realized that any bits that go unused tautologically guarantee less than optimal performance. I played with the instructions a bit, and it worked out well to allow addressing of registers with 9 bits. Right now, the compiler mainly works as a stack machine and doesn't use more than 10 registers at a time (usually just one or two). In the future, I think it would make sense to make the number of registers big enough that it would fit inside the lowest level CPU cache, since this CPU is mainly designed to be emulated. Storing temporary data in registers instead of the stack has the advantage of not needing to push and pop all the time.
  - tptacek 11y ago
    I like the idea of visualizing the parse tree, and another fun place to go would be to do a real IR, bring things into SSA form, show phi function placement, and then do register allocation and optimization.
    - rayiner 11y ago
      I have in the past used xdot files to good effect for visualizing phi function placement and interference graphs. What would be great is a way to make the result interactive (e.g. drill down from basic blocks to instructions, selectively get more or less detail on voluminous data like live ranges). Graphviz unfortunately isn't suitable for that.
      - dman 11y ago
        Would you pay for such a tool?
        11y ago
ticviking 11y ago
I haven't been able to play with this much yet, but it feels an accessible way to play with low level computer ideas. I feel like this could hit a sweet spot between "toy computer that doesn't do anything fun", and "fully modern computer that is too complicated to understand the entire stack on" for code camps and teenagers. I'm definitely following with interest, and will be seeing if I can make some fun activities using this for our upcoming code camp. I would love to know what contributions you would like from the community.
- robertelder 11y ago
  I'm glad to hear that because, although I would like to take this project somewhere where it would be used for real engineering applications, I think it would only currently have practical applications in teaching. I would love to work on this project full time, but unless I could derive income from a project like this, is just isn't practical. In terms of contributions from the community, I think the two most valuable things any of you could do at this point would be: 1) Tell other people and share + like etc on Facebook Twitter etc. 2) Reach out to me and provide feedback on why this project interests you, and where you think it could go. You can reach me at 'robert at robertelder.org'
tinco 11y ago
It all looks great. Could you perhaps license your test suite and maybe your libc stuff MIT? I understand if you want to keep your kernel and compiler under some license that still allows for commercial control (like GPL or some dual license scheme) but there's not much commercial value in the test suite and I think it could be of value to others :)
- robertelder 11y ago
  What do you think about Apache 2.0?
  - tinco 11y ago
    I myself like to keep the world neatly divided in GPL and MIT, but Apache 2.0 looks permissive enough :) What makes you prefer Apache 2.0 over MIT?
    - robertelder 11y ago
      The Apache license explicitly mentions patents. I think that for a project to be successful it should be painless for big companies to use it. Lawyers and IP people at large companies generally have a much more serious tone than programmers when it comes to considering which licenses they can include in their stack.
zvrba 11y ago
Related to your project: At my university, we learned basics of assembler programming in an IDE made by one of the professors. It was written in Xaw/Athena widget se. Now I see somebody has ported it to javascript: http://ivanzuzak.info/FRISCjs/webapp/ http://ivanzuzak.info/FRISCjs/webapp/

tptacek 11y ago

This is wonderful.

Zardoz84 11y ago

mmm... I'm thinking about of implementing it (or write a wrapper) on the Trillek virtual computer. The virtual computer architecture is like a S100 bus computer of 32 bits. Any CPU of 32 bits could be plugged on it, and One Page CPU it's far more simple that our TR3200 cpu. To put it, I only need to implement a class that follow this interface : https://github.com/trillek-team/trillek-vcomputer-module/blob/develop/include/cpu.hpp https://github.com/trillek-team/trillek-vcomputer-module/blo... and access ROM/RAM and memory mapped devices using vcomp class read/write funtions.

tinco 11y ago

Great project Rob! I'm writing a C compiler in Haskell at the moment, and I'm totally going to borrow your test suite and libc, it looks very neat :) edit: Oh jeez, I just saw your license file. I'll have to wait :)

robertelder 11y ago
I figured that the license doesn't really matter for now, since I don't expect anyone to actually use this for production code (there are still quite a few bugs, and it is not very user friendly). A couple people have brought up the licensing, so I may give that some attention. I was considering Apache 2.0. Any opinions? If you want, you can subscribe your email on http://recc.robertelder.org/ http://recc.robertelder.org/ and I'll update you when the license has changed.
- david-given 11y ago
  Right now the license forbids anyone from looking at the code, which is publicly available on github, so everyone who has ever looked at your github page --- I'm guessing, every commenter here --- is violating it. That's not good. I'd strongly recommend that you change it ASAP. The main two options are: - The 'I just don't care' license: BSD 2-clause. This allows anyone to do anything with it, other than claim they wrote it. It's the most open of the licenses. http://opensource.org/licenses/BSD-2-Clause http://opensource.org/licenses/BSD-2-Clause - The 'One true way' license: GPLv3. This is the classic copyleft license. It requires any modifications to the program to also be GPLv3'd. This requires quite a lot more thought, because if your compiler runtime is also GPLv3'd it means your compiler can't be used to produce distributable non GPLv3'd binaries, which is bad; so you'll have to license your runtime differently. GPLv3 also makes it really hard to use your program commercially (you may consider this a positive or negative quality). http://opensource.org/licenses/gpl-3.0.html http://opensource.org/licenses/gpl-3.0.html Most other licenses are basically just variations on the them. Also, writing your own license, particularly a more restrictive one ('this software can only by used for education', say) will basically kill any use of your program. And it'll probably be invalid, too. Writing licenses is hard. ... A cautionary tale follows: there is a C compiler called vbcc (http://www.compilers.de/vbcc.html http://www.compilers.de/vbcc.html). It's incredibly small, fast, and produces great code. It's easy to port. But it's released under a look-but-don't-touch license which forbids distribution of modified archives. This means that if I, say, modify it to include a compiler backend for Infocom's Z-machine, which I did (http://cowlark.com/vbcc-z-compiler http://cowlark.com/vbcc-z-compiler), then I cannot distribute a version of vbcc which contains that backend! Instead I have to distribute the unmodified archive and a patch. This makes it way too hard to develop for, and additionally means that no software distributor will touch it. When vbcc's author finally gets hit by a bus, vbcc will die. Which is a shame; it's really nice. Please don't do that!