8 ms·
C compiler with support for structs written in Assembly
- exikyut 8y agoVital details: From http://git.savannah.nongnu.org/cgit/stage0.git/tree/README http://git.savannah.nongnu.org/cgit/stage0.git/tree/README (which also declares GPL3): > This is a set of manually created hex programs in a Cthulhu Path to madness fashion. Which only have the goal of creating a bootstrapping path to a C compiler capable of Compiling GCC, with only the explicit requirement of a single 1 KByte binary or less. > Additionally, all code must be able to be understood by 70% of the population of programmers. If the code can not be understood by that volume, it needs to be altered until it satifies the above requirement. From https://savannah.nongnu.org/projects/stage0/ https://savannah.nongnu.org/projects/stage0/: > A class of minimal bootstrap binaries that has a reproducible build on all platforms. Providing a verifiable base for defeating the trusting trust attack. Cooooooool.
- rasengan 8y agoI am in the bottom 30 percentile of programmers. :(
- shawn 8y agoEveryone was. Work hard and force yourself to learn something alien each day.
- brutal_chaos 8y ago[A bit OT] Brilliantly said. I always enjoy reading encouraging statements like these; especially in response to self doubt.
- todd8 8y agoShawn’s perspective is great. Like chess players or golfers, there is (almost) always someone better, but practice improves our ability. Programming takes practice, but it gets more satisfying the more you do of it. My first programming language was Fortran, but I couldn’t get my first program to compile! I never did. I switched to something easier, Basic. Then went back to Fortran successfully. I toyed with programming for years before deciding on it as a career. Ten years from my first attempts I was a good programmer doing really interesting things. In twenty years I was architecting distributed systems for IBM’s new AIX operating systems. In thirty years I was chief scientist at a very successful software company. Key to my success was to keep learning from people smarter than me by reading, taking classes, and practice doing hard things. Although it has now been fifty years, I still love programming.
- giomasce 8y agoI would not say that, in its current form, that code is understandable by 70% of programmers, unless you define that word "programmer" in a very restrictive sense.
- maxpert 8y agoMind blown
- jacquesm 8y agoThat's impressive. I wished I had time today to put this through its paces.
- z29LiTp5qUC30n 8y agoThe C version is alot easier to read: https://github.com/oriansj/M2-Planet https://github.com/oriansj/M2-Planet
- ailideex 8y agobutwhy.jpg ...
- sandov 8y agoThen how do they usually bootstrap C? Do they write a C compiler without structs, then program a C with structs compiler using C without structs?
- garmaine 8y agoThey cross-compile on a machine that already has a C compiler.
- fb03 8y agoBut how was the first compiler bootstrapped? Serious question (how can a compiler of X language be written in X?)
- coliveira 8y agoThey wrote in another language. I think the bootstrap C compiler was written in BCPL.
- zadler 8y agoThsts awesome. Id love to see a lineage tree of early language development.
- pieterr 8y agoMore info can be found here (straight from the source...) https://www.bell-labs.com/usr/dmr/www/chist.html https://www.bell-labs.com/usr/dmr/www/chist.html https://www.bell-labs.com/usr/dmr/www/primevalC.html https://www.bell-labs.com/usr/dmr/www/primevalC.html
- garmaine 8y agoAnd, for example, C++ was originally a C program that transpiled C++ -> C.
- sigjuice 8y ago
- mmastrac 8y agoI started working on a similar VM-based bootstrap for getting from bare metal to C compiler. If anyone is interested in collaborating let me know. The idea is that you implement a simple, ASCII-based VM for your platform and that's enough to bootstrap a number of different assemblers, to a basic C compiler, to a full-fledged C compiler and (very minimal) POSIX environment. The goal is twofold: a base for trusted compilation like this one, and a way to guarantee long-term executability of various archiving programs (ie: guarantee that you can unrar a file in 20 years with minimal work). EDIT: very rough repo is here - https://github.com/mmastrac/bootstrap https://github.com/mmastrac/bootstrap
- ailideex 8y agoWhy not just cross compile!?
- giomasce 8y agoBecause when you cross compile you have to trust your compiler. https://en.wikipedia.org/wiki/Backdoor_(computing)#Compiler_backdoors https://en.wikipedia.org/wiki/Backdoor_(computing)#Compiler_...
- ailideex 8y agoAnd when you compile regularly you don't?
- giomasce 8y agoYes, but if you want to rebuild the trust chain you cannot restart from an untrusted compiler. The idea is to bootstrap a trusted compiler from nothing (i.e., from little enough so that you can check it directly), and then use the trusted compiler for everything else.
- geezerjay 8y agoWhen you compile regularly you don't, mainly because you're more interested in ensuring your code isn't broken and you have no intention to ship the binaries to anyone.
- tempodox 8y agoWhat assembler syntax is this? The comments don't say.
- z29LiTp5qUC30n 8y agoM0 macro assembly, it is implemented here: http://git.savannah.nongnu.org/cgit/stage0.git/tree/stage1/M0-macro.hex2 http://git.savannah.nongnu.org/cgit/stage0.git/tree/stage1/M... Which is Built in Hex2 which is implemented here: http://git.savannah.nongnu.org/cgit/stage0.git/tree/stage1/stage1_assembler-2.hex1 http://git.savannah.nongnu.org/cgit/stage0.git/tree/stage1/s... Which is built in Hex1 which is implemented here: http://git.savannah.nongnu.org/cgit/stage0.git/tree/stage1/stage1_assembler-1.hex0 http://git.savannah.nongnu.org/cgit/stage0.git/tree/stage1/s... Which is self-hosting and implemented here: http://git.savannah.nongnu.org/cgit/stage0.git/tree/stage1/stage1_assembler-0.hex0 http://git.savannah.nongnu.org/cgit/stage0.git/tree/stage1/s... And was written using the bare metal text editor found here: http://git.savannah.nongnu.org/cgit/stage0.git/tree/stage1/SET.s http://git.savannah.nongnu.org/cgit/stage0.git/tree/stage1/S...
- meffie 8y agoLooks to be bytecode for a vm; http://git.savannah.nongnu.org/cgit/stage0.git/tree/vm_instructions.c http://git.savannah.nongnu.org/cgit/stage0.git/tree/vm_instr...
- another-cuppa 8y agoJust a few hours ago there was a conversation in #gentoo-chat about the Thompson attack. I wondered how difficult it would be to write a C compiler that can compile GCC. How far away is this?
- dleslie 8y agoIt just needs to compile enough C to compile a compiler that can compile GCC.
- em3rgent0rdr 8y agoYes. And should emphasize that the bootstrapping compiler doesn't need to be an optimizing compiler, and doesn't need to compile fast, since it is only to be used once for compiling the next stage of compiler.
- chubot 8y agoIsn't GCC written in C++ now? I think they are probably aiming for a very old version of GCC written in C (or that's all they can really hope for). The now-defunct Aboriginal Linux project was doing a similar sort of bootstrapping, and the build dependencies for GCC was a big issue: http://landley.net/aboriginal/ http://landley.net/aboriginal/ To work around this, it never used anything later than gcc-4.2.1 from 2007, while we're now on GCC 8.2. http://gcc.gnu.org/releases.html http://gcc.gnu.org/releases.html EDIT: Yes, it appears GCC uses C++: https://lwn.net/Articles/542457/ https://lwn.net/Articles/542457/
- earenndil 8y agoYes, but you can use gcc 4.2.1 to compile a newer version which does use c++.
- z29LiTp5qUC30n 8y agoWell, there is work to have MesCC bootstrapped off this: https://gitlab.com/janneke/mes https://gitlab.com/janneke/mes Which can build GCC. So literally once that step is done, we would have a full path...
- pnathan 8y agoVery, very nice. Not often you get to study non-trivial assembly programs. Some context is this: https://bootstrapping.miraheze.org/wiki/Stage0 https://bootstrapping.miraheze.org/wiki/Stage0
- akkartik 8y agoThis is fantastic. I see it's mostly written by the author of the project. Why the heck isn't it in the repo's Readme?!
- pecg 8y agoThis program looks interesting. Maybe it could help in the bootstrap process of tcc, though I don't know if it is written in ANSI C. On a related note, I will try to test compiling lua's runtime environment and interpreter, because I'm sure it is written using standard C already.
- ailideex 8y ago> Maybe it could help in the bootstrap process Why would it be more useful than just cross compiling?
- nickpsecurity 8y agoPart of the goal of these efforts is doing that. For example, here's a page tracking which tools can compile which others: https://bootstrapping.miraheze.org/wiki/C_compilers https://bootstrapping.miraheze.org/wiki/C_compilers
- lboasso 8y agoBootstrapping a compiler is fun. I wrote a self-hosting compiler for the Oberon-07 language, targeting the JVM: https://github.com/lboasso/oberonc https://github.com/lboasso/oberonc Initially the project was written in Java, after enough features were working the bootstrap phase could start.
- pavelbr 8y agoIf I wanted to actually run this compiler, how would I build it?
- z29LiTp5qUC30n 8y agogit clone 'https://git.savannah.gnu.org/git/stage0.git' https://git.savannah.gnu.org/git/stage0.git' make test That should do the full bootstrap in about 1 minute
- deleted 8y ago[deleted]
- userbinator 8y agoI don't recognise the architecture despite it saying x86, but it reminds me of ARM. Recursive descent seems to be the go-to parsing technique for compilers both big and small now. I like how all the repetitive functions for each level have been refactored into a "general_recursion" function, but if you want to make it even simpler and yet more extendable, table-driven precedence climbing would be ideal: http://www.engr.mun.ca/~theo/Misc/exp_parsing.htm#climbing http://www.engr.mun.ca/~theo/Misc/exp_parsing.htm#climbing
- chrisseaton 8y ago> Recursive descent seems to be the go-to parsing technique for compilers both big and small now ... if you want to make it even simpler and yet more extendable, table-driven precedence climbing would be ideal I think the fact that everyone just writes recursive descent parers tells us in practice that there isn't sufficient value in using more techniques like table-driven variants and they don't make anything practically simpler.
- dfox 8y agoThat depends on what you are parsing. For in-fix expressions shunting yard algorithm is significantly more clear (ie. you don't have to resort to left-right recursion tricks and invent labels like "product") and allows you to extend the set of supported operators by simply adding table entry (which you can do even while parsing and thus support user-defined operators)
- chrisseaton 8y ago> For in-fix expressions shunting yard algorithm is significantly more clear If it was significantly more clear, people would use it in practice! This makes me think it is not in fact significantly more clear. I did research work in parsers, and I work professionally in compilers now, and guess what when I need a parser for in-fix expressions I just write a recursive descent one manually, it's never an issue.
- 8y ago
- gmueckl 8y agoIt would be kind of cool to start small with entirely verifiable hardware for the first bootstrapping stages. I am phantasizing about a sort of ceremony in which the whole bootstrap process is done live in front of an audience starting with a discrete computer (using e.g. this board as a CPU https://monster6502.com https://monster6502.com), absolutely no electronic non-volatile memory and the first programs read into the computer from punch cards or punch tape. This would be used to create later stages for more powerful hardware and the end result (after maybe one or two hardware switches) is hopefully a minimal C compiler or similar that can be used to bootstrap production compilers like GCC. Ideally, this binary is shown to be completely identical to a binary built by a typical build process. Even if such a ceremony is ultimately not very useful, it could still be seen as a kind of artistic performance.
- Filligree 8y agoAs for 'why bootstrap'... Since Reflections on Trusting Trust has been linked already, I'm going to offer something else. Today's nightmare: https://www.teamten.com/lawrence/writings/coding-machines/ https://www.teamten.com/lawrence/writings/coding-machines/
- lathiat 8y agoI'm going to have nightmares for a decade after reading that...
- axaxs 8y agoThanks for this, I'd never read it before and it was very interesting. But by the end I had more questions than answers. In this situation, my first thought would never be 'i need to write my own compiler', vs say, trying any combo of the handful of c compilers on other boxes. Also it's not clear to me about the ending. Was this a real story? I couldn't tell if the 'letter' was meant to be a dystopian what-if, real, or anything in between.
- Filligree 8y agoThe entire page is fiction, presumably. Hopefully. That being said, it's not professional fiction. It's possibly the first work of the writer, and works because of talent (probably) and a good premise (definitely), but it's sorely lacking in polish and editing. I wouldn't look too closely at the cracks. :)
- jtjj222 8y agoNot related, but Coding Machines reminded me of the relatively unknown movie Pontypool, which has a similar premise (I won't spoil it).
- Filligree 8y agoI looked it up on Wikipedia, sorry. :) It does look like an interesting movie. I think I'll take a look.
- newnewpdro 8y agoI'd appreciate a bare minimum C compiler capable of building gcc written in either assembly or x86 machine code that can be trivially audited in as-executed form.
- sam0x17 8y agoThis is super impressive. What might be some compelling technical reasons to use this over existing C implementations? I just don't know so I thought I'd ask.
- pome 8y agoIf someone is interesting about how to bootstrapping, this[0][1] tutorial is just awesome, tiny compiler from nothing (raw machine codes and hex!) to the self-hosting! [0] https://web.archive.org/web/20120712111627/http://www.rano.org/bcompiler.html https://web.archive.org/web/20120712111627/http://www.rano.o... [1] https://github.com/certik/bcompiler https://github.com/certik/bcompiler (Fork in GitHub) Also worth check - https://www.t3x.org/t3x/book.html https://www.t3x.org/t3x/book.html :-)
- quickoats 8y agofirst thing i notice is how much commenting generally accompanies assembly source listings