9 ms·
Rust GCC backend: Why and how
- MangoToupe 10mo agoI find it shocking that 20 years after LLVM was created, gcc still hasn't moved towards modularization of codegen.
- ayende 10mo agoIsn't that very much intentional on the part of GCC?
- demurgos 10mo agoIt is intentional to avoid non-free projects from building on top of gcc components. I am not familiar enough with gcc to know how it impacts out-of-tree free projects or internal development. The decision was taken a long time ago, it may be worth revisiting it.
- janc_ 10mo agoOver the years several frontends for languages that used to be out-of-tree for years have been integrated. So both working in-tree & outside are definitely possible.
- colejohnson66 10mo agoSomewhat. Stallman claims to have tried to make it modular,[0] but also that he wants to avoid "misuse of [the] front ends".[1] The idea is that you should link the front and back ends, to prevent out-of-process GPL runarounds. But because of that, the mingling of the front and back ends ended up winning out over attempts to stay modular. [0]: https://lists.gnu.org/archive/html/emacs-devel/2015-02/msg00693.html https://lists.gnu.org/archive/html/emacs-devel/2015-02/msg00... [1]: https://lists.gnu.org/archive/html/emacs-devel/2015-01/msg00015.html https://lists.gnu.org/archive/html/emacs-devel/2015-01/msg00...
- phkahler 10mo ago>> The idea is that you should link the front and back ends, to prevent out-of-process GPL runarounds. Valid points, but also the reason people wanting to create a more modular compiler created LLVM under a different license - the ultimate GPL runaround. OTOH now we have two big and useful compilers!
- colechristensen 10mo agoGood lord Stallman is such a zealot and hypocrite. It's not open vs. closed it's mine vs. yours and he's openly declaring that he's nerfing software in order to prevent people from using it in a way he doesn't like. And refusing to talk about it in public because normal people hate that shit "misunderstanding" him. --- From the post: I let this drop back in March -- please forgive me. > Maybe that's the issue for GCC, but for Emacs the issue is to get detailed > info out of GCC, which is a different problem. My understanding is that > you're opposed to GCC providing this useful info because that info would > need to be complete enough to be usable as input to a proprietary > compiler backend. My hope is that we can work out a kind of "detailed output" that is enough for what Emacs wants, but not enough for misuse of GCC front ends. I don't want to discuss the details on the list, because I think that would mean 50 messages of misunderstanding and tangents for each message that makes progress. Instead, is there anyone here who would like to work on this in detail?
- bigfishrunning 10mo agoHe should just re-license GCC to close whatever perceived loophole, instead of actively making GCC more difficult to work with (for everyone!). RMS has done so much good, but he's so far from an ideal figure.
- tovej 10mo agoHow in the world would you relicense GCC
- colechristensen 10mo ago
- wahern 10mo agoNot anymore. Modularization is somewhat tangential, but for awhile Stallman did actively oppose rearchitecting GCC to better support non-free plugins and front-ends. But Stallman lost that battle years ago. AFAIU, the current state of GCC is the result of intentional technical choices (certain kinds of decoupling not as beneficial as people might think--Rust has often been stymied by lack of features in LLVM, i.e. defacto (semantic?) coupling), works in progress (decoupling ongoing), or lack of time or wherewithal to commit to certain major changes (decoupling too onerous).
- torginus 10mo agoPersonally, I think when you are making bad technical decisions in service of legal goals (making it harder to circumvent the GPL), that's a sure sign that you made a wrong turn somewhere.
- 1718627440 10mo agoWhy? When your goal is to have free software, having non-free software with better architecture won't suit you.
- bigstrat2003 10mo agoI would describe this more as "trying to prevent others from having non-free software if they wish to", which is a lot more questionable imo.
- LeFantome 10mo agoSome in the Free Software community do not believe that making it harder to collaborate will reduce the amount of software created. For them, you are going to get the software and the choice is just “free” or not. And they imagine that permissively license code bases get “taken” and so copyleft licenses result in more code for “the community”. I happen to believe that barriers to collaboration results in less software for everybody. I look at Clang and GCC and come away thinking that Clang is the better model because it results in more innovation and more software that I can enjoy. Others wonder why I am so naive and say that collaborating on Clang is only for corporate shills and apologists. You can have whatever opinion you want. I do not care about the politics. I just want more Open Source software. I mean, so do the others guys I imagine but they don’t always seem to fact check their theories. We disagree about which model results in more software I can use.
- pjmlp 10mo agoLLVM wasn't the first modularization of codegen, see Amsterdam Compiler Kit for prior art, among others. GCC approach is on purpose, plus even if they wanted to change, who would take the effort to make existing C, C++, Objective-C, Objective-C++, Fortran, Modula-2, Algol 68, Ada, D, and Go frontends adopt the new architecture? Even clang with all the LLVM modularization is going to take a couple of years to move from plain LLVM IR into MLIR dialect for C based languages, https://github.com/llvm/clangir https://github.com/llvm/clangir
- 1718627440 10mo agoIt is a political not a technical decision. Essentially the same like the Linux kernel not encouraging the use of out-of-tree kernel modules. https://gcc.gnu.org/legacy-ml/gcc/2000-01/msg00572.html https://gcc.gnu.org/legacy-ml/gcc/2000-01/msg00572.html
- surajrmal 10mo agoAnd it shows how silly the idea is. gcc still sees plenty of forks from vendors who don't upstream, and llvm sees a lot more commercial participation. Unfortunately the Linux kernel equivalent doesn't exist.
- hedgehog 10mo agoThere are several open BSDs.
- School-Cotton 10mo agoAFAIK there's no evidence to suggest that permissive vs. copyleft license is the reason for the relative lack of success of the BSDs vs. Linux.
- pjmlp 10mo agoPlayStation and macOS kind of show what happens with upstream. As did all the UNIXes that used to rule before companies started sponsoring Linux kernel development, and were quite happily taking BSD code into them, alongside UNIX System V original code.
- leoc 10mo agoIt's also nakedly hypocritical behaviour on Stallman's part. Hoping (whether in vain or not) that GCC being Too Big to Fork ( https://news.ycombinator.com/item?id=6810259 https://news.ycombinator.com/item?id=6810259 ) will keep people from having access to the AST interface really isn't substantially different from saying "why do you need source code, can't you just disassemble the binary hahaha".
- keyle 10mo agoIf the author reads this... I'd be very interested if the author could provide a post with a more in depth view of the passes, as suggested!
- petcat 10mo ago> Little side-note: If enough people are interested by this topic, I can write a (much) longer explanation of these passes. Yes, please!
- bfjjejskdjd 10mo ago[flagged]
- mastax 10mo ago> On that note: GCC doesn't provide a nice library to give access to its internals (unlike LLVM). So we have to use libgccjit which, unlike the "jit" ("just in time", meaning compiling sub-parts of the code on the fly, only when needed for performance reasons and often used in script languages like Javascript) part in its name implies, can be used as "aot" ("ahead of time", meaning you compile everything at once, allowing you to spend more time on optimization). Is libgccjit not “a nice library to give access to its internals?”
- saghm 10mo agoI could be wrong, but my surface level understanding is that it's more of a library version of the external API of GCC than one that gives access to the internals.
- deleted 10mo ago[deleted]
- LukeShu 10mo agolibgccjit is much higher level than what's documented in the "GCC Internals" manual.
- compiler-guy 10mo agoTo use an illustrative (but inevitably flawed) metaphor: Using libgccjit for this is a bit like networking two computers via the MIDI protocol. The MIDI protocol is pretty good for what it is designed for, and you can make it work for actual real networking, but the connections will be clunky, unergonomic, and will be missing useful features that you really want in a networking protocol.
- jacquesm 10mo agoOh come on, SLIP over MIDI is tried and true.
- compiler-guy 10mo agoNote the word order here. Googling “slip over midi” gives a lot of fashion blogging about mini dresses and slips that one wears under them, so I’m not quite sure what you mean. But if you mean “midi over slip”, then that is the inverse case from what I am suggesting. Midi over slip (and slip could be any tcpip substrate, such as ethernet) has midi messages as the payload, carried via tcpip. I’m talking about using midi messages to carry tcpip payloads. You can absolutely do it, but it isn’t really what the protocol is designed for.
- 1718627440 10mo agoI don't necessary like the focus on Rust, but if it happens, then we need to have support in the free compiler!
- lionkor 10mo agoWhy not? Like what about the technology or ecosystem do you disagree with
- throwaway17_17 10mo agoNot parent, but I share the ambivalence (at best) or outright negativity (at worst) toward the focus on Rust. It is a question of preference on my part, I don’t like the language and I do not want to see it continue to propagate through the software I use and want to control/edit/customize. This is particularly true of having Rust become entrenched in the depths of the open-source software I use on my personal and work machines. For me, Rust is just another dependency to add to a system and it also pulls along another compiler and the accompanying LLVM. I’m not going to learn a language that I disagree with strongly on multiple levels, so the less Rust in my open source the more control I retain over my software. So for me the less entrenched Rust remains the more ability I keep to work on the software I use. That said, if Rust is going to continue entrenching itself in the open source software that is widely in use, it should at least be able to be compiled with by the mainline GPL compiler used and utilized by the open source community. Permissive licenses are useful and appreciated in some context, but the GPL’d character of the Linux stack’s core is worth fighting to hold onto. It’s not Rust in open source I have a problem with, it is Rust being added to existing software that I use that I don’t want. A piece of software, open source, written in Rust is equivalent to proprietary software from my perspective. I’ll use it, but I will always prefer software I can control/edit/hack on as the key portions of my stack.
- lionkor 10mo ago> I disagree with strongly on multiple levels Fair enough, but what are those disagreements? I was fully in the camp of not liking it, just because it was shoved down every projects throat. I used it, it turns out its fantastic once you get used to the syntax, and it replaced almost all other languages for me. I just want to know if there are any actual pain points beyond syntax preference. Edit: I partially agree with the compiler argument, but it's open source, and one of the main reasons the language is so fantastic IS the compiler, so I can stomach installing rustc and cargo.
- grokx 10mo agoWhen I studied compiler theory, a large part of the compilation involved a lexical analyser (e.g. `flex`) and a syntax analyser (e.g. `bison`), that would produce an internal representation of the input code (the AST), used to generate the compiled files. It seems that the terminology as evolved, as we speak more broadly of frontends and backends. So, I'm wondering if Bison and Flex (or equivalent tools) are still in use by the modern compilers? Or are they built directly in GCC, LLVM, ...?
- brooke2k 10mo agoNot sure about GCC, but in general there has been a big move away from using parser generators like flex/bison/ANTLR/etc, and towards using handwritten recursive descent parsers. Clang (which is the C/C++ frontend for LLVM) does this, and so does rustc.
- gpderetta 10mo agoI believe that GCC also moved to a handwritten parser, at least for c++, a couple of decades ago.
- afdbcreid 10mo agoI don't know a single mainstream language that uses parser generators. Python used to, and even they have moved. AFAIK the reason is solely error messages: the customization available with handwritten parsers is just way better for the user.
- muvlon 10mo agoI'll let you decide whether it counts as "mainstream", but the principal implementation of Nix has a very old school setup using bison and flex: https://github.com/NixOS/nix/blob/master/src/libexpr/parser.y https://github.com/NixOS/nix/blob/master/src/libexpr/parser.... https://github.com/NixOS/nix/blob/master/src/libexpr/lexer.l https://github.com/NixOS/nix/blob/master/src/libexpr/lexer.l
- 10mo ago
- notepad0x90 10mo agoI would just like to encourage all Rust devs to distribute binaries. No matter what compiler you choose, or what Rust version, users shouldn't have to build from source. I mostly see this with small projects to be fair.
- MerrimanInd 10mo agoAnother reason to have a second compiler is for safety-critical applications. In the assessment of safety-critical tools if something like a compiler can have a second redundant version then each one of them can be certified to a lower criticality level since they'll crosscheck each other. When a tool is single-sourced the level of qualification goes up quite significantly.
- steveklabnik 10mo agorustc (via Ferrocene) is already being qualified, and form what I hear it’s been fairly easy to do so, for various reasons.
- MerrimanInd 10mo agoYeah it is and that's a great effort, I've worked with that team on various things. But the industry is still itching for a second compiler with no crossover (can't just be another LLVM frontend or rustc fork) for those certification reasons. Not that people want to replace rustc! It's just a cert requirement.
- steveklabnik 10mo agoYeah certainly wouldn't hurt :)
- kunley 10mo agoA perhaps naive question: does it have a chance to be faster than LLVM backend?