7 ms·
I still just don't understand why they insist on building their own toolchain. It just doesn't make sense to me. When you set out to build a programming langua
by arcticbull 12y ago
I still just don't understand why they insist on building their own toolchain. It just doesn't make sense to me.
When you set out to build a programming language, what is your objective? To create a sweet new optimizer? To create a sweet new assembler? A sweet new intermediate representation? AST? Of course not. You set out to change the way programmers tell computers what to do.
So why do this insist on duplicating:
(1) An intermediate representation.
(2) An optimizer.
(3) An assembler.
(4) A linker.
And they didn't innovate in any of those areas. All those problems were solved with LLVM (and to some more difficult to interact with extent GCC). So why solve them again?
It's like saying you want to build a new car to get from SF to LA and starting by building your own roads. Why would you not focus on what you bring to the table: A cool new [compiler] front-end language. Leave turning that into bits to someone who brings innovation to that space.
This is more of a genuine question.
- dsymonds 12y agoWhat's your counter-proposal? If you're building a new language, you need a new AST. You can't represent Go source code in a C++ AST. There are alternate compilers for Go, in the form of gccgo and llgo. But those are both very slow to build (compared to the Go tree that takes ~30s to build the compiler, linker, assembler and standard library). And the "gc" Go compiler runs a lot faster than gccgo (though it doesn't produce code that's as good), and compilation speed is a big part of Go's value proposition.
- arcticbull 12y agoI would never set out to build a language I wanted people to use and not build it as a front-end for LLVM. I don't want to write an optimizer or assembler. I don't doubt for one second that llgo takes a longer time to compile. And in exchange for slower compile times you benefit from many PHDs worth of optimizations in LLVM. And every single target architecture they support. It's easy to build something faster when it does less. I'll admit there's no blanket right answer to that tradeoff.
- dsymonds 12y agoYes, that's why there's both gc and gccgo (llgo came later). Apart from the rigour of having two independent compilers, they are seeking different tradeoffs. gc is very interested in running fast, and gccgo benefits from decades of work that have been put into gcc's various optimisations. Does that answer your original statement that you didn't understand why we build our own toolchain?
- bsdetector 12y agoWell I still don't understand. Russ says it was for segmented stacks, but doesn't explain why those were necessary. You say it was for compile speed, yet gcc and llvm can crank out millions of lines a code a second at similar optimization levels as the Go compiler. Neither of these are convincing explanations.
- pjmlp 12y agoThen you will be stuck with the C view of world of what a linker is supposed to do. Just look at Modula-2 and Object Pascal toolchains as examples of compile speeds and incremental compilation features that could run circles around contemporanean C compilers. Or the lack of proper module system, which requires linker help.
- chimeracoder 12y ago> There are alternate compilers for Go, in the form of gccgo and llgo. But those are both very slow to build (compared to the Go tree that takes ~30s to build the compiler, linker, assembler and standard library). For any non-Gophers reading this: I write Go as my primary language, and have for the past two and a half years. I just timed the respective compilation speeds on a handful of my larger projects using both gc and gccgo (and tested on a separate computer as well just for kicks). gccgo was marginally slower, though not enough to be appreciable. In the case of two projects, gccgo was actually slightly faster. The Go compiler/linker/assembler/stdlib are probably larger and more complex than the most complex project on my local machine at the moment, but I think my projects are a reasonable barometer of what a typical Go programmer might expect to work with (as opposed to someone working on the Go language itself). The more pressing issue as far as I'm concerned is that gccgo is on a different release schedule than gc (because it ships with the rest of the gcc collection). That's not to say it's not worth optimizing either compiler further when it comes to compilation speed, but it's important for people considering the two compilers to understand the sense of scale we're talking about - literally less than a second for most of my projects. Literally, the time it takes you to type 'go build' is probably more significant.
- dsymonds 12y agoThanks. That's good data. I haven't seen any measurements for a few years. It's good to see that gccgo has caught up. Which version of gc did you test? Yes, the release schedule is another important reason for building our own toolchain. Being in control of one's destiny is often underrated.
- chimeracoder 12y agoOn this machine, gc 1.4.1 vs. gcc 4.9.2 (with no extra flags). My other machine has gc's tip, but runs Wheezy so it's probably an older version of gcc... it wasn't much different either way. I would barely have noticed it if I hadn't been timing it.
- bdarnell 12y agoThe Go team does want to innovate on the toolchain. A key factor in the design of Go is the belief that once a language is "good enough", developers are better served by a superior toolchain (and specifically faster compilation) than by a fancier language. They want to own the toolchain so they can optimize it for Go and make their own tradeoffs about speed versus features.
- evmar 12y agoI read somewhere (but I can't think of the keywords to find it now) that they found the greater flexibility in owning their toolchain was worth the cost. For example they changed their data layout for GC purposes and changed the segmented stack approach over the course of their development and had they been tied to LLVM or gcc they'd have spent much of their time fighting against those implementations, or politicing to convince the maintainers to add additional complexity to their systems for an unproven langauge. (My example is weak because I am trying to retell their reasons and my recollection is vague.) I think they still haven't succeeded in bringing gcc up to par with their current approach.
- seryoiupfurds 12y agoAre you thinking of this comment? https://news.ycombinator.com/item?id=8817990 https://news.ycombinator.com/item?id=8817990
- evmar 12y agoThat's the one, thanks!
- pcwalton 12y agoLLVM supports precise GC now via the late safepoint placement infrastructure [1]. This infrastructure should be sufficient to support both the copying stacks and a precise GC. This is a recent addition and did not exist at the time Go was created, however. [1]: http://llvm.org/docs/Statepoints.html http://llvm.org/docs/Statepoints.html
- mseepgood 12y ago> I still just don't understand why they insist on building their own toolchain. It just doesn't make sense to me. To quote rsc from https://news.ycombinator.com/item?id=8817990 https://news.ycombinator.com/item?id=8817990: "It's a small toolchain that we can keep in our heads and make arbitrary changes to, quickly and easily. Honestly, if we'd built on GCC or LLVM, we'd be moving so slowly I'd probably have left the project years ago." "For example, no standard ABIs and toolchains supported segmented stacks; we had to build that, so it was going to be incompatible from day one. If step one had been "learn the GCC or LLVM toolchains well enough to add segmented stacks", I'm not sure we'd have gotten to step two."
- bsdetector 12y agoWhich is of course no answer at all. Their own explanation for wasting hundreds of thousands of man-hours on a "quirky and flawed" separate compiler, linker, assembler, runtime, and tools is because they absolutely needed an implementation detail that is completely invisible to programs and which they are now replacing because it wasn't a good idea in the first place (segmented stacks). And it's worth writing out a 1000 word rationalization that doesn't bother even mention the reason that implementation was necessary in the first place, to better run on 32-bit machines. In 2010. Or they say that they had to reinvent the entire wheel, axle, cart, and horse so that five years later they could start working on a decent garbage collector. Never mind that five years later other people did the 'too hard and too slow' work on LLVM that a decent garbage collector needs. What foresight, that. That's not sense, that's people rationalizing away wasting years of their time doing something foolish and unnecessary.
- crawshaw 12y agoThe replacement to segmented stacks is copying stacks, which as far as my knowledge of LLVM takes me, would be very difficult to add. You need a stack map of pointers to successfully move pointed-to objects on the stack from the old region to the new. There is a great deal of work going on in LLVM on this issue for precise GC of other languages, and (from the outside) it looks like more hours have been spent on it than on the entire Go toolchain. As Go developers don't have the resources or expertise to make such wide-ranging changes to LLVM, it would have blocked Go development. GCC is similar. Those working on gccgo are trying to work out how to add precise GC and copying stacks. It is much more complex than it was on the gc toolchain. There is great value in having a simple toolchain that is completely understood by the developers working on it. In fact, that very idea, that code you depend on should be readable and widely understandable, is one of the goals of Go. Applying the goal to the toolchain is a case of eating our own ideological dogfood.
- wbond 12y agoI was impressed by the toolchain when I first peaked at Go because it was dead simple to get up and running on any platform, especially Windows. For gcc you have to deal with MinGW. Isn't LLVM just now getting to the point where it can build native Windows applications? This is one area where I hope Rust makes progress. MinGW/Msys2 is just kind of gross stuff to deal with.
- RayDonnelly 12y agoCare to explain what's gross about MSYS2/MinGW-w64? I'm genuinely interested in making it less gross.
- infogulch 12y agoWant to make it less gross? Make it completely go away. Installation of Go or Python is just like any other Windows install. You download an installer.exe or .msi, run it, and you're done. Things compile or run immediately, and you don't have to start using a "special" terminal just for it to work. My experience with MinGW is very different. Especially for dependent languages. "Step 1: Install MinGW" what does that even MEAN?: "Ok, I ran this installer, and it brought up the MinGW Installation Manager. Is it done? What am I supposed to do here? Which one do I choose? How do you even select a package? What even ARE packages? OK, so I select something then go the Package menu and select Mark for Installation. It's not doing anything. Is it done now? Close window. Nope that didn't work. Open it back up. Oh, so after marking a package I have to go to the Installation menu and choose Apply Changes. ..." This actually happened to someone I was trying to help over the phone. Heaven forbid they get lost in All Packages and get confused by the dozens of packages each with half a dozen versions and each with three different, non-descriptive "classes". Installation needs to be braindead simple. During installation it should show a list of extra languages that can be installed, where you can't uncheck 'base' (with an "Advanced Options" button in the corner that opens up the standard installation manager instead). It should set up any environment variables, including PATH, (and including restarting explorer to refresh the env) and it shouldn't require the use of any terminal other than cmd.exe (despite it being terrible). If you're installing something else entirely that depends on MinGW, their installer should be able to bundle the MinGW installer, and it should install without having to make any choices. It should detect if MinGW is already installed and install packages there instead, still completely automated. Make it go away.