17 ms·
This is where go’s insistence on reinventing the wheel feels terribly misplaced. Every major debug format has a way to associate code locations with line number
by nneonneo 7y ago
This is where go’s insistence on reinventing the wheel feels terribly misplaced. Every major debug format has a way to associate code locations with line numbers. Every major debug format also has a way to separate the debug data from the main executable (.dSYM, .dbg, .pdb). In other words, the problem that the massive pclntab table (over 25% of a stripped binary!) is trying to solve is already a well-trodden and solved problem. But go, being go, insists on doing things their own way. The same holds for their wacky calling convention (everything on the stack even when register calling convention is the platform default) and their zero-reliance on libc (to the point of rolling their own syscall code and inducing weird breakage).
Sure, the existing solutions might not be perfect, but reinventing the wheel gets tiresome after a while. Contrast this with Rust, which has made an overt effort to fit into existing tooling: symbols are mangled using the C++ mangler so that gdb and friends understand them, rust outputs nice normal DWARF stuff on Linux so gdb debugging just works, Rust uses platform calling convention as much as possible, etc. It means that a wealth of existing tooling just works.
- krasin 7y agoI agree with some of your points; but zero-reliance on libc is the reason why it's so easy to use Go in containers; and Docker is one of the primary reasons why Go is popular. It's what they have got right.
- Skunkleton 7y agoYou could statically link in libc and get the same effect.
- IshKebab 7y agoSays someone who has never actually tried to do that. You can statically link with Musl. But you can't really statically link with Glibc.
- josteink 7y agoTechnically speaking OP only suggested statically linking (a) libc, not glibc specifically, and musl is a libc.
- Skunkleton 7y agoNot sure what’s up with the combative tone? FWIW, I work as an operating systems developer. We sell several products incorporating a few in house libc variants. They are most certainly statically linked.
- saagarjha 7y agoYou can, it just might do some strange things when dealing with iconv.
- Spivak 7y agoStatically linking libc is it’s own minefield. It can and is done but even if you statically link everything else you should almost always dynamically link against your platform’s libc.
- paulddraper 7y agoStatically linking libc is harder than dynamically linking it, but certainly easier than rewriting it.
- yencabulator 7y agoGood thing we don't want to use much of libc functionality from Go, so nobody needs to reimplement all of it. It's not like people would be begging to call strcpy. All that's needed is the syscalls.
- saagarjha 7y agoWhy? I see no reason you couldn’t on platforms with a stable syscall ABI, other than the standard reasons against static linking.
- thu2111 7y agoIt depends on the libc. glibc doesn't really "want" to be statically linked. It can be done sometimes, depending on how it's used, the phase of the moon and so on, but breaks from time to time until it's repaired. Some of the issues are fundamental. For instance the dynamic linker is a part of the C libraries. If you statically link libc and then dynamically load another shared library, you can end up with two copies of the C libraries loaded at once. The C library expects various external files to be found on disk, in a format that isn't totally forwards/backwards compatible. That's reasonable when things are dynamically linked because the linker and C library abstract the developer from format changes. If you statically link you should really statically link the data files too, but the C toolchain has no provision for that.
- 7y ago
- labawi 7y agoYou'd still have to get a static libc (not usually preinstalled), possibly compile it for a different os/architecture..
- p_l 7y agoExcept glibc does not really support static linking of you want network support.
- pjmlp 7y agoglibc is just one possible ISO C implementation with lots of extra stuff on top.
- stqism 7y agoAn issue here is that statically linking against glibc is generally regarded as a poor idea and I’ve seen a couple non trivial programs that refuse to run with other libc’s.
- deleted 7y ago[deleted]
- rapsey 7y agoIf you want static linking use musl..
- y4mi 7y agoYou'd supposedly sacrifice performance though. At least that's the common complaint for Alpine docker images... It's based on musl and halve of the community always complains about serious performance degradation
- pjmlp 7y agoExcept that Docker was originally written in Java by the former team that actually started the project, and nowadays contains modules written in OCaml taken from the MirageOS project, for the macOS and Windows variants of Docker. So how much they got right regarding Docker's success and Go is a bit debatable.
- kjeetgill 7y agoI think the parent was reffering to using Go in Docker containers, not using Go for implementing Docker itself. That said, I agree that Docker was the first major project written in Go many people were exposed to and probably had some influence.
- kick 7y agoI'm just wondering, how is your line about the code taken from the MirageOS project relevant? Nobody uses the Windows and macOS variants of Docker in production.
- ffk 7y agoDocker was not written in Java. It was shell scripts, python, then go. Dotcloud was primarily a python shop. Perhaps you are thinking of a different project?
- michaelcampbell 7y ago> Docker is one of the primary reasons why Go is popular. Based on what data?
- twic 7y agoI've deployed all sorts of things which dynamically link libc in containers. This just isn't an issue in practice.
- Jasper_ 7y agoThe calling convention is a serious wtf. They're relying on store-load forwarding to make the stack as free as a register, but that's iffy at best and changes heavily between microarchitectures.
- jjtheblunt 7y agoI'd assert the calling convention is strange by design: there is the underlying reality that, to support actual closures and lambdas, as Go does, in the Lisp sense, not the fake Java sense, one can't use the C calling conventions. In particular, it's not true that a called function can expect to find bindings for its variables on a call stack, because of the upward funargs issue: some bound variables for a called function in the presense of true lambdas and thus closures will necessarily NOT be found on the C call stack, because of the dissociation of scope with liveness in the presence of lambda (anonymous functions).
- kjeetgill 7y agoI'm not familiar with the issue: what makes Java's lambdas/closures fake? Is it that bound variables need to be effectively final?
- hinkley 7y agoI don’t know if they’ve done anything new, but as originally implemented, they were inner classes.
- erik_seaberg 7y agoThe inner class gets copies of the variables, so imperative code that wants to reassign them isn't allowed because it probably won't do what you expected. The goal is not to GC stack frames. But I'm not sure why the didn't create an inner class to hold the closed-over variables in non-final fields (moving them from the stack to the heap) for both the function and all closures it creates. (Obligatory "doctor, it hurts when I use mutable state!")
- antics 7y agoI am not a fan of Go, and I also wish these things were true (and more[1], actually), but I find it hard to agree that its priorities are "terribly misplaced." Inside the context of Go's goals (e.g., "compile fast") and non-goals (e.g., "make it easy to attach debuggers to apps replicated a zillion times in Borg") these trade-offs make a lot of sense to me. Like: Go rewrote their linker, I think, 3 times, to increase the speed. If step 1 was to wade through the LLVM backend, I am not sure this would have happened. Am I missing something? I love Rust, but Go is focused on a handful of very specific use cases. Rust is not. I don't know that I can fault Go for choosing implementation details that directly enable those use cases. [1]: http://dtrace.org/blogs/wesolows/2014/12/29/golang-is-trash/ http://dtrace.org/blogs/wesolows/2014/12/29/golang-is-trash/
- mmastrac 7y agoI totally agree with the above - was never able to click with Go _but_ I totally understand how reinventing the wheel has worked well for them. The days when the Go project fired up were different than the days when Rust started. Rust made different tradeoffs by relying on LLVM and it has advantages (free optimizations!) and disadvantages of their own.
- ominous_prime 7y agoThe first releases of each were only 8 months apart, far from “different days”. The projects simply have different goals.
- temac 7y agoWell go also uses its own assembler, on top of that a kind of modified garbage version of real ones. You can only justify so many reinventions of the wheel, yet they redid everything.
- rm445 7y agoDid they actually redo everything, or does it just look that way from starting from the Plan9 toolchain? Which could also be said to be re-doing everything, but from a much earlier starting point. IIRC Go started out shipping with a port of the Plan9 C compiler and toolchain - it was bootstrapped by building the C compiler with your system C compiler, then building the Go compiler. Which, until re-written in Go circa-2013, was in Plan-9 style C. It all looks deeply idiosyncratic but it was a toolchain the initial implementors were highly familiar with.
- fooyc 7y ago> to the point of rolling their own syscall code It makes the Go concurrency mechanism possible, this is not just a kind of whim. Most importantly, this allows the scheduler to hook on syscalls in order to schedule an other routine. But this also allows to control what happens during a syscall, since the libc tends to do more than just calling the kernel in its syscall wrappers, which might not be thread safe or might not play well with stack manipulations. This has never been a problem in my experience.
- jcranmer 7y agoThe problem is that there is exactly one OS that maintains the system call ABI as a stable API: Linux. On other systems, trying to invoke the system calls manually and bypassing the C wrapper opens you up into undefined behavior, and this was particularly problematic on OS X, which occasionally made assumptions about the userspace calling code that weren't true for the Go wrapper shell, since it wasn't the expected system wrapper library.
- eikenberry 7y agoThe problem is that that one OS is right. Systems calls form an API and it needs to be stable and managed. We (developers) have been working on this issue for years and have at least attempted solutions (eg. semantic versioning) where most OS developers feel free to break them on a whim. It is a terrible practice that forces others to spend their time working around.
- ChrisSD 7y agoI would note that system calls only form an external API if the developer says they are an external API. Which it is for Linux but for other OSes the external API is a C library, kernel32, etc. But right and wrong aside, there's the practical matter of reality. You can't simply pretend everything works how you want them to. At the end of the day you have to deal with how they actually work.
- saagarjha 7y agoThe Linux model is not the "right" one, it's a choice that they've made. Just like static linking isn't the "right" choice either, it's an option with its own drawbacks. Other OSes provide an approved, API stable layer to access the OS; it's just not the syscall layer.
- steveklabnik 7y agoSmall note, we don't use the C++ mangler (https://github.com/rust-lang/rfcs/pull/2603 https://github.com/rust-lang/rfcs/pull/2603), and did the upstream work in GDB to get it to understand things. (There's also more work to do: https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3AA-debuginfo https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Ais... ) That being said, yes, we see integration into the parent platform as being an important design constraint for Rust. I think Go made reasonable choices for what they're trying to do, though. It's all tradeoffs.
- kibwen 7y agoIndeed, though it's worth mentioning that the Rust mangling scheme is based on that of the Itanium C++ ABI.
- atombender 7y agoA lot of the insularity and weirdness comes from the Plan 9 heritage. Go's authors (Rob Pike, Ken Thompson, and Russ Cox) cannibalized/ported a bunch of their own Plan 9 stuff during initial development. For example, I believe the original compiler was basically a rewrite of the Inferno C compiler. This is a large part of why Go is not based on GCC or LLVM, why it has its own linker, its own assembly language, its own syscall interface, its own debug format, its own runtime (forgoing libc), and so on. Clearly Go's designers were more than a little contrarian in their way of doing things, but that's not the whole answer. Being able to repurpose existing code is an efficiency multiplier during the bootstrapping phase. But when bootstrapping is done, you have to consider the ROI of going back and redoing some things or keep a design that works pretty well. The Go team is undoubtedly aware of some of these issues, but probably don't consider them to be a priority. In some cases the tools are a benefit. Go's compiler and linker are extremely fast, which I appreciate as a developer. A possible compromise would be to offer a slower build pipeline for production builds, which made use of LLVM and its many man-years of code optimizations.
- eikenberry 7y agoPersonally I more wish Rust would take this approach. Rust desperately needs a fast, developer oriented compiler. The slow compile times is potentially Rusts biggest flaw, to the point where I find it keeps me off the language for anything non-trivial. Even better might be a Rust interpreter, so you'd get REPL and fast development cycles.
- mixedCase 7y agoIsn't Cranelift going to be usable for that?
- pcwalton 7y agoThis is why Cranelift is being worked on. There is also a Rust interpreter, miri. I think starting with LLVM was the right decision (and one that was I was primarily responsible for). Rust would lose most of its benefits if it didn't produce code with performance on par with C++. LLVM is not the fastest compiler in the world (though it's not like it's horribly slow either), but its optimization pipeline is unmatched. I don't see replicating LLVM's code quality as feasible without a large team and a decade of work. Middling code gen performance is an acceptable price to pay until we get Cranelift; the alternative, developing our own backend, would mean not being able to deploy Rust code at all in many scenarios.
- userbinator 7y agoLet's not forget their attempt at inventing yet another Asm syntax for x86, when there is already the horrible GNU/AT&T as well as the official syntax of the CPU documentation.
- kjksf 7y agoGo's assembler syntax is inherited from Plan 9 project, which started in late 1980 and first released in 1992. For context, gcc was first released in 1987 i.e. about the same time that Plan 9 started. Go authors didn't attempt to re-invent asm syntax. They re-used the work they did over 30 years ago. And at the time Plan 9 happened it was hardly re-inventing anything either. It was still the time of invention. References: * https://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs https://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs * https://en.wikipedia.org/wiki/GNU_Compiler_Collection https://en.wikipedia.org/wiki/GNU_Compiler_Collection
- userbinator 7y agoAnd at the time Plan 9 happened it was hardly re-inventing anything either. Intel's Asm syntax was defined in 1978 with the release of the 8086, and the 32-bit superset in 1985 with the 386. CP/M, DOS, and later Windows assemblers all used the official syntax.
- p_l 7y agoPlan9 assembler syntax didn't start out on x86, and is kept the same across all platforms as much as possible.
- int_19h 7y agoThe question remains, why not reuse the work that somebody else did even earlier, and that has a lot more adoption already?
- CamperBob2 7y agoI installed golang the other day to check it out for the first time. For whatever reason, I chose to input the 'Hello world' program from golang.org by typing it in manually. As with most C/C++ code I would typically write, I put the brackets on their own lines. Welp, so much for Go.
- michaelcampbell 7y agoLike all opinionated formatters, you adjust to it, or you don't. I don't hate gofmt, other than tabs. Sweet jesus.
- CamperBob2 7y agoIt's not about the brace format. It's about the mentality that went into the underlying design decision. The more you look into why they did it that way [1], the more dysfunctional their decisionmaking process sounds. Basically, while (e.g.) Python's mandated formatting style arose from Guido van Rossum's philosophy of best programming practice, Go's mandated style arose from the fact that it was easier to implement from the point of view of compiler authors who had evidently never used a lexer before, much less written one. [1] https://stackoverflow.com/questions/17153838/why-does-golang-enforce-curly-bracket-to-not-be-on-the-next-line https://stackoverflow.com/questions/17153838/why-does-golang...
- omegabravo 7y ago> the more dysfunctional their decisionmaking process sounds. can you expand, because I can't see their dysfunctional decision making from the link you provided.
- CamperBob2 7y agoThe second answer? "Go uses brace brackets for statement grouping, a syntax familiar to programmers who have worked with any language in the C family. Semicolons, however, are for parsers, not for people, and we wanted to eliminate them as much as possible. To achieve this goal, Go borrows a trick from BCPL: the semicolons that separate statements are in the formal grammar but are injected automatically, without lookahead, by the lexer at the end of any line that could be the end of a statement. This works very well in practice but has the effect that it forces a brace style." Nothing about that is OK. Nothing about it makes sense.
- bouncycastle 7y agoI think by doing it everything their own way, they are not shackled to all of these dependencies - especially to some rusty old C++ compiler. That way, among other benefits, they get some very nice compiler speeds.
- mattbillenstein 7y agoReinventing the wheel is sometimes a feature - using other people's stuff, you gain their features, but you inherit their bugs, their release timelines, whatever overhead they baked in which they thought was okay, etc. You lose the ability to customize and optimize because it's no longer your code... It's all just tradeoffs in the end - I think golang is finding some success because they didn't make the same tradeoffs everyone else did.