10 ms·
Go at Google: Language Design in the Service of Software Engineering
- onan_barbarian 14y agoFrom the article: "In 1984, a compilation of ps.c, the source to the Unix ps command, was observed to #include <sys/stat.h> 37 times by the time all the preprocessing had been done. Even though the contents are discarded 36 times while doing so, most C implementations would open the file, read it, and scan it all 37 times. Without great cleverness, in fact, that behavior is required by the potentially complex macro semantics of the C preprocessor." "Great cleverness" apparently means "the ability to detect that a file has a standard 'include once' pattern and only process it once". It hardly seems like rocket science.
- koenigdavidmj 14y agoThere's plenty of magic that can be done in the C preprocessor that makes it sometimes possible to #include a file twice with different effects than #include-ing it once. #pragma once and Objective-C's #import declaration both do that, though.
- TallGuyShort 14y agoI can see a reason why the potentially complex macro semantics of the C preprocessor would require this behavior, and why you would certainly need to re-evaluate certain macros you had already processed whenever it was included. Doing that without impacting any of the preprocessor's functionality? Great cleverness - for really no reason than faster compiles - which isn't that big of a deal to most people.
- drivebyacct2 14y agoAnd if you're looking for rocket science you clearly missed a section.
- JulianMorrison 14y agoNo, that's called "dirty nonstandard corner cutting that will cause you to curse up a storm at $VENDOR who wrote their compiler to be fast but it breaks on $POPULAR_LIB which gets cute with the preprocessor, and yes they shouldn't do that but are you going to rewrite it?"
- cjensen 14y agoThat's incorrect. Optimizing the guard macros works in all cases as long as you use the standard pattern and the C Preprocessor memorizes the name of the guard macro.
- deleted 14y ago[deleted]
- deleted 14y ago[deleted]
- jlgreco 14y agoHow does this happen then? "The construction of a single C++ binary at Google can open and read hundreds of individual header files tens of thousands of times."
- cjensen 14y agoAn excellent question which deserved an answer in the paper.
- deleted 14y ago[deleted]
- georgemcbay 14y agoWalter Bright (who created D to solve some of the same problems Go aims to solve) wrote something a few years back which touches on this topic: http://www.drdobbs.com/cpp/c-compilation-speed/228701711 http://www.drdobbs.com/cpp/c-compilation-speed/228701711
- JulianMorrison 14y agoOff hand, I can imagine problems with: multiple guarded blocks at the top level, things outside guards, things that look like guards but aren't (#IFDEF conditional sections, perhaps), things that don't look like guards that are, things that affect which guards are switched on in a dynamic way (detect a feature and enable a lib)... Some of those could be non issues. There could be other issues I lack the experience to guess. It's no use saying "so don't do that" - not all code is under your control. Honestly if it were simple, it would be solved.
- paulhodge 14y ago1984 was a long time ago, I think most/all compilers ended up adding this optimization at some point. It is somewhat complex to do it correctly, see: http://gcc.gnu.org/onlinedocs/cppinternals/Guard-Macros.html http://gcc.gnu.org/onlinedocs/cppinternals/Guard-Macros.html
- haldean 14y agoSomeone can still throw in an #undef __YOUR_HEADER_H__ somewhere, and while that may be playing with fire, the spec says that the header file should be included again. There's so many corner cases (and ifdefs are used for so much more than guards) that you really do need to reevaluate it every time.
- pdw 14y agoIt really doesn't seem that complicated. When reading an include file the first time, check if everything is enclosed in a preprocessor conditional, one without else-branch. If so, remember the condition and re-evaluate it before opening the file again.
- haberman 14y agoThe dependency story becomes significantly less compelling if something like this proposed module system for C/C++ takes off: http://news.ycombinator.com/item?id=4832568 http://news.ycombinator.com/item?id=4832568
- bishop_mandible 14y ago"For the record, around 1984 Tom Cargill observed that the use of the C preprocessor for dependency management would be a long-term liability for C++ and should be addressed." And in 2012 Apple tries to address it: http://news.ycombinator.com/item?id=4832568 http://news.ycombinator.com/item?id=4832568
- myg204 14y agoGo is 3 years old. Also, golang addresses more than just this issue, the concurrency approach is interesting for example.
- cjensen 14y agoD is 11 years old and solves the exact same problems plus it uses a more traditional approach to objects. The big difference is Go has Google+Pike pushing it and might actually get widely adopted. Getting some improvement is not a bad thing.
- agentS 14y agoConsidering one of the goals of the language is simplicity, I don't see how D has solved this problem.
- pjmlp 14y agoBecause everyone else doesn't want to throw away the last 20 years of language design. Go's search for simplicity is seen as a solution looking for a problem in the language design communities.
- danieldk 14y agoGo's search for simplicity is seen as a solution looking for a problem in the language design communities. But most programmers are not in language design communities. There is a big potential for a 'safer C' or a 'faster Python'. Yes, that's a simplification, but also how people will see Go.
- cjensen 14y agoWhat gets my goat about this paper is the amount of misleading information in it. Re-parsing files multiple times is a solved problem using the standard header guard macro pattern combined with a slightly intelligent C Preprocessor. Yet quotes about the horror of header blow-up pervade this paper as if this is an exciting new problem that has been solved. It isn't; it was solved more than 20 years ago.
- enneff 14y agoYes, there are ways around it, and the paper discusses one way Google addressed it. But that's not the point. It's used as just one example to illustrate why C/C++ is not ideal for working at Google's scale.
- cjensen 14y agoI agree there's much more to the paper, and it's well worth the read. Bounds checking, avoiding pointer arithmetic, garbage collection, and concurrency all make sense at Google's scale. I just think the bashing was misguided in an attempt to better justify adding a subset of Ada to C
- enneff 14y agoI think it's misguided to try and characterize Go as the addition of a subset of Ada to C. ;-)
- akkartik 14y agoYes, the guard macro pattern works. But it's something programmers manage, so it's a band-aid rather than a solution. If you change a header you need to recompile all files that include it, even if the include was conditional. Now your build system starts to look inside files rather than just check timestamps. Similar tool bloat happens over and over again in large codebases. I'm not convinced Go has the right solution. But it's disingenuous to claim that it's competing with the C preprocessor. It's leagues better. You should enumerate all the information in the paper you consider to be misleading.
- paupino_masano 14y agoI found this article really interesting, though I would like to see more information on the design decisions for the syntax (i.e. types/return types come after). They did say it was easier to parse from a human and computer perspective, however personally (being used to C based languages I guess) I find it a little more difficult to parse. I guess I'd just like to see some references to this rather than insinuation. Anyone have any?
- enneff 14y agohttp://blog.golang.org/2010/07/gos-declaration-syntax.html http://blog.golang.org/2010/07/gos-declaration-syntax.html
- paupino_masano 14y agoExcellent - thank you for that. I'm not sure I completely agree with their reasoning: the example only really got complicated when dealing with function pointers, however that could be simplified with a macro (or a delegate in other languages). I do agree that C can get complex to read, however I'm not sure switching the type/parameter option is the answer. But then again, I've probably become accustomed to this format. The real test will be using the language for a few weeks... I better start some small hobby projects in Go just to see if I end up liking it!
- drivebyacct2 14y agoint *i, j; is more than a good enough reason for me. Makes interviews interesting, though: "Why are you repeatedly writing your variables and function parameters incorrectly?"
- paupino_masano 14y agoInteresting example. I'm still a newbie to Go, so would that be written as: var i int* var j int My apologies if this seems like a simple question: I haven't played with the language enough to know :) [EDIT: Sorry, I didn't realize how to format code!]
- swdunlop 14y agoSlide 14 is particularly important for new Go programmers. Go's garbage collector does not have access to certain optimizations because of the requirement to support pointers to internal members of structures and arrays. The loss of these optimizations are compensated by nesting structures instead of just always using pointers to imitiate pass by reference semantics found in other GC'd languages, such as Java or Python. Using them correctly can reduce the number of objects to be collected and analyzed.
- cpeterso 14y agoHere is a recent Mozilla talk about the Firefox build system. The speaker, Gregory Szorc, provides LOTS of data about the performance of parallel and recursive makefiles and Mozilla's investigation of new build tools. https://air.mozilla.org/the-future-of-the-firefox-build-system/ https://air.mozilla.org/the-future-of-the-firefox-build-syst...
- rwj 14y agoI enjoyed this article. Pike is an advocate for the language, but he openly discusses the trade-offs that were addressed by the language. Too often, the downsides to a language/technology are ignored, leading to technically boring talks.
- millstone 14y agoI continue to be confused by how Go gets positioned. From the article: "Go is a programming language designed by Google to help solve Google's problems, and Google has big problems. The hardware is big and the software is big. There are many millions of lines of software, with servers mostly in C++ and lots of Java and Python....And of course, all this software runs on zillions of machines, which are treated as a modest number of independent, networked compute clusters" The author moved effortlessly from "Google's problems" to "servers" that are "networked compute clusters." That's quite a leap, because it's certainly not true that "all this software" runs in that manner. Android does not, Chrome does not, Chrome OS does not, the iOS YouTube app does not, Google Earth does not, etc. And there's plenty of not-big hardware that Google writes for, like laptops and phones. And the reason I bring this up is because Go was explicitly pitched as a systems programming language - see the PDF http://www.stanford.edu/class/ee380/Abstracts/100428-pike-stanford.pdf http://www.stanford.edu/class/ee380/Abstracts/100428-pike-st.... Programming tools, IDEs, web browsers, operating systems ("maybe") are among the tasks specifically called out as suitable for Go. But now this article tells us that Go was designed for "large server software". So the question is, what changed? Was Go found to be unsuitable for tasks like web browsers? If so, in what way?
- hvs 14y agoLanguages seem to find a "killer app" much like most technologies and then tend to be used primarily for that purpose. Could Go do all of those things? Sure, with enough time, resources, and libraries, it could. But Google has a problem now (in scale) that Go excels at and can provide value now, so they use it for that. That said, I would guess that most of Google's code is still C++, Python, and Java.
- millstone 14y agoThere's two senses of "scale:" large software, or software running with large data sets. The article uses the first notion: "at scale, for large programs with large numbers of dependencies, with large teams of programmers working on them." So Go was designed for large software systems, and maybe it's really good at those things. Or maybe not, because nobody's written one yet. The article says so: "We don't have enough experience yet, especially with big programs." So Go is unproven in this way. Speculatively, my guess is that it won't be able to "at scale" its current state. For example, Go errors out with unused variables, imports, or indeed for all warnings. Imagine trying to make a piece of software like WebKit or LLVM compile on all of the platforms it supports, with all of the compilers they support, without introducing a single warning (or extraneous header or unused variable) on any of them. It would be a maintenance nightmare. You need some flexibility - demanding perfect hygiene for every cell on an NxM matrix of supported platforms is infeasible. In the other sense of scale - large data sets - Go apparently works well enough to be involved in YouTube. That's cool and is a credit to it.
- bfung 14y agoI find the guard macros discussion in this post most amusing. Just use Go and there's no point in talking about guard macros! =P
- aschroder 14y agoI've been reading about Go a lot lately, starting with Effective Go, and now the tutorial book Learning Go. (and periodically referring to the FAQ when I'm stumped by design decisions) It's been enjoyable learning it, though I'm yet to write any code in anger with Go. I find some of the concepts around concurrency and OO design have really made me think about the way I write in other languages. One thing I cannot quite make sense of though is the treatment of errors/exceptions. This article says: "Exceptions make it too easy to ignore them rather than handle them, passing the buck up the call stack.." But the example given seems to me to do exactly that: if err != nil { return err } Worse still if you forget to do something with err, or don't handle a specific err !=nil situation, it seems easy for control to flow past the error handling and into code that expects no errors. Are there small, easily understood open source projects written in Go, that would provide tangible examples of the benefit of this error handling approach? The learning materials thus far have not really convinced me, maybe code in the wild would.
- enneff 14y agoI don't think you'll be convinced until you have worked on a Go project of your own. Even then it might take a little while to fully see the point. Go's error handling model is simpler than exception handling. There are fewer surprises. That's pretty much it.
- frou_dh 14y agoDo you know whether any calls to the standard library panic over the package boundary? Edit - Things like regexp.MustCompile aside.
- 4ad 14y agoNo panics over package boundaries. Breaking this is almost sacrilegious in Go.
- frou_dh 14y agoYet the standard library does just that? Only on package regexp or elsewhere too? I'm not trying to say I know better than the designers, just that the waters are muddied by supposed rules being broken off the bat.
- jebblue 14y agoWhat is with all these Go articles? I wish Go would Go the way if the DoDo bird. Java rocks client and server, web and think client.
- ChuckMcM 14y agoI wonder sometimes if Go will become to Google what TAL was to Tandem. A language uniquely suited to solve problems unique to the design space in which they operate. In Tandem's case it was non-stop computing, in Google's case its kilocluster distributed algorithms. I really appreciate watching it evolve.
- dualogy 14y agoDoesn't apply here, Go is largely as flexible, generic and versatile as C, minus a few restrictions and plus a few (awesome) additions. And the CGO interface lets you bind C libs as needed. I write OpenGL apps in Go. Surely this is in no way characteristic of Google's internal use-cases. Today's machines are multi-core and are not as bare-bones as to require manually carefully coded malloc/free calls. That's what's changed since C and that's what Go is (very well) designed for. Then a few syntactic goodies that don't imply performance side-effects or any black magic but that C didn't have: duck typing, struct composition, interfaces, all that.
- danieldk 14y agoGo is really too general for that to happen. From my (still limited) experience, Go is a safer C with garbage collection, interfaces and goroutines. What I like about Go is that the authors did not over-engineer the language and is pretty much as simple as C as a consequence. Obviously, things like the garbage collector and bounds checking make Go slower than C. On the other hand, although the Go spec does not mention the stack or heap, you still have pretty much control over memory layout. E.g. making a slice of structs will be backed by an array of (unboxed) structs, pretty much as you'd expect.
- burntsushi 14y agoI wrote a fully featured X window manager in pure Go [1] that I use full time. And it was pure joy compared to doing the same in C. [1] - https://github.com/BurntSushi/wingo https://github.com/BurntSushi/wingo
- cromwellian 14y agoI think the "return an error code" approach is just as unreadable as nested or multiple try/catch blocks when the number of failure states grows large. I think the "Maybe Monad" approach of functional languages and Scala, combined with pattern matching in the language tends to look cleaner, e.g. some_expression match { case Some(x) => do_something_with(x) case None => no result (e.g. null in C/Java/etc) case Error(x) => handle error x } One of the nice things about the monad approach is they can be combined together. Consider the following chain of function calls which may return null at any point: int val = foo().bar().baz().blah(); To deal with this with if/else statements, you need 3 nested if-statement for a single line of expression. In a language like Scala, you can just write: for (a <- foo(); b <- a.bar(); c <- b.baz()) c.blah(); I think there is an inherent tension between explicitly detailing everything and boilerplate in terms of readability and writeability. I haven't seen enough Go code or written much of anything to say anything about it, except that it worries me. I do think the multi-return stuff can limit the amount of accident ignoring of error conditions, but the Go language itself doesn't provide any high level syntactic constructs to make dealing with the errors less painful, whereas with the Monad approach (which essentially is multi-return), the pattern matching tends to look cleaner IMHO.
- irahul 14y ago> "Maybe Monad" approach of functional languages and Scala, combined with pattern matching in the language tends to look cleaner, e.g. It looks the same to me. case maybeValue of Just value -> ... Nothing -> ... isn't any different from: if (val != null) { .... } else { ... } http://news.ycombinator.com/item?id=4721748 http://news.ycombinator.com/item?id=4721748 > Consider the following chain of function calls which may return null at any point: int val = foo().bar().baz().blah(); Like val = foo.bar.blah.boo rescue nil Or try { val = foo.bar().blah().boo() } catch(NullPointerException ex) { } Or try: val = foo.bar().blah().boo() except AttributeError: pass http://news.ycombinator.com/item?id=4726266 http://news.ycombinator.com/item?id=4726266 Personally, I do see Maybe/Option as cleaner, but it isn't novel as people make it out to be.
- 14y ago
- pretoriusB 14y ago>Go is a programming language designed by Google to help solve Google's problems, and Google has big problems. I believe that this part of the speech is wrong, or overstated (and Rob did it on purpose). There ARE actual texts about how Go came to be, and none of it involves Google, as a corporate entity, wanting something particular and asking for some specific set of requirements, the way, say Ada was designed by the defence departments, of Erlang might have been designed by Ericsson. The story, as told also by Rob Pike elsewhere, speaks of some guys getting together t hack on a new language on some pain points THEY had, and mostly as a side project. And, really, Go was never put front and center by Google, or that Google seems to care a lot about it and advertise. They've done far more for V8 and even Dart, including building a large team of full time paid compiler guys to work on those, whereas the did nothing of the sort for Go. Not much adoption at Google either: some mysql load balancing service in YouTube, a caching layer for Google downloads, a Google doodle served by Go, and beta support in GAE pretty much sums it. Sure there should be some other small projects, but nothing earth shattering, the way C++/Java are used.
- chimeracoder 14y ago> Sure there should be some other small projects, but nothing earth shattering, the way C++/Java are used Go is an incredibly young language - it only just received its first official release a few months ago, and yet it already has two complete and independent compiler implementations (both gc and the more optimized[1] gccgo), as well as a staggering number of libraries for most common tasks. As someone who enjoys toying around with developing languages, I've been blown away by how solid the entire Go ecosystem is at this early stage. > The story, as told also by Rob Pike elsewhere, speaks of some guys getting together t hack on a new language on some pain points THEY had, and mostly as a side project. I think every language (or every project) has as many stories of its origin as there are creators - if not more. They're not mutually exclusive, as you seem to make it look. [1] in that it takes advantage of gcc, which is incredibly well-optimized for non-concurrent applications, that is
- jzelinskie 14y agollgo (an llvm go compiler written in go) is also happily on its way to being a third compiler. Andrew Wilkins recently made a statement that he was able to get some Go code running in Chrome via PNaCl. Pretty exciting stuff.