15 ms·

Rewriting a large production system in Go

238 points by qdie 13y ago

orenbarzilai 13y ago
Can you share your new system benchmarks comparing to the old c++ system?
- mdwelsh 13y ago
  Unfortunately, probably not. Hopefully one day.
zzzeek 13y ago
> If I could get out of the 1970s and use an editor other than vi, maybe I would get some help from an IDE in this regard, but I staunchly refuse to edit code with any tool that requires using a mouse. So you've shown that Go is appealing to rigid curmudgeons. Personally I'm still hung up on the "every function (edit: that does anything which might itself return an error code, which in large scale code is quite a lot) must return an error code" thing. Just like not understanding the purpose and usefulness of graphical editors, the creators of Go seemingly (despite their legendary status...) seemingly not understanding the purpose and usefulness of exceptions (waving them off as "they result in convoluted code" with no explanation) is keeping me pretty skeptical. It just requires I take in some well written, idiomatic Go code so that I can finally "get it", how I'd live without exceptions, but I'm too busy being productive in Python (and enjoying Pypy's ever increasing speedups) to get that interested.
- BarkMore 13y ago
  It is not the case that every function most return an error. The Go creators do understand the purpose and usefulness of exceptions. They chose not to use exceptions with this knowledge. See http://golang.org/doc/faq#exceptions http://golang.org/doc/faq#exceptions for their reasons.
  - jlgreco 13y ago
    Go basically does have exceptions, though they call it something else, it is slightly less powerful than you would normally expect, and the idiomatic usage is very different. Honestly I that Go's idiomatic usage of panic/recover is a good idea even in other languages. Exceptions passing package boundaries is typically a pain in the ass in practice, it makes for ugly code. (My "day job" is typically Java these days, some Scala. I am not speaking from inexperience like many Go detractors like to assume. There seems to be a perverse notion that anyone who dislikes exceptions must not understand them. Silly.)
  - sgift 13y ago
    I don't know. The quality of this FAQ entry is in my opinion narrowed by their inclusion of FileNotFoundException, which it seems they did not understand. The use case for this exception is not to allow lazy programmers to use it instead of checking that a file exists, but to signal to a programmer that his world view of the state of his program may be wrong despite his best efforts, e.g. if(file.exists()) { //do something } There's a race condition between the if and do something which invalidates the programmers world view (someone can delete the file between these two statements). And this is an exceptional situation the program has to deal with. Error codes may tell the programmer this, but it is quite likely that the programmer will just ignore it, because "I've already checked that it exists - what could possibly go wrong?". Exceptions, especially checked exceptions (in my opinion the only good exceptions for "normal" program code)(1), force the programmer to deal with this problem. Or to say - deliberately - "Hey, program, I don't care for the stability of my software. Just explode if this happens!". (1) Languages which have only unchecked exceptions do, in my opinion, cave in to the laziness of programmers: "but, but, it is so much WORK to deal with all of this. Can't it just go away? Please?" - the result is code which can explode everywhere. This is even worse than no exceptions. With return codes you know at least that you have to check the code yourself very, very carefully all the time.
    - enneff 13y ago
      > There's a race condition between the if and do something which invalidates the programmers world view (someone can delete the file between these two statements). And this is an exceptional situation the program has to deal with. It's not an exceptional situation. It's a bug in your program. And it's not lazy to open() and check for existence. It's actually the _only_ sane way to check that the file exists, if you plan to open it. > Error codes may tell the programmer this, but it is quite likely that the programmer will just ignore it, because "I've already checked that it exists - what could possibly go wrong?" The following code is obviously wrong because of the race condition you mention. Nobody in their right mind would write code like this. if exists(file) { f = open(file) // do something with f } else { // handle "file not found" case } This code is correct, to some degree: try { f = open(file) // do something with f } catch (file not found error) { // handle "file not found" case } As is this Go code: f, err := os.Open(file) if os.IsNotExist(err) { // handle "file not found" case } else if err != nil { // handle any other error that might arise } // do something with f The thing is, the above code is not me being extra careful about checking errors. It's just bog standard Go code. Checking error values is the only way to write even half-decent Go code, so everyone does it all the time. No reasonable programmer would write Go code like this: _, err := os.Stat(file) if err != nil { // handle "file not found" case } else { f, _ := os.Open(file) // do something with f // (or explode if file doesn't exist) } To my Go programmer eyes, the underscore (where I'm ignoring the error) in the os.Open line sticks out like a sore thumb. You wouldn't write it, and when reading you would certainly notice it as bad code (or at the very least extraordinary code).
- lbarrow 13y ago
  There's no requirement at all that functions return error codes. If a function can't fail, or if you don't care about if it fails, don't return an error.
- codezero 13y ago
  http://golang.org/doc/faq#exceptions http://golang.org/doc/faq#exceptions A function isn't required to return an error code, but if you plan to deal with errors, it's the easiest way to do that. I think the Go creators understand the purpose and usefulness of exceptions, they just chose to not implement them in favor of other approaches which work fine, of course, it's a matter of taste, as with almost any "language difference" argument. If you get real hung up on these differences, one might argue that you are the curmudgeon. I didn't find learning/implementing a service in Go to be terribly challenging, even though I translated an existing Python service to Go. There were frustrating caveats that I had to work around because of some of the great things Python does, but over all the benefits of doing it in Go paid off, and it's about getting something for your time/effort. Your last argument is totally legitimate though, but it's hard to say what would help you "get it" by throwing random bits of code at you, it's really something that you have to care about, spend the time to dig into, and make the realization for yourself. It may never happen, and as long as Python does everything you need, of course you'd have no incentive to use Go. There seems to be a common theme (generally speaking) on HN that after the front page reaches some saturation on a particular subject, people start being very critical of it, not on its merit, but because it is taking up space where they expect to see a diverse set of content. I can sympathize with this, but I think it's best to try to dig for the value that others seem to be getting out of things, rather than sighing at the constant cheers of others, just my opinion though.
- rshm 13y ago
  I was hung up on error as return instead of exceptions but came in terms eventually. Now i think, compared to exceptions the error handeling is better off with go's approach. They are local, and require more concious effort. Specially while coding for the critical systems this helps as you are not passing the buck to some other section of the code.
- MetaCosm 13y ago
  Don't confuse not understanding something with understanding it and thinking it is a poor idea. The Go authors understand exceptions perfectly -- they just thought that it was a bad idea: http://golang.org/doc/faq#exceptions http://golang.org/doc/faq#exceptions
  - zzzeek 13y ago
    yes I've read that and their reason is "it results in convoluted code" - which is not at all my experience, programming in Java and Python for many years, it's worse in Java for sure due to the heavy emphasis on checked exceptions, but in Python they are a dream. "It also tends to encourage programmers to label too many ordinary errors, such as failing to open a file, as exceptional." also not true in my experience. I really disagree with the notion of hobbling a language just to prevent beginners from writing bad code. Beginners will always write bad code no matter what. I'm not a beginner, and I really don't need to be denied useful tools just because beginners will misuse them - I mean we're talking about improving on C/C++ for chrissakes, in the hands of a beginner those languages are like nuclear weapons. This particular FAQ entry makes it seem very much like the authors have just not seen exceptions used effectively, which I find kind of astonishing. I watched Bruce Eckel's talk at Pycon, "Rethinking Errors - Learning from Scala and Go" (http://us.pycon.org/2013/schedule/presentation/52/ http://us.pycon.org/2013/schedule/presentation/52/) and I was so ready to be converted. But his arguments were pretty unconvincing. Can't someone just make this case convincingly?
    - dragonwriter 13y ago
      If you think exceptions are the right way, you can use Go's exceptions (called panics). Whatever they thought about the desirability of using exceptions, its not like the creators of Go didn't build them into the language.
      - zzzeek 13y ago
        If I'm going to use Go, or any language, I definitely want to use it idiomatically. Don't fret, if this were eight years ago you'd have seen me railing on significant whitespace. And we all know how that went.
- dragonwriter 13y ago
  Go doesn't require all functions to return error codes, and Go has exceptions (called panics). In Go, conventionally, the publicly exposed functions in a module shouldn't usually panic but should use error returns to report unusual conditions. But that's a convention to keep the behavior of functions clear from the interfaces, not a language limitation.
  - pekk 13y ago
    The language has panics. But you aren't supposed to use them. So...
    - mseepgood 13y ago
      Of course you are supposed to use them when appropriate; they have their uses just as goto can have its valid uses. The implementation of the standard library contains some examples. But they should be used judiciously and not for mere error handling and not across package boundaries.
    - elithrar 13y ago
      > The language has panics. But you aren't supposed to use them. So... Not true. Panic/defer/recover is there to be used. But panic/recover should typically be internal to a package and/or used in a situation where the program itself suffers an error; not because of faulty user input/validation/etc. Edit: What mseepgood said!
    - dragonwriter 13y ago
      > The language has panics. But you aren't supposed to use them. Conventionally, you generally shouldn't use them in a way that crosses the public interface of the module in which they are used.
- mdwelsh 13y ago
  The joke goes, "If every other line of code you write starts with 'if err != nil', you might be a Go programmer."
  - JulianMorrison 13y ago
    No, we can just see the "if err != nil". It's still there in Java or Ruby or Python, lurking in the notional space between the lines and waiting to branch to an exception handler you installed in the caller's caller's caller.
    - pcwalton 13y ago
      Same in Go, because of panic and recover. For example, "foo.Bar", if foo is a nil pointer, can invisibly branch to an exception handler you installed in the caller's caller's caller.
      - JulianMorrison 13y ago
        Can but idiomatically, usually does not. In most code, that would behave like a segfault with a more helpful crash log.
- pjmlp 13y ago
  That is exactly because of those type of developers in the industry that the latest Bret Victor's talk is so interesting http://vimeo.com/71278954 http://vimeo.com/71278954 Basically he goes on to show how many people still code like the 70's instead of adopting languages and paradigms that were already possible in late 60's systems.
  - zzzeek 13y ago
    i hate watching videos but yes, this is really interesting - von neumann: "I don't see why anyone would need anything other than machine code". I deal with the resistance thing a lot in my work with ORMs, I should integrate some of this into my writing.
- gatestone 13y ago
  Exceptions are just glorified gotos. The longest single time in my life chasing a bug was because an exception was firing somehere down udner and nobody noticed, because it was kind of part of the logic but a fringe case.
  - zzzeek 13y ago
    this is a thoroughly debunked argument that Joel tried to make. Unlike goto, exceptions have stack traces, so when used correctly, their source and propagation are immediately obvious and traceable. An unreported exception in your buggy program is certainly no worse than an ignored error code.
    - gatestone 13y ago
      The exception was catches somewhere It formed an unexpected, rare execution path that nobody could think of. It was not detectable by reading the local code. A missing error return would have been.
      - zzzeek 13y ago
        inappropriately catching and ignoring an exception is the same as ignoring an error return code, deeper down the call stack and not in the code you're looking at. the difference is, catching and ignoring the exception requires that it be actively done, whereas ignoring an error code is the default if no action is taken.
        gatestone 13y ago
        You have a point. I'll think about it ;-)
pvdm 13y ago
Unless it is open sourced, I can't validate the claims.
- codezero 13y ago
  If you have the time, rewrite one of your own services in Go, open source it, and share the post here on Hacker News.
- packetslave 13y ago
  I'm not sure anyone is asking you to "validate the claims". If you'd like some reassurance that OP knows what he's talking about, you might consider that Harvard tends not to hire stupid professors and then give them tenure.
  - pvdm 13y ago
    I am well aware of Matt Welsh's reputation and what institution he was from. However, even Einstein or Feynman's scientific papers are subjected to peer review.
- mdwelsh 13y ago
  I would love to open source this system, but even if we did I'm not sure we'd be able to convince you that using Go was better than some other language in terms of developer productivity. How would having access to the source help?
  - pvdm 13y ago
    You mentioned in the article that you read a section of code by one of the lead engineers and immediately you became convinced that using Go is the "way-to-go". Well, I would like to experience that eureka moment as well.
    - mdwelsh 13y ago
      Good point. Maybe you could check out one of the many other open source Go projects out there as an alternative. I'm not sure reading our code would give you that eureka moment, since it only made sense to me since I was so familiar with the old code :-)
derefr 13y ago
You know, every time I see some Googler shocked at the effectiveness and various advantages of coding in Go, I wonder why Google never adopted Erlang. They could have been getting all these same advantages (and then some) a decade ago :)
- pekk 13y ago
  Are Go and Erlang that fungible with each other? Honest question.
  - jlgreco 13y ago
    Not particularly. They are basically just both languages that were inspired by CSP; the similarities don't go much deeper.
    - Dn_Ab 13y ago
      A minor correction. Erlang was inspired by actors while Go is inspired by CSP. Both models share a lot but have a few distinguishing factors in how they handle communication. http://en.wikipedia.org/wiki/Communicating_sequential_processes#Comparison_with_the_Actor_Model http://en.wikipedia.org/wiki/Communicating_sequential_proces..., http://en.wikipedia.org/wiki/Actor_model_and_process_calculi http://en.wikipedia.org/wiki/Actor_model_and_process_calculi
  - derefr 13y ago
    Not in theory, but yes in practice. For just one example, Go produces static native binaries, while Erlang produces bytecode for a virtual machine. But the Erlang virtual machine is tiny and it's standard practice (with tool support) to ship it with your application as a "release", so either way you get the effects of having one self-sufficient blob of code in a folder that you can "just run" without having to think about runtimes or libraries. What I would say is that, for every IO-bound highly-concurrent C++ project Google is rewriting into Go, the same project could have been rewritten into Erlang, and they'd see most of the same advantages relative to C++: better, more transparent concurrency; "batteries included" for solving various client-server and distribution problems; being able to just drop a code package on a server and run it; etc.
lbarrow 13y ago
It's interesting to see that more and more internal projects at Google are getting redone in Go. However, this sentence stands out to me in the article: "Before doing the rewrite, we realized we needed only a small subset of the functionality of the original system -- perhaps 20% (or less) of what the other projects were doing with it." I'm guessing that a lot of the benefit of the rewrite came simply from the simplification of the core logic and dropping extraneous functionality. That said, having written a little bit of both C++ and Go, I can completely see why the author found that Go was both far more readable and more maintainable than C++.
- dodyg 13y ago
  This is why a "go" rewrite story is not that appealing to me. You can rewrite a subset of a system and gain many improvements simply by benefits of hindsight and clearer requirements regardless of languages.
  - gizzlon 13y ago
    Obviously, but the author is smart enough to acknowledge this and take it into consideration. So.. what's your point?
    - dodyg 13y ago
      Original production code is messy, especially the ones that have been maintained over the years through multiple requirement changes. A rewrite will make those messiness go away, for now, regardless on languages used.
      - asadigger 13y ago
        Never mind. The OP was a professor. That is how an academic paper is written: explicitly mentioning the real reason in just one or two sentences to avoid attacks from reviews, then associating a very minor or irrelevant fact to the phenomenon as the novel finding and main contribution of their work.
- 13y ago
kailuowang 13y ago
It disheartens me a bit when people compare Go with C++. It gave me the impression that Go was created to be better C++ (or Java) mostly to make life easier for the thousands of developers at Google who are stuck with those two languages.
- lbarrow 13y ago
  My understanding is that that is, in a single sentence, exactly why the project initially got started at Google. http://www.infoq.com/presentations/Go-Google http://www.infoq.com/presentations/Go-Google
- enneff 13y ago
  That is at least partially the case. The other part of the story is that no existing languages were specifically designed for programming at a large scale. Details: http://talks.golang.org/2012/splash.article http://talks.golang.org/2012/splash.article
- mdwelsh 13y ago
  OP here. What do you think we should be using apart from C++ or Java?
  - azth 13y ago
    Scala. Rust also looks extremely promising, though not mature yet. It is getting there quickly though.
    - Lewisham 13y ago
      DISCLAIMER: I was an intern at Google As with most programming languages, there is a small Scala following internally, but it's not a production language. I don't know anything about any discussions about Scala. If I was thinking of deploying Scala, I'd be worried that Scala just doesn't offer enough over Java to warrant writing in a new language. Writing in Scala means not that you have to know it, but everyone who inherits it, and everyone who has to interoperate with it. The overhead is just too great to make a good case for writing something in Scala.
bane 13y ago
> So programming in Go is making me soft. Something similar happened to me years ago when I started picking up Perl. All the BS C++ and Java made me go through to get anything done seemed like such a huge waste of time I ended just writing lots of stuff in Perl until I had pretty much forgotten how to C++ and how to Java. Go might be a new Perl in that sense. I'd also add that as Google moves more and more production systems onto Go, it will literally become responsible for handling billions of dollars of business. That and because Google is using it (and everybody wants to work at Google so knowing some Go ahead of time will be useful) will lead universities to start covering it and before long this virtuous cycle will result in Go everywhere. I've also noticed a number of posts here over the last few months extolling the virtues of moving off of slow frameworks built around slow scripting languages in terms of huge reductions in infrastructure costs. For each system that Google brings on-line with Go, even if it took a team of 10 6 months, they've likely saved themselves millions of dollars in new capex infrastructure acquisition. Imagine serving 40x the requests off of the same old hardware. Alternatively, you might be able to use slower, but more energy efficient hardware to serve the same number of users and save on power and heat management. All this combined with Moore's law means this is a fantastic idea at this kind of scale.
- npalli 13y ago
  "For each system that Google brings on-line with Go, even if it took a team of 10 6 months, they've likely saved themselves millions of dollars in new capex infrastructure acquisition" I thought the old systems were in c++. So, how did the infrastructure savings accrue? In fact, I don't recall anyone doing analysis on that front.
  - dgacmu 13y ago
    Matt linked to my course, but also: I did a tiny analysis of this with my pi searcher (http://da-data.blogspot.com/2013/05/improving-pi-searchers-speed-by-moving.html http://da-data.blogspot.com/2013/05/improving-pi-searchers-s... ) -- it's a toy compared to the kind of system that Matt described, but my experience was extremely positive. Rewriting it in Go made it easier to architect the system right to take advantage of persistence. It's hugely faster. We also have a paper accepted to SOSP this year (the academic weenies will understand that) where the system - a Paxos variant - is implemented in Go. We also implemented about four other Paxos variants in Go to compare against them, a task that would have been absolutely grueling in C++. In Go it wasn't too bad. Our performance isn't super-blazing, but it turns out to be one of the faster publicly available Paxos implementations anyway. I'll have to write up a bit about our experience with it -- I'm with Matt 100% on this one.
jussij 13y ago
> Go's type inference makes for lean code, but requires you to dig a little to figure out what the type of a given variable is if it's not explicit. So given code like: foo, bar := someFunc(baz) > You'd really like to know what foo and bar actually are, in case you want to add some new code to operate on them. The gocode utility (which works with several editors) does exactly this.
grannyg00se 13y ago
"So given code like: foo, bar := someFunc(baz) You'd really like to know what foo and bar actually are" Wouldn't your editor, mouseless though it may be, provide for that? Does ctags not work with vi? And why use vi over vim?
- mdwelsh 13y ago
  OP here. I do actually use vim but I have yet to adopt most of the fancy plugins that vim provides -- put me in a time machine back to 1977 and I would be very capable of programming on any UNIX system that you'd drop me in front of, provided it had vi installed. I'm not defending this choice of lifestyle; it's just how I learned to program :-)
  - lifebeyondfife 13y ago
    Even in the 2010s I've found proponents of the UNIX console style of programming because, as you say, they can be dropped in front of any similar system and off they go. However, tools improve and you're denying yourself productivity benefits. One of the advantages of a statically typed language is that the computer knows the type of the arguments. I love environments that allow me to 'mouse over' a function or variable etc. and tell me exactly what it is (and allow me to navigate to its definition). You don't have to use a mouse but what's the disadvantage in having alternatives to keyboard shortcuts where appropriate?
rgarcia 13y ago
Re: figuring out the signature of functions or the definition of structs, I've found godef useful: http://godoc.org/code.google.com/p/rog-go/exp/cmd/godef http://godoc.org/code.google.com/p/rog-go/exp/cmd/godef The Go emacs mode has godef-jump (C-c C-j) and godef-describe (C-c C-d): http://golang.org/misc/emacs/go-mode.el http://golang.org/misc/emacs/go-mode.el
sytelus 13y ago
The number of strongly typed garbage collected compiled languages is surely shrinking. I can understand someone's disgust towards Java considering dwindling velocity in adding next-gen features and now sleeping in same bed as lawyer run Oracle. That pretty much had left C# in the arena until Go came on the scene. I was honestly hoping Go would give us head-to-head battle with C# but from initial looks Go pretty much feels like C# 1.0. No generics, no partial classes, no linq... C# 5.0 even has async and dynamic types. I like minimalism but these features are something we take for granted in languages that were created post-2000 era. So it all boils down to people choosing Go for singular reason: It's not from Microsoft! If licensing for C# wasn't a barrier and let's stretch our imagination by thinking C# was created by some other group in Google, would Go have a chance?
- pjmlp 13y ago
  Yep, Go would be hardly mentioned here if it wasn't being developed at Google. Having said this, Go can be a nice replacement for many use cases one would use C for.
  - scott_s 13y ago
    Go seems to be gaining traction. Does it matter if that's because of where it came from? Maybe in a cultural, I-want-to-think-about-why-some-languages-get-adopted sense, sure, then it can matter. And I think it's worth having those discussions. But I get the impression that people think there's some absolute injustice in the idea that people are adopting a language, and it's partly because it's from Google. Consider: "Yes, C would be hardly mentioned here if it wasn't being developed at Bell Labs."
    - im3w1l 13y ago
      It matters if you try to count backwards from popularity to quality, by correcting for other factors.
    - pjmlp 13y ago
      > "Yes, C would be hardly mentioned here if it wasn't being developed at Bell Labs." Fully agree, most likely if UNIX hadn't been adopted at important American universities, C wouldn't have spread as it did.
lucian1900 13y ago
Interesting that Go's lack of HM type inference is that problematic. In, Rust or Haskell, one could simply use that value further on with some type and have it inferred correctly.
dodyg 13y ago
Go is a nice language with awesome features except that it doesn't fucking allow unused imports/variables. It makes exploratory programming with Go annoying as hell.
- im3w1l 13y ago
  don't { print x for {x : unused variables} x() for {x : function from unused imports} }
deleted 13y ago
[deleted]

238 points by qdie 13y ago

orenbarzilai 13y ago

Can you share your new system benchmarks comparing to the old c++ system?

mdwelsh 13y ago
Unfortunately, probably not. Hopefully one day.

zzzeek 13y ago

> If I could get out of the 1970s and use an editor other than vi, maybe I would get some help from an IDE in this regard, but I staunchly refuse to edit code with any tool that requires using a mouse. So you've shown that Go is appealing to rigid curmudgeons. Personally I'm still hung up on the "every function (edit: that does anything which might itself return an error code, which in large scale code is quite a lot) must return an error code" thing. Just like not understanding the purpose and usefulness of graphical editors, the creators of Go seemingly (despite their legendary status...) seemingly not understanding the purpose and usefulness of exceptions (waving them off as "they result in convoluted code" with no explanation) is keeping me pretty skeptical. It just requires I take in some well written, idiomatic Go code so that I can finally "get it", how I'd live without exceptions, but I'm too busy being productive in Python (and enjoying Pypy's ever increasing speedups) to get that interested.

BarkMore 13y ago
It is not the case that every function most return an error. The Go creators do understand the purpose and usefulness of exceptions. They chose not to use exceptions with this knowledge. See http://golang.org/doc/faq#exceptions http://golang.org/doc/faq#exceptions for their reasons.
- jlgreco 13y ago
  Go basically does have exceptions, though they call it something else, it is slightly less powerful than you would normally expect, and the idiomatic usage is very different. Honestly I that Go's idiomatic usage of panic/recover is a good idea even in other languages. Exceptions passing package boundaries is typically a pain in the ass in practice, it makes for ugly code. (My "day job" is typically Java these days, some Scala. I am not speaking from inexperience like many Go detractors like to assume. There seems to be a perverse notion that anyone who dislikes exceptions must not understand them. Silly.)
- sgift 13y ago
  I don't know. The quality of this FAQ entry is in my opinion narrowed by their inclusion of FileNotFoundException, which it seems they did not understand. The use case for this exception is not to allow lazy programmers to use it instead of checking that a file exists, but to signal to a programmer that his world view of the state of his program may be wrong despite his best efforts, e.g. if(file.exists()) { //do something } There's a race condition between the if and do something which invalidates the programmers world view (someone can delete the file between these two statements). And this is an exceptional situation the program has to deal with. Error codes may tell the programmer this, but it is quite likely that the programmer will just ignore it, because "I've already checked that it exists - what could possibly go wrong?". Exceptions, especially checked exceptions (in my opinion the only good exceptions for "normal" program code)(1), force the programmer to deal with this problem. Or to say - deliberately - "Hey, program, I don't care for the stability of my software. Just explode if this happens!". (1) Languages which have only unchecked exceptions do, in my opinion, cave in to the laziness of programmers: "but, but, it is so much WORK to deal with all of this. Can't it just go away? Please?" - the result is code which can explode everywhere. This is even worse than no exceptions. With return codes you know at least that you have to check the code yourself very, very carefully all the time.
  - enneff 13y ago
    > There's a race condition between the if and do something which invalidates the programmers world view (someone can delete the file between these two statements). And this is an exceptional situation the program has to deal with. It's not an exceptional situation. It's a bug in your program. And it's not lazy to open() and check for existence. It's actually the _only_ sane way to check that the file exists, if you plan to open it. > Error codes may tell the programmer this, but it is quite likely that the programmer will just ignore it, because "I've already checked that it exists - what could possibly go wrong?" The following code is obviously wrong because of the race condition you mention. Nobody in their right mind would write code like this. if exists(file) { f = open(file) // do something with f } else { // handle "file not found" case } This code is correct, to some degree: try { f = open(file) // do something with f } catch (file not found error) { // handle "file not found" case } As is this Go code: f, err := os.Open(file) if os.IsNotExist(err) { // handle "file not found" case } else if err != nil { // handle any other error that might arise } // do something with f The thing is, the above code is not me being extra careful about checking errors. It's just bog standard Go code. Checking error values is the only way to write even half-decent Go code, so everyone does it all the time. No reasonable programmer would write Go code like this: _, err := os.Stat(file) if err != nil { // handle "file not found" case } else { f, _ := os.Open(file) // do something with f // (or explode if file doesn't exist) } To my Go programmer eyes, the underscore (where I'm ignoring the error) in the os.Open line sticks out like a sore thumb. You wouldn't write it, and when reading you would certainly notice it as bad code (or at the very least extraordinary code).
lbarrow 13y ago
There's no requirement at all that functions return error codes. If a function can't fail, or if you don't care about if it fails, don't return an error.
codezero 13y ago
http://golang.org/doc/faq#exceptions http://golang.org/doc/faq#exceptions A function isn't required to return an error code, but if you plan to deal with errors, it's the easiest way to do that. I think the Go creators understand the purpose and usefulness of exceptions, they just chose to not implement them in favor of other approaches which work fine, of course, it's a matter of taste, as with almost any "language difference" argument. If you get real hung up on these differences, one might argue that you are the curmudgeon. I didn't find learning/implementing a service in Go to be terribly challenging, even though I translated an existing Python service to Go. There were frustrating caveats that I had to work around because of some of the great things Python does, but over all the benefits of doing it in Go paid off, and it's about getting something for your time/effort. Your last argument is totally legitimate though, but it's hard to say what would help you "get it" by throwing random bits of code at you, it's really something that you have to care about, spend the time to dig into, and make the realization for yourself. It may never happen, and as long as Python does everything you need, of course you'd have no incentive to use Go. There seems to be a common theme (generally speaking) on HN that after the front page reaches some saturation on a particular subject, people start being very critical of it, not on its merit, but because it is taking up space where they expect to see a diverse set of content. I can sympathize with this, but I think it's best to try to dig for the value that others seem to be getting out of things, rather than sighing at the constant cheers of others, just my opinion though.
rshm 13y ago
I was hung up on error as return instead of exceptions but came in terms eventually. Now i think, compared to exceptions the error handeling is better off with go's approach. They are local, and require more concious effort. Specially while coding for the critical systems this helps as you are not passing the buck to some other section of the code.
MetaCosm 13y ago
Don't confuse not understanding something with understanding it and thinking it is a poor idea. The Go authors understand exceptions perfectly -- they just thought that it was a bad idea: http://golang.org/doc/faq#exceptions http://golang.org/doc/faq#exceptions
- zzzeek 13y ago
  yes I've read that and their reason is "it results in convoluted code" - which is not at all my experience, programming in Java and Python for many years, it's worse in Java for sure due to the heavy emphasis on checked exceptions, but in Python they are a dream. "It also tends to encourage programmers to label too many ordinary errors, such as failing to open a file, as exceptional." also not true in my experience. I really disagree with the notion of hobbling a language just to prevent beginners from writing bad code. Beginners will always write bad code no matter what. I'm not a beginner, and I really don't need to be denied useful tools just because beginners will misuse them - I mean we're talking about improving on C/C++ for chrissakes, in the hands of a beginner those languages are like nuclear weapons. This particular FAQ entry makes it seem very much like the authors have just not seen exceptions used effectively, which I find kind of astonishing. I watched Bruce Eckel's talk at Pycon, "Rethinking Errors - Learning from Scala and Go" (http://us.pycon.org/2013/schedule/presentation/52/ http://us.pycon.org/2013/schedule/presentation/52/) and I was so ready to be converted. But his arguments were pretty unconvincing. Can't someone just make this case convincingly?
  - dragonwriter 13y ago
    If you think exceptions are the right way, you can use Go's exceptions (called panics). Whatever they thought about the desirability of using exceptions, its not like the creators of Go didn't build them into the language.
    - zzzeek 13y ago
      If I'm going to use Go, or any language, I definitely want to use it idiomatically. Don't fret, if this were eight years ago you'd have seen me railing on significant whitespace. And we all know how that went.
dragonwriter 13y ago
Go doesn't require all functions to return error codes, and Go has exceptions (called panics). In Go, conventionally, the publicly exposed functions in a module shouldn't usually panic but should use error returns to report unusual conditions. But that's a convention to keep the behavior of functions clear from the interfaces, not a language limitation.
- pekk 13y ago
  The language has panics. But you aren't supposed to use them. So...
  - mseepgood 13y ago
    Of course you are supposed to use them when appropriate; they have their uses just as goto can have its valid uses. The implementation of the standard library contains some examples. But they should be used judiciously and not for mere error handling and not across package boundaries.
  - elithrar 13y ago
    > The language has panics. But you aren't supposed to use them. So... Not true. Panic/defer/recover is there to be used. But panic/recover should typically be internal to a package and/or used in a situation where the program itself suffers an error; not because of faulty user input/validation/etc. Edit: What mseepgood said!
  - dragonwriter 13y ago
    > The language has panics. But you aren't supposed to use them. Conventionally, you generally shouldn't use them in a way that crosses the public interface of the module in which they are used.
mdwelsh 13y ago
The joke goes, "If every other line of code you write starts with 'if err != nil', you might be a Go programmer."
- JulianMorrison 13y ago
  No, we can just see the "if err != nil". It's still there in Java or Ruby or Python, lurking in the notional space between the lines and waiting to branch to an exception handler you installed in the caller's caller's caller.
  - pcwalton 13y ago
    Same in Go, because of panic and recover. For example, "foo.Bar", if foo is a nil pointer, can invisibly branch to an exception handler you installed in the caller's caller's caller.
    - JulianMorrison 13y ago
      Can but idiomatically, usually does not. In most code, that would behave like a segfault with a more helpful crash log.
pjmlp 13y ago
That is exactly because of those type of developers in the industry that the latest Bret Victor's talk is so interesting http://vimeo.com/71278954 http://vimeo.com/71278954 Basically he goes on to show how many people still code like the 70's instead of adopting languages and paradigms that were already possible in late 60's systems.
- zzzeek 13y ago
  i hate watching videos but yes, this is really interesting - von neumann: "I don't see why anyone would need anything other than machine code". I deal with the resistance thing a lot in my work with ORMs, I should integrate some of this into my writing.
gatestone 13y ago
Exceptions are just glorified gotos. The longest single time in my life chasing a bug was because an exception was firing somehere down udner and nobody noticed, because it was kind of part of the logic but a fringe case.
- zzzeek 13y ago
  this is a thoroughly debunked argument that Joel tried to make. Unlike goto, exceptions have stack traces, so when used correctly, their source and propagation are immediately obvious and traceable. An unreported exception in your buggy program is certainly no worse than an ignored error code.
  - gatestone 13y ago
    The exception was catches somewhere It formed an unexpected, rare execution path that nobody could think of. It was not detectable by reading the local code. A missing error return would have been.
    - zzzeek 13y ago
      inappropriately catching and ignoring an exception is the same as ignoring an error return code, deeper down the call stack and not in the code you're looking at. the difference is, catching and ignoring the exception requires that it be actively done, whereas ignoring an error code is the default if no action is taken.
      - gatestone 13y ago
        You have a point. I'll think about it ;-)

pvdm 13y ago

Unless it is open sourced, I can't validate the claims.

codezero 13y ago
If you have the time, rewrite one of your own services in Go, open source it, and share the post here on Hacker News.
packetslave 13y ago
I'm not sure anyone is asking you to "validate the claims". If you'd like some reassurance that OP knows what he's talking about, you might consider that Harvard tends not to hire stupid professors and then give them tenure.
- pvdm 13y ago
  I am well aware of Matt Welsh's reputation and what institution he was from. However, even Einstein or Feynman's scientific papers are subjected to peer review.
mdwelsh 13y ago
I would love to open source this system, but even if we did I'm not sure we'd be able to convince you that using Go was better than some other language in terms of developer productivity. How would having access to the source help?
- pvdm 13y ago
  You mentioned in the article that you read a section of code by one of the lead engineers and immediately you became convinced that using Go is the "way-to-go". Well, I would like to experience that eureka moment as well.
  - mdwelsh 13y ago
    Good point. Maybe you could check out one of the many other open source Go projects out there as an alternative. I'm not sure reading our code would give you that eureka moment, since it only made sense to me since I was so familiar with the old code :-)

derefr 13y ago

You know, every time I see some Googler shocked at the effectiveness and various advantages of coding in Go, I wonder why Google never adopted Erlang. They could have been getting all these same advantages (and then some) a decade ago :)

pekk 13y ago
Are Go and Erlang that fungible with each other? Honest question.
- jlgreco 13y ago
  Not particularly. They are basically just both languages that were inspired by CSP; the similarities don't go much deeper.
  - Dn_Ab 13y ago
    A minor correction. Erlang was inspired by actors while Go is inspired by CSP. Both models share a lot but have a few distinguishing factors in how they handle communication. http://en.wikipedia.org/wiki/Communicating_sequential_processes#Comparison_with_the_Actor_Model http://en.wikipedia.org/wiki/Communicating_sequential_proces..., http://en.wikipedia.org/wiki/Actor_model_and_process_calculi http://en.wikipedia.org/wiki/Actor_model_and_process_calculi
- derefr 13y ago
  Not in theory, but yes in practice. For just one example, Go produces static native binaries, while Erlang produces bytecode for a virtual machine. But the Erlang virtual machine is tiny and it's standard practice (with tool support) to ship it with your application as a "release", so either way you get the effects of having one self-sufficient blob of code in a folder that you can "just run" without having to think about runtimes or libraries. What I would say is that, for every IO-bound highly-concurrent C++ project Google is rewriting into Go, the same project could have been rewritten into Erlang, and they'd see most of the same advantages relative to C++: better, more transparent concurrency; "batteries included" for solving various client-server and distribution problems; being able to just drop a code package on a server and run it; etc.

lbarrow 13y ago

It's interesting to see that more and more internal projects at Google are getting redone in Go. However, this sentence stands out to me in the article: "Before doing the rewrite, we realized we needed only a small subset of the functionality of the original system -- perhaps 20% (or less) of what the other projects were doing with it." I'm guessing that a lot of the benefit of the rewrite came simply from the simplification of the core logic and dropping extraneous functionality. That said, having written a little bit of both C++ and Go, I can completely see why the author found that Go was both far more readable and more maintainable than C++.

dodyg 13y ago
This is why a "go" rewrite story is not that appealing to me. You can rewrite a subset of a system and gain many improvements simply by benefits of hindsight and clearer requirements regardless of languages.
- gizzlon 13y ago
  Obviously, but the author is smart enough to acknowledge this and take it into consideration. So.. what's your point?
  - dodyg 13y ago
    Original production code is messy, especially the ones that have been maintained over the years through multiple requirement changes. A rewrite will make those messiness go away, for now, regardless on languages used.
    - asadigger 13y ago
      Never mind. The OP was a professor. That is how an academic paper is written: explicitly mentioning the real reason in just one or two sentences to avoid attacks from reviews, then associating a very minor or irrelevant fact to the phenomenon as the novel finding and main contribution of their work.
13y ago

kailuowang 13y ago

It disheartens me a bit when people compare Go with C++. It gave me the impression that Go was created to be better C++ (or Java) mostly to make life easier for the thousands of developers at Google who are stuck with those two languages.

lbarrow 13y ago
My understanding is that that is, in a single sentence, exactly why the project initially got started at Google. http://www.infoq.com/presentations/Go-Google http://www.infoq.com/presentations/Go-Google
enneff 13y ago
That is at least partially the case. The other part of the story is that no existing languages were specifically designed for programming at a large scale. Details: http://talks.golang.org/2012/splash.article http://talks.golang.org/2012/splash.article
mdwelsh 13y ago
OP here. What do you think we should be using apart from C++ or Java?
- azth 13y ago
  Scala. Rust also looks extremely promising, though not mature yet. It is getting there quickly though.
  - Lewisham 13y ago
    DISCLAIMER: I was an intern at Google As with most programming languages, there is a small Scala following internally, but it's not a production language. I don't know anything about any discussions about Scala. If I was thinking of deploying Scala, I'd be worried that Scala just doesn't offer enough over Java to warrant writing in a new language. Writing in Scala means not that you have to know it, but everyone who inherits it, and everyone who has to interoperate with it. The overhead is just too great to make a good case for writing something in Scala.

bane 13y ago

> So programming in Go is making me soft. Something similar happened to me years ago when I started picking up Perl. All the BS C++ and Java made me go through to get anything done seemed like such a huge waste of time I ended just writing lots of stuff in Perl until I had pretty much forgotten how to C++ and how to Java. Go might be a new Perl in that sense. I'd also add that as Google moves more and more production systems onto Go, it will literally become responsible for handling billions of dollars of business. That and because Google is using it (and everybody wants to work at Google so knowing some Go ahead of time will be useful) will lead universities to start covering it and before long this virtuous cycle will result in Go everywhere. I've also noticed a number of posts here over the last few months extolling the virtues of moving off of slow frameworks built around slow scripting languages in terms of huge reductions in infrastructure costs. For each system that Google brings on-line with Go, even if it took a team of 10 6 months, they've likely saved themselves millions of dollars in new capex infrastructure acquisition. Imagine serving 40x the requests off of the same old hardware. Alternatively, you might be able to use slower, but more energy efficient hardware to serve the same number of users and save on power and heat management. All this combined with Moore's law means this is a fantastic idea at this kind of scale.

npalli 13y ago
"For each system that Google brings on-line with Go, even if it took a team of 10 6 months, they've likely saved themselves millions of dollars in new capex infrastructure acquisition" I thought the old systems were in c++. So, how did the infrastructure savings accrue? In fact, I don't recall anyone doing analysis on that front.
- dgacmu 13y ago
  Matt linked to my course, but also: I did a tiny analysis of this with my pi searcher (http://da-data.blogspot.com/2013/05/improving-pi-searchers-speed-by-moving.html http://da-data.blogspot.com/2013/05/improving-pi-searchers-s... ) -- it's a toy compared to the kind of system that Matt described, but my experience was extremely positive. Rewriting it in Go made it easier to architect the system right to take advantage of persistence. It's hugely faster. We also have a paper accepted to SOSP this year (the academic weenies will understand that) where the system - a Paxos variant - is implemented in Go. We also implemented about four other Paxos variants in Go to compare against them, a task that would have been absolutely grueling in C++. In Go it wasn't too bad. Our performance isn't super-blazing, but it turns out to be one of the faster publicly available Paxos implementations anyway. I'll have to write up a bit about our experience with it -- I'm with Matt 100% on this one.

jussij 13y ago

> Go's type inference makes for lean code, but requires you to dig a little to figure out what the type of a given variable is if it's not explicit. So given code like: foo, bar := someFunc(baz) > You'd really like to know what foo and bar actually are, in case you want to add some new code to operate on them. The gocode utility (which works with several editors) does exactly this.

grannyg00se 13y ago

"So given code like: foo, bar := someFunc(baz) You'd really like to know what foo and bar actually are" Wouldn't your editor, mouseless though it may be, provide for that? Does ctags not work with vi? And why use vi over vim?

mdwelsh 13y ago
OP here. I do actually use vim but I have yet to adopt most of the fancy plugins that vim provides -- put me in a time machine back to 1977 and I would be very capable of programming on any UNIX system that you'd drop me in front of, provided it had vi installed. I'm not defending this choice of lifestyle; it's just how I learned to program :-)
- lifebeyondfife 13y ago
  Even in the 2010s I've found proponents of the UNIX console style of programming because, as you say, they can be dropped in front of any similar system and off they go. However, tools improve and you're denying yourself productivity benefits. One of the advantages of a statically typed language is that the computer knows the type of the arguments. I love environments that allow me to 'mouse over' a function or variable etc. and tell me exactly what it is (and allow me to navigate to its definition). You don't have to use a mouse but what's the disadvantage in having alternatives to keyboard shortcuts where appropriate?

rgarcia 13y ago

Re: figuring out the signature of functions or the definition of structs, I've found godef useful: http://godoc.org/code.google.com/p/rog-go/exp/cmd/godef http://godoc.org/code.google.com/p/rog-go/exp/cmd/godef The Go emacs mode has godef-jump (C-c C-j) and godef-describe (C-c C-d): http://golang.org/misc/emacs/go-mode.el http://golang.org/misc/emacs/go-mode.el

sytelus 13y ago

The number of strongly typed garbage collected compiled languages is surely shrinking. I can understand someone's disgust towards Java considering dwindling velocity in adding next-gen features and now sleeping in same bed as lawyer run Oracle. That pretty much had left C# in the arena until Go came on the scene. I was honestly hoping Go would give us head-to-head battle with C# but from initial looks Go pretty much feels like C# 1.0. No generics, no partial classes, no linq... C# 5.0 even has async and dynamic types. I like minimalism but these features are something we take for granted in languages that were created post-2000 era. So it all boils down to people choosing Go for singular reason: It's not from Microsoft! If licensing for C# wasn't a barrier and let's stretch our imagination by thinking C# was created by some other group in Google, would Go have a chance?

pjmlp 13y ago
Yep, Go would be hardly mentioned here if it wasn't being developed at Google. Having said this, Go can be a nice replacement for many use cases one would use C for.
- scott_s 13y ago
  Go seems to be gaining traction. Does it matter if that's because of where it came from? Maybe in a cultural, I-want-to-think-about-why-some-languages-get-adopted sense, sure, then it can matter. And I think it's worth having those discussions. But I get the impression that people think there's some absolute injustice in the idea that people are adopting a language, and it's partly because it's from Google. Consider: "Yes, C would be hardly mentioned here if it wasn't being developed at Bell Labs."
  - im3w1l 13y ago
    It matters if you try to count backwards from popularity to quality, by correcting for other factors.
  - pjmlp 13y ago
    > "Yes, C would be hardly mentioned here if it wasn't being developed at Bell Labs." Fully agree, most likely if UNIX hadn't been adopted at important American universities, C wouldn't have spread as it did.

lucian1900 13y ago

Interesting that Go's lack of HM type inference is that problematic. In, Rust or Haskell, one could simply use that value further on with some type and have it inferred correctly.

dodyg 13y ago

Go is a nice language with awesome features except that it doesn't fucking allow unused imports/variables. It makes exploratory programming with Go annoying as hell.

im3w1l 13y ago
don't { print x for {x : unused variables} x() for {x : function from unused imports} }

deleted 13y ago

[deleted]