10 ms·

The Reliability of Go

133 points by babawere 13y ago

schappim 13y ago
Summary: At a GoSF meetup audience member asked if Go was reliable... - Op was surprised because he had been using it in production for nearly 3 years. - Previously used it at a Financial Company (wholesaler for institutional share traders and stock brokers). - Op used it previously for system Monitoring, as a data store and messaging middleware. - Go's adoption is gathering pace thanks to the terse syntax, straightforward powerful standard library, excellent tooling and concurrency primitives. [Sic. And performance] - Go's core development team a reason for why it is showing maturity beyond it's years. - Op changed positions so he could work on Go full-time.
- carbocation 13y ago
  Question: is this an automated summary or are you doing these summaries yourself manually? On topic, it seems odd that a Python process that would take 3 minutes to scan nodes could be reduced to a go script that takes under 1 second. Perhaps this is entirely due to going from serial to parallel, but the timeout thing that he mentioned sounds like an error in the Python code rather than a language problem. And I say this as someone who uses/enjoys go.
  - joshbaptiste 13y ago
    Indeed.. I would have liked to hear what took the Python script 3x longer. The longest part I would assume would be the network requests themselves which is inherent regardless of language.
    - eudoxus 13y ago
      its actually 180x faster if it went from 3 mins to less then a second
    - wglb 13y ago
      Sounds like it the execution time required to processing the vast amounts of data produced. go is quite likely to beat python in compute bound or data crunching tasks.
      - Patrick_Devine 13y ago
        Go is definitely faster than Python for those types of tasks, however I'm suspicious of it being that much faster. I've crunched massive data sets in Python and as long as you're aware of its limitations, you can make things pretty zippy.
benbjohnson 13y ago
I wholeheartedly agree with this article. I've had great success with Go's reliability. Another important point to make is that when programs written in Go do fail, it tends to be trivial to figure out why. There's no type hierarchy like most OOP languages and no method_missing magic like Ruby. I can typically just trace the logic backwards from the error to the cause. Most bug reports I receive for my open source Go projects get fixed in minutes.
- khyryk 13y ago
  I suspect it's Ruby (dynamic typing, syntactic flexibility, etc.) rather than OOP itself that's the root of the issue when it comes to tracing logic backwards.
mitchellh 13y ago
I have both production systems (ircrelay.com) and well-used client software (packer.io) written in Go, so I'd like to comment on this from both perspectives, server-side and desktop-side. First off, both from the get-go have been extremely reliable. IRCRelay has a custom IRC router that is able to route IRC connections to their proper bouncer server, then just stream the contents to and from. This server was written once, and has been running in production for over a year without a single downtime incident or crash incident. Likewise, Packer reliability has been astounding, even to me. It really rarely crashes, and when it does it is because I'm usually skirting around the type system of Go (casting). The PRIMARY reason Go is so reliable is also a reason many people hate go: you _have_ to handle every error. There are no exceptions, you know if a line of code can fail or not, and you have to handle it. This leads to a lot of "if err != nil" checks, and it can be really, really annoying sometimes. But the truth is: once you get that thing into production, it is never going to crash. Never. MAYBE nil slice access, but that's about it. Note the compiler doesn't actively enforce this, but it is Go best practice to handle every error. (Pedantics: yes, you can assign errors to "_" variables to ignore them or just not accept any return values, but this is very actively discouraged in Go and it is one of those you're-doing-it-wrong kind of things) Also: a compiler. Coming from Ruby, having a compiler is just amazing. Obviously compilers have been around forever and Go's isn't special in any way. But after living in a dynamic language heavy environment for many years, I can't imagine going back to not having a compiler. It just makes writing new features, refactoring old ones, etc. all just so easy. It catches all the bone-head mistakes that cause a majority of crashes in code I previously wrote. Another reason: Testing is heavily encouraged and baked right into the official build tool. This makes it so easy to write tests that you always do. It isn't really an opt-in thing, because it is one of those things you just DO if you write Go. This makes it so that even with explicit error checking and the type system, you're fairly certain your logic is reasonably correct as well. And finally, you're statically compiling your applications. Once they're running, no external dependencies can mess that up. Dependency management from _source_ form is a problem with Go, one that is actively acknowledge and being worked on. But once that program is compiled, it is safe forever as long as you run it on a proper kernel. So my experience is anecdotal, but there are real language features and community ideologies at play here that make Go a really stable language to write programs in.
sudonim 13y ago
We've been using Go in production for 4 or so months at customer.io and have been tremendously happy with the performance. My cofounder will rewrite some core function in go, deploy it to prod and get a 10x improvement. We are total converts.
- dbaupp 13y ago
  10x improvement compared to what?
  - jrallison 13y ago
    Our MVP was written in Ruby.
    - danieldk 13y ago
      You know that nearly any compiled language would give such speed-ups compared to Ruby?
      - pjmlp 13y ago
        Maybe even Turbo Basic would do the trick! :)
      - jrallison 13y ago
        Of course, which was the main driver to moving to a compiled language. We just happened to choose Go :) I've done a lot of Java and C++ in the past, and wasn't looking to go back to either of those.
th0br0 13y ago
The comments thread on this makes me sort of sad. So... Go rocks because a) you've got to handle all the errors b) there is no method_missing (or its ilk) In general, all those features have been offered by compiled languages for years (even if they might've just been compiler warnings). What makes Go so special other than the lack of deep class hierarchies? Sure, it's more "reliable" by design than 100% of those dynamic languages out there, but that doesn't really make the language more attractive. Coming from an OO world, I find Go's "inheritance" model far inferior to what, e.g., Haskell or Scala offer. Given the comments here I think that I can finally understand somewhat why Go is taking off the way it is, but it feels like a faulty "solution".
- khyryk 13y ago
  a isn't necessarily the case. Consider fmt.Println: func Println(a ...interface{}) (n int, err error) I've never seen any code that handled err -- I don't mean ignoring it with _, I mean not even acknowledging its existence -- without the compiler saying anything.
  - dilap 13y ago
    Yeah, but if print fails, what are you going to do? And it's not like ignoring this error is going to make your code blow up.
    - khyryk 13y ago
      The point is that I can't immediately tell which errors will be mysteriously ignored. This, together with the existence of panics, undermines the stance that one can tell exactly what a certain piece of Go code will do just by looking at, say, a function prototype.
      - cmccabe 13y ago
        Don't be fatuous. If you wrote all of the code, you should be able to tell exactly what it does. If there's some code you're calling that was written by someone else, you have to trust it to do the right thing. Exceptions versus return codes doesn't change that fact. People usually don't check the return code of fmt.Printf because they simply don't care. Most of the time, you're just logging to a terminal, which basically never fails. And if logging did fail, you wouldn't want to abort the program because of it. The only time you might want to check the return value of printf is if you're writing a simple utility designed to transform something passed on stdin to something on stdout.
petercooper 13y ago
I'd seen a couple of references to Go being used by financial companies in London (both for prototyping and in live systems) and this post seems to provide yet more confirmation. Is finance in particular looking to be a forthcoming hot market for Go due to its balance of performance and developer friendliness?
ShabbyDoo 13y ago
What's notable is that no one has yet posted here saying, "Go stunk for me. Perf and reliability were both awful. I went back to blahblah and threw away all my Go code." It's usually easy to find detractors of any technology.
- dsymonds 13y ago
  I think that's because Go is reasonably good for, say, 90% of situations, and if you're in the 10% where it is not a good fit then you already know that and you know better than to try using Go in that situation, and thus few people find Go stinking. On the other side is things like Python, which are reasonably good for a much lower fraction of things (e.g. 30%), but if you're in the 70% you very well might not realise it does not suit your situation and thus experience Python stinking.
- graue 13y ago
  I actually have seen complaints about Go's performance relative to other compiled languages, like C++, D and Java. I don't think it's coincidental that most of the Go converts come from a dynamic-language background. If you're porting from Ruby, 2-3x slower than Java still seems blazing fast. For me, Go misses the sweet spot entirely. A year ago I was curious about both Go and Clojure, my interest piqued by the strong concurrency support each has. I picked Clojure and have grown more and more confident in that choice. Why not Go? In a language with nil pointers (Tony Hoare's "billion-dollar mistake"), no generics, and sub-JVM performance, static typing just doesn't seem worth it. Also, poor interop with other languages means Go can only draw from its own small community for libraries (contrast Clojure which makes using mature, well-tested Java libs trivial). When Rust is more mature, it may be what I hoped Go would be: the safe language, with lightweight threads, that makes me never have to write C or C++ again.
  - ruggeri 13y ago
    Go didn't stick for me either. Go is interesting and I really enjoyed channels, but no generic programming really disappointed me. I guess supporting the generic paradigm is pretty tricky (C++/Java templates/generics aren't dear to my heart), but not being able to implement map/filter/reduce functions...
trimbo 13y ago
> A market maker is basically a wholesaler for institutional share traders and stock brokers. During my time there I replaced several key systems components with Go. I'd like to hear more about how you deal with Go's garbage collection in a real time system like this.
- burntsushi 13y ago
  He may not have had to deal with it at all. He replaced certain components of it with Go code, but none of them may have required real time guarantees.
  - trimbo 13y ago
    In later posts, he refers to doing order books and such: http://andrewwdeane.blogspot.co.uk/2013/09/bullhorn.html http://andrewwdeane.blogspot.co.uk/2013/09/bullhorn.html
pauldix 13y ago
At Errplane we've written our primary datastore in Go and have been running it in production since January. It has been totally reliable, performant, and easy to work with. Easily handles the 100m requests per day we get. At this point it's my language of choice for any back end systems.
- alexk 13y ago
  Would love to know more about the datastore that you've built. Any chance you would elaborate on this?
  - pauldix 13y ago
    Sure! It's really just a Go wrapper around LevelDB. LevelDB is an in process database (like SQLite) written in C++. It's a key value store where the keyspace is ordered (so you can do range scans). However, we've written enough code around it to turn it into a scalable distributed time series database with a pubsub layer built in. It does rollups on the fly or can preemtively downsample data coming in. We're actually working on a new version of it that we intend to open source at the end of the year.
    - alexk 13y ago
      That'd be awesome if you open sourced this!
    - gigq 13y ago
      That sounds pretty useful, something I would love to see open sourced as I've thought about making a distributed LevelDB based DB as well. Any word on how it handles the distribution? Are we talking a quorum based system like Cassandra/Dynamo or are we talking more like a master / key range type system like HBase.
      - pauldix 13y ago
        The version we're working on now uses the go Raft implementation to handle cluster configuration. Then each server periodically (< 30s) performs read repairs to ensure that it has the latest writes for its spots in the hash ring. Writes can either be done via either write once or quorum, with write once being the most often used thing. Since most of the time this data is in aggregate, we don't care about losing a few writes.
taliesinb 13y ago
Unfortunately, by insisting that all data start in a zeroed state (as opposed to forbidding the use of uninitialized data, as Rust does), Go makes the same "Billion Dollar Mistake" that language designers have been making for decades: http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare http://www.infoq.com/presentations/Null-References-The-Billi...
- mseepgood 13y ago
  And Dijkstra considered goto harmful. He wasn't right either. You can be sure that the Go designers were aware of Hoare's opinion.
  - taliesinb 13y ago
    Hoare only gave this talk in 2009, whereas Go development started in 2007. Did Hoare express this opinion publicly (or privately, to the Go developers) before then? I can't easily find any indication that he did. I can imagine goto makes region analysis hellishly more difficult, and region analysis is what gives Rust the ability to statically reason about reference lifetimes, which makes it practical to remove null references from the language. So on the Rust view, goto is even more antithetical to safe code than on Dijkstra!
    - azth 13y ago
      Indeed. Plus, I have never read a convincing response from the Go authors as to why they included null in the type system.
      - burntsushi 13y ago
        It's been hashed out over and over again on the mailing list. Their stance boils down to language design cohesion. In particular, avoiding null in your type system while preserving orthogonality in your language design would require a completely different language than the on that Go embodies. (This is in stark contrast with say, the Haskell community, which will add any new feature produced by original research.) If you dig through the mailing list posts, avoiding null in the type system can be difficult to reconcile with "zero values", which each type has. More subtlety, avoiding null in your type system might require adding support for sum types, which conflicts in weird ways with Go's interfaces. I think a lot of people tend to assume that null should always be eliminated from type systems because doing so can be done for free. But I disagree that it can be done for free, and my evidence is in the aforementioned mailing list posts. The other reason why I suspect we'll never see the elimination of null in Go's type system is that it just isn't that big of a source of bugs. Anecdotally, I very rarely see my program crash because of a nil pointer error, even while doing active development. I could hypothesize as to why this is, but I'll just leave you with this thought: the type system isn't the only thing that can reduce the occurrence of certain classes of bugs.
stesch 13y ago
Wow, a lot of users of dynamic languages change to Go.
uniclaude 13y ago
I work with Go on a daily basis (and it is my own choice), and I would like to offer a slightly different opinion than most people on this thread. Go is reliable, fast, and its main benefits are simple concurrency, a good, well written standard library, and a blazing fast compiler. Moreover, the Go tour is very nice, and it gives enough material to any programmer to start writing short programs using http, images, channels,... in only a few hours. This said, very often, using Go feels backwards to me. It could be because of I have personal gripes with the package management system, but I easily ignore those cosmetic things when I'm getting stuff done, so it's not. After getting used to type systems like the one Haskell has, Go feels very primitive. Moreover, the lack of generics and support for functional programming makes my programs longer, less readable, and less friendly for teamwork. Even though I still sell Go to my friends as a replacement for most of the scripting languages they use, (Typed) Clojure and Haskell are languages that are way friendlier to me. I would take Go over C or C++ any day, simply because it would allow me to write relatively readable programs in less time, but if the task fits within what Clojure or Haskell does right, I would not hesitate for a second. As of today, I primarily use Go for infrastructure work, and it really shines there. I am going to replace a lot of Python code with it, and it's not necessarily for the performance, but for the ease of deployment, the reliability, the fact that Python developers can read it, and the static typing. That is to say I don't disagree with the author, but I want to provide another point of view. The language I believe will deliver what I hoped with Go is Rust. I do not believe, however that it is ever going to reach the hype level that Go has, but it seems more promising to me.
- twotwotwo 13y ago
  ++to both sides of this, though I think I weight the positives a little more heavily than you do. The stdlib, compiler, and ecosystem stuff like the tour/godoc/playground matter more than they get credit for (the value of any language can't be determined just from its spec), and that hype helps get libraries written, tools improved, etc. I also share the sentiment that there's one set of gripes that are annoying but don't actually prevent getting stuff done (if I'm reading your comment right). I wouldn't mind C++-style const params, some way to briefly relax the (usually quite useful) fatwa against unused names in the heat of writing/testing code, a concise built-in non-close() way to broadcast to everyone waiting on a channel, etc. But, point is, none of that really stops or substantially slows a project down. And Go has other, almost complementary, quirks I like, like expressions at the toplevel or the prohibition on discarding results of certain functions or constants that are just numbers or... I really hope that by Go 2 there's a better way to write containers or functional tools. Right now leading gophers occasionally argue that it'd be bad to extend Go until your code never has to repeat itself, and sure, you can get by with what Go has now. But that all glosses over that there's a substantial cost in expressiveness to some things Go's missing. I'd even accept something like parameterized package imports (import intset "container/set<int>" loads package set<T> substituting T for int). Those would be "Go-y" in that instantiation would have to be explicit and the type system would be unaffected (both unlike C++), though it'd sacrifice some power/brevity. Pike also offhandedly mentioned polymorphic functions in his "Less is exponentially more" post a while back, and I'd take those too. :)

133 points by babawere 13y ago

Summary: At a GoSF meetup audience member asked if Go was reliable... - Op was surprised because he had been using it in production for nearly 3 years. - Previously used it at a Financial Company (wholesaler for institutional share traders and stock brokers). - Op used it previously for system Monitoring, as a data store and messaging middleware. - Go's adoption is gathering pace thanks to the terse syntax, straightforward powerful standard library, excellent tooling and concurrency primitives. [Sic. And performance] - Go's core development team a reason for why it is showing maturity beyond it's years. - Op changed positions so he could work on Go full-time.

carbocation 13y ago
Question: is this an automated summary or are you doing these summaries yourself manually? On topic, it seems odd that a Python process that would take 3 minutes to scan nodes could be reduced to a go script that takes under 1 second. Perhaps this is entirely due to going from serial to parallel, but the timeout thing that he mentioned sounds like an error in the Python code rather than a language problem. And I say this as someone who uses/enjoys go.
- joshbaptiste 13y ago
  Indeed.. I would have liked to hear what took the Python script 3x longer. The longest part I would assume would be the network requests themselves which is inherent regardless of language.
  - eudoxus 13y ago
    its actually 180x faster if it went from 3 mins to less then a second
  - wglb 13y ago
    Sounds like it the execution time required to processing the vast amounts of data produced. go is quite likely to beat python in compute bound or data crunching tasks.
    - Patrick_Devine 13y ago
      Go is definitely faster than Python for those types of tasks, however I'm suspicious of it being that much faster. I've crunched massive data sets in Python and as long as you're aware of its limitations, you can make things pretty zippy.

benbjohnson 13y ago

I wholeheartedly agree with this article. I've had great success with Go's reliability. Another important point to make is that when programs written in Go do fail, it tends to be trivial to figure out why. There's no type hierarchy like most OOP languages and no method_missing magic like Ruby. I can typically just trace the logic backwards from the error to the cause. Most bug reports I receive for my open source Go projects get fixed in minutes.

khyryk 13y ago
I suspect it's Ruby (dynamic typing, syntactic flexibility, etc.) rather than OOP itself that's the root of the issue when it comes to tracing logic backwards.

mitchellh 13y ago

I have both production systems (ircrelay.com) and well-used client software (packer.io) written in Go, so I'd like to comment on this from both perspectives, server-side and desktop-side. First off, both from the get-go have been extremely reliable. IRCRelay has a custom IRC router that is able to route IRC connections to their proper bouncer server, then just stream the contents to and from. This server was written once, and has been running in production for over a year without a single downtime incident or crash incident. Likewise, Packer reliability has been astounding, even to me. It really rarely crashes, and when it does it is because I'm usually skirting around the type system of Go (casting). The PRIMARY reason Go is so reliable is also a reason many people hate go: you _have_ to handle every error. There are no exceptions, you know if a line of code can fail or not, and you have to handle it. This leads to a lot of "if err != nil" checks, and it can be really, really annoying sometimes. But the truth is: once you get that thing into production, it is never going to crash. Never. MAYBE nil slice access, but that's about it. Note the compiler doesn't actively enforce this, but it is Go best practice to handle every error. (Pedantics: yes, you can assign errors to "_" variables to ignore them or just not accept any return values, but this is very actively discouraged in Go and it is one of those you're-doing-it-wrong kind of things) Also: a compiler. Coming from Ruby, having a compiler is just amazing. Obviously compilers have been around forever and Go's isn't special in any way. But after living in a dynamic language heavy environment for many years, I can't imagine going back to not having a compiler. It just makes writing new features, refactoring old ones, etc. all just so easy. It catches all the bone-head mistakes that cause a majority of crashes in code I previously wrote. Another reason: Testing is heavily encouraged and baked right into the official build tool. This makes it so easy to write tests that you always do. It isn't really an opt-in thing, because it is one of those things you just DO if you write Go. This makes it so that even with explicit error checking and the type system, you're fairly certain your logic is reasonably correct as well. And finally, you're statically compiling your applications. Once they're running, no external dependencies can mess that up. Dependency management from _source_ form is a problem with Go, one that is actively acknowledge and being worked on. But once that program is compiled, it is safe forever as long as you run it on a proper kernel. So my experience is anecdotal, but there are real language features and community ideologies at play here that make Go a really stable language to write programs in.

sudonim 13y ago

We've been using Go in production for 4 or so months at customer.io and have been tremendously happy with the performance. My cofounder will rewrite some core function in go, deploy it to prod and get a 10x improvement. We are total converts.

dbaupp 13y ago
10x improvement compared to what?
- jrallison 13y ago
  Our MVP was written in Ruby.
  - danieldk 13y ago
    You know that nearly any compiled language would give such speed-ups compared to Ruby?
    - pjmlp 13y ago
      Maybe even Turbo Basic would do the trick! :)
    - jrallison 13y ago
      Of course, which was the main driver to moving to a compiled language. We just happened to choose Go :) I've done a lot of Java and C++ in the past, and wasn't looking to go back to either of those.

th0br0 13y ago

The comments thread on this makes me sort of sad. So... Go rocks because a) you've got to handle all the errors b) there is no method_missing (or its ilk) In general, all those features have been offered by compiled languages for years (even if they might've just been compiler warnings). What makes Go so special other than the lack of deep class hierarchies? Sure, it's more "reliable" by design than 100% of those dynamic languages out there, but that doesn't really make the language more attractive. Coming from an OO world, I find Go's "inheritance" model far inferior to what, e.g., Haskell or Scala offer. Given the comments here I think that I can finally understand somewhat why Go is taking off the way it is, but it feels like a faulty "solution".

khyryk 13y ago
a isn't necessarily the case. Consider fmt.Println: func Println(a ...interface{}) (n int, err error) I've never seen any code that handled err -- I don't mean ignoring it with _, I mean not even acknowledging its existence -- without the compiler saying anything.
- dilap 13y ago
  Yeah, but if print fails, what are you going to do? And it's not like ignoring this error is going to make your code blow up.
  - khyryk 13y ago
    The point is that I can't immediately tell which errors will be mysteriously ignored. This, together with the existence of panics, undermines the stance that one can tell exactly what a certain piece of Go code will do just by looking at, say, a function prototype.
    - cmccabe 13y ago
      Don't be fatuous. If you wrote all of the code, you should be able to tell exactly what it does. If there's some code you're calling that was written by someone else, you have to trust it to do the right thing. Exceptions versus return codes doesn't change that fact. People usually don't check the return code of fmt.Printf because they simply don't care. Most of the time, you're just logging to a terminal, which basically never fails. And if logging did fail, you wouldn't want to abort the program because of it. The only time you might want to check the return value of printf is if you're writing a simple utility designed to transform something passed on stdin to something on stdout.

petercooper 13y ago

I'd seen a couple of references to Go being used by financial companies in London (both for prototyping and in live systems) and this post seems to provide yet more confirmation. Is finance in particular looking to be a forthcoming hot market for Go due to its balance of performance and developer friendliness?

ShabbyDoo 13y ago

What's notable is that no one has yet posted here saying, "Go stunk for me. Perf and reliability were both awful. I went back to blahblah and threw away all my Go code." It's usually easy to find detractors of any technology.

dsymonds 13y ago
I think that's because Go is reasonably good for, say, 90% of situations, and if you're in the 10% where it is not a good fit then you already know that and you know better than to try using Go in that situation, and thus few people find Go stinking. On the other side is things like Python, which are reasonably good for a much lower fraction of things (e.g. 30%), but if you're in the 70% you very well might not realise it does not suit your situation and thus experience Python stinking.
graue 13y ago
I actually have seen complaints about Go's performance relative to other compiled languages, like C++, D and Java. I don't think it's coincidental that most of the Go converts come from a dynamic-language background. If you're porting from Ruby, 2-3x slower than Java still seems blazing fast. For me, Go misses the sweet spot entirely. A year ago I was curious about both Go and Clojure, my interest piqued by the strong concurrency support each has. I picked Clojure and have grown more and more confident in that choice. Why not Go? In a language with nil pointers (Tony Hoare's "billion-dollar mistake"), no generics, and sub-JVM performance, static typing just doesn't seem worth it. Also, poor interop with other languages means Go can only draw from its own small community for libraries (contrast Clojure which makes using mature, well-tested Java libs trivial). When Rust is more mature, it may be what I hoped Go would be: the safe language, with lightweight threads, that makes me never have to write C or C++ again.
- ruggeri 13y ago
  Go didn't stick for me either. Go is interesting and I really enjoyed channels, but no generic programming really disappointed me. I guess supporting the generic paradigm is pretty tricky (C++/Java templates/generics aren't dear to my heart), but not being able to implement map/filter/reduce functions...

trimbo 13y ago

> A market maker is basically a wholesaler for institutional share traders and stock brokers. During my time there I replaced several key systems components with Go. I'd like to hear more about how you deal with Go's garbage collection in a real time system like this.

burntsushi 13y ago
He may not have had to deal with it at all. He replaced certain components of it with Go code, but none of them may have required real time guarantees.
- trimbo 13y ago
  In later posts, he refers to doing order books and such: http://andrewwdeane.blogspot.co.uk/2013/09/bullhorn.html http://andrewwdeane.blogspot.co.uk/2013/09/bullhorn.html

pauldix 13y ago

At Errplane we've written our primary datastore in Go and have been running it in production since January. It has been totally reliable, performant, and easy to work with. Easily handles the 100m requests per day we get. At this point it's my language of choice for any back end systems.

alexk 13y ago
Would love to know more about the datastore that you've built. Any chance you would elaborate on this?
- pauldix 13y ago
  Sure! It's really just a Go wrapper around LevelDB. LevelDB is an in process database (like SQLite) written in C++. It's a key value store where the keyspace is ordered (so you can do range scans). However, we've written enough code around it to turn it into a scalable distributed time series database with a pubsub layer built in. It does rollups on the fly or can preemtively downsample data coming in. We're actually working on a new version of it that we intend to open source at the end of the year.
  - alexk 13y ago
    That'd be awesome if you open sourced this!
  - gigq 13y ago
    That sounds pretty useful, something I would love to see open sourced as I've thought about making a distributed LevelDB based DB as well. Any word on how it handles the distribution? Are we talking a quorum based system like Cassandra/Dynamo or are we talking more like a master / key range type system like HBase.
    - pauldix 13y ago
      The version we're working on now uses the go Raft implementation to handle cluster configuration. Then each server periodically (< 30s) performs read repairs to ensure that it has the latest writes for its spots in the hash ring. Writes can either be done via either write once or quorum, with write once being the most often used thing. Since most of the time this data is in aggregate, we don't care about losing a few writes.

taliesinb 13y ago

Unfortunately, by insisting that all data start in a zeroed state (as opposed to forbidding the use of uninitialized data, as Rust does), Go makes the same "Billion Dollar Mistake" that language designers have been making for decades: http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare http://www.infoq.com/presentations/Null-References-The-Billi...

mseepgood 13y ago
And Dijkstra considered goto harmful. He wasn't right either. You can be sure that the Go designers were aware of Hoare's opinion.
- taliesinb 13y ago
  Hoare only gave this talk in 2009, whereas Go development started in 2007. Did Hoare express this opinion publicly (or privately, to the Go developers) before then? I can't easily find any indication that he did. I can imagine goto makes region analysis hellishly more difficult, and region analysis is what gives Rust the ability to statically reason about reference lifetimes, which makes it practical to remove null references from the language. So on the Rust view, goto is even more antithetical to safe code than on Dijkstra!
  - azth 13y ago
    Indeed. Plus, I have never read a convincing response from the Go authors as to why they included null in the type system.
    - burntsushi 13y ago
      It's been hashed out over and over again on the mailing list. Their stance boils down to language design cohesion. In particular, avoiding null in your type system while preserving orthogonality in your language design would require a completely different language than the on that Go embodies. (This is in stark contrast with say, the Haskell community, which will add any new feature produced by original research.) If you dig through the mailing list posts, avoiding null in the type system can be difficult to reconcile with "zero values", which each type has. More subtlety, avoiding null in your type system might require adding support for sum types, which conflicts in weird ways with Go's interfaces. I think a lot of people tend to assume that null should always be eliminated from type systems because doing so can be done for free. But I disagree that it can be done for free, and my evidence is in the aforementioned mailing list posts. The other reason why I suspect we'll never see the elimination of null in Go's type system is that it just isn't that big of a source of bugs. Anecdotally, I very rarely see my program crash because of a nil pointer error, even while doing active development. I could hypothesize as to why this is, but I'll just leave you with this thought: the type system isn't the only thing that can reduce the occurrence of certain classes of bugs.

stesch 13y ago

Wow, a lot of users of dynamic languages change to Go.

uniclaude 13y ago

I work with Go on a daily basis (and it is my own choice), and I would like to offer a slightly different opinion than most people on this thread. Go is reliable, fast, and its main benefits are simple concurrency, a good, well written standard library, and a blazing fast compiler. Moreover, the Go tour is very nice, and it gives enough material to any programmer to start writing short programs using http, images, channels,... in only a few hours. This said, very often, using Go feels backwards to me. It could be because of I have personal gripes with the package management system, but I easily ignore those cosmetic things when I'm getting stuff done, so it's not. After getting used to type systems like the one Haskell has, Go feels very primitive. Moreover, the lack of generics and support for functional programming makes my programs longer, less readable, and less friendly for teamwork. Even though I still sell Go to my friends as a replacement for most of the scripting languages they use, (Typed) Clojure and Haskell are languages that are way friendlier to me. I would take Go over C or C++ any day, simply because it would allow me to write relatively readable programs in less time, but if the task fits within what Clojure or Haskell does right, I would not hesitate for a second. As of today, I primarily use Go for infrastructure work, and it really shines there. I am going to replace a lot of Python code with it, and it's not necessarily for the performance, but for the ease of deployment, the reliability, the fact that Python developers can read it, and the static typing. That is to say I don't disagree with the author, but I want to provide another point of view. The language I believe will deliver what I hoped with Go is Rust. I do not believe, however that it is ever going to reach the hype level that Go has, but it seems more promising to me.

twotwotwo 13y ago
++to both sides of this, though I think I weight the positives a little more heavily than you do. The stdlib, compiler, and ecosystem stuff like the tour/godoc/playground matter more than they get credit for (the value of any language can't be determined just from its spec), and that hype helps get libraries written, tools improved, etc. I also share the sentiment that there's one set of gripes that are annoying but don't actually prevent getting stuff done (if I'm reading your comment right). I wouldn't mind C++-style const params, some way to briefly relax the (usually quite useful) fatwa against unused names in the heat of writing/testing code, a concise built-in non-close() way to broadcast to everyone waiting on a channel, etc. But, point is, none of that really stops or substantially slows a project down. And Go has other, almost complementary, quirks I like, like expressions at the toplevel or the prohibition on discarding results of certain functions or constants that are just numbers or... I really hope that by Go 2 there's a better way to write containers or functional tools. Right now leading gophers occasionally argue that it'd be bad to extend Go until your code never has to repeat itself, and sure, you can get by with what Go has now. But that all glosses over that there's a substantial cost in expressiveness to some things Go's missing. I'd even accept something like parameterized package imports (import intset "container/set<int>" loads package set<T> substituting T for int). Those would be "Go-y" in that instantiation would have to be explicit and the type system would be unaffected (both unlike C++), though it'd sacrifice some power/brevity. Pike also offhandedly mentioned polymorphic functions in his "Less is exponentially more" post a while back, and I'd take those too. :)