4 ms·

Open-Source Go Libraries

81 points by jtolds 12y ago

chimeracoder 12y ago
> While Go has very deliberately not implemented class hierarchies, a quick perusal of Go's net and os packages should indicate that sometimes error hierarchies are useful. I just want to point out - I would hesitate from drawing too many conclusions from "net" and "os" (the latter in particular). Not only is "os" some of the oldest Go code, but it is also designed to wrap conveniently around external interfaces that follow very different idioms. Another example would be the constant declarations[0], which use ALL_CAPS - this is actually not idiomatic in Go, but it is used here (a) because it was done very early on, and (b) because it more closely matches the C libraries it interfaces with[1]. I need to take a closer look at this errors package before drawing an opinion about the rest of it, but this sentence caught my attention. [0] http://golang.org/pkg/os/#pkg-constants http://golang.org/pkg/os/#pkg-constants [1] https://github.com/golang/lint/issues/32 https://github.com/golang/lint/issues/32
tptacek 12y ago
Adam Langley will disagree with me on this point, but if you were a client and put a choice of Golang crypto/tls and OpenSSL in front of me, I'd professionally recommend crypto/tls. This doesn't come up that much, because most production-scale deployments terminate TLS at a cluster of load balancers, so you don't really get to choose which TLS library you use in prod. But if you have the option, I get warmer fuzzies from crypto/tls than from native bindings to the scariest library in cryptography. This could change in a year or so when the forks settle out, but I hope not, because crypto/tls is a very nice piece of code.
- jtolds 12y ago
  Totally. I'd love to see crypto/tls fixed. Until then, though, I'd rather not choose the option with known, existing attack vectors. Too bad it took such a horrible bug to shake everyone into auditing these libraries. Edit: fwiw, the hw acceleration is a pretty big deal for us. We have little ARM devices.
  - tptacek 12y ago
    Are you working from the presumption that Golang crypto/tls admits practical side-channel attacks and OpenSSL doesn't? I would question that assumption. Again: if I had to make a recommendation between the two, then from a security perspective I'd lean crypto/tls. Regardless, if I was stuck on hardware acceleration, I'd probably stick a reverse proxy in front of my Golang code before I incorporated an OpenSSL Golang binding.
    - jtolds 12y ago
      Having a separate process for SSL termination is of course completely doable, but was a little more challenging. We use client-side certificates for application level authentication logic, so we would have needed to side-channel the authentication information from our reverse proxy to our app. Again, completely, totally doable, was just harder for our transliteration project [1]. Good advice. [1] https://www-staging.spacemonkey.com/blog/posts/go-space-monkey https://www-staging.spacemonkey.com/blog/posts/go-space-monk...
- axaxs 12y ago
  The problem seems to be that very few people understand timing attacks, and even fewer are capable or willing to fix it. This is evidenced by the fact I've seen this same argument made three or more times now, and all link to a completely technically void twitter post. Not that AGL isn't a brilliant man, mind you, but it seems people just link to the post with no real knowledge of whether the problem exists or what it means. I like go and have contributed where I can, but like most, my time is limited. You're a respected member of this community with a respect for go. Let's say we work to fix this by 1.4. Are you willing?
Xorlev 12y ago
Nice set of libraries. I wonder if they looked at log4go before writing their own logger, and if so what drove them to do so.
- ominous_prime 12y ago
  is there an active fork of log4go? The original hasn't been updated in 2 years, with a couple serious issues which have been open at least as long. I would also love a flexible logging package (that isn't glog ;).
  - mratzloff 12y ago
    I use seelog (https://github.com/cihub/seelog https://github.com/cihub/seelog) and have been pretty happy with it.
kbd 12y ago
In their errors library - http://godoc.org/github.com/SpaceMonkeyGo/errors http://godoc.org/github.com/SpaceMonkeyGo/errors - they write: > It doesn't take long during Go development before you may find yourself wondering where an error came from. In other languages, as soon as an error is raised, a stack trace is captured and is displayed as part of the language's error handling. Go error types are simply basic values, and no such magic happens to tell you what line or what stack an error came from. I was thinking the other day that Go's "if err != nil { return err }" is just the explicit version of what's done implicitly with exceptions, except the 'err' doesn't remember where it came from. This lack of information seems like a pretty big inconvenience. How do Go developers typically deal with this problem?
- nknighthb 12y ago
  I'm not sure what kind of inconvenience it seems like. If you do extensive logging (which you should), and you set Llongfile or Lshortfile (which you should), you will at least know where the error bubbled up into your own code. And you can always get a stacktrace from runtime/debug.PrintStack(), if you want, though I've only had real cause to use it once or twice. (In the event of a panic, you do automatically get a stack trace, though most of the time when that happens I barely have cause to glance at it.) Idiomatic Go is not so deeply layered as some Java stacks I've seen. It is more horizontal than vertical. If you call a function in somebody else's library and get an error back, the source of the error is probably not ten layers further down.
- grey-area 12y ago
  If you're logging errors, you can use the runtime package to find out where a logging statement was called. So you can log on error and record the line, function name etc: pc, _, line, _ := runtime.Caller(1) name := runtime.FuncForPC(pc).Name() Not that you need this much, but it is there if you need to return information on exactly where an error came from when logged, just put that in your logging function and whenever you log it'll trace where the log was called from, or you could store the info in your error object if you wish. log4go has some examples of this sort of thing I think. Typically I've only used that when there was an intractable bug though, normally you're not going to need it if your error objects are specific enough, it's easy to find where they came from and store whatever info you want in them, which I assume is what they're doing here.
tagrun 12y ago
> Go TLS does not support hardware accelerated crypto. Go uses AES-NI instructions on amd64 when available http://tip.golang.org/src/pkg/crypto/aes/asm_amd64.s http://tip.golang.org/src/pkg/crypto/aes/asm_amd64.s > Go TLS does not have wide support for backwards compatibility with less secure and older protocol versions (a good thing, until you have no other option). LibreSSL people beg to differ about this "feature". > Go TLS does not give much control over the certificate validation process, which makes it difficult or impossible to add additional checks and validations. What exactly is this about? > Oh, and Go TLS is vulnerable to timing attacks. A mitigation is in WIP https://codereview.appspot.com/94850043 https://codereview.appspot.com/94850043
- jtolds 12y ago
  AES-NI doesn't work on ARM, which is what we have. >> Go TLS does not have wide support for backwards compatibility with less secure and older protocol versions (a good thing, until you have no other option). > LibreSSL people beg to differ about this "feature". Agreed agreed! Don't use old protocols! >> Go TLS does not give much control over the certificate validation process, which makes it difficult or impossible to add additional checks and validations. > What exactly is this about? OpenSSL allows you to specify your own validation callback, to add additional checks to the certificate validation step. The only certificate validation option Go's TLS gives you is either InsecureSkipVerify or not. >> Oh, and Go TLS is vulnerable to timing attacks. > A mitigation is in WIP https://codereview.appspot.com/94850043 https://codereview.appspot.com/94850043 awesome!
  - tagrun 12y ago
    > AES-NI doesn't work on ARM, which is what we have. I'm afraid this falls into scope of the low-priority issue 4299 then: https://code.google.com/p/go/issues/detail?id=4299 https://code.google.com/p/go/issues/detail?id=4299 > OpenSSL allows you to specify your own validation callback, to add additional checks to the certificate validation step. The only certificate validation option Go's TLS gives you is either InsecureSkipVerify or not. I'm not sure about the necessity or the details of the particular use case, but if it is general enough, you can file an issue.

81 points by jtolds 12y ago

> While Go has very deliberately not implemented class hierarchies, a quick perusal of Go's net and os packages should indicate that sometimes error hierarchies are useful. I just want to point out - I would hesitate from drawing too many conclusions from "net" and "os" (the latter in particular). Not only is "os" some of the oldest Go code, but it is also designed to wrap conveniently around external interfaces that follow very different idioms. Another example would be the constant declarations[0], which use ALL_CAPS - this is actually not idiomatic in Go, but it is used here (a) because it was done very early on, and (b) because it more closely matches the C libraries it interfaces with[1]. I need to take a closer look at this errors package before drawing an opinion about the rest of it, but this sentence caught my attention. [0] http://golang.org/pkg/os/#pkg-constants http://golang.org/pkg/os/#pkg-constants [1] https://github.com/golang/lint/issues/32 https://github.com/golang/lint/issues/32

tptacek 12y ago

Adam Langley will disagree with me on this point, but if you were a client and put a choice of Golang crypto/tls and OpenSSL in front of me, I'd professionally recommend crypto/tls. This doesn't come up that much, because most production-scale deployments terminate TLS at a cluster of load balancers, so you don't really get to choose which TLS library you use in prod. But if you have the option, I get warmer fuzzies from crypto/tls than from native bindings to the scariest library in cryptography. This could change in a year or so when the forks settle out, but I hope not, because crypto/tls is a very nice piece of code.

jtolds 12y ago
Totally. I'd love to see crypto/tls fixed. Until then, though, I'd rather not choose the option with known, existing attack vectors. Too bad it took such a horrible bug to shake everyone into auditing these libraries. Edit: fwiw, the hw acceleration is a pretty big deal for us. We have little ARM devices.
- tptacek 12y ago
  Are you working from the presumption that Golang crypto/tls admits practical side-channel attacks and OpenSSL doesn't? I would question that assumption. Again: if I had to make a recommendation between the two, then from a security perspective I'd lean crypto/tls. Regardless, if I was stuck on hardware acceleration, I'd probably stick a reverse proxy in front of my Golang code before I incorporated an OpenSSL Golang binding.
  - jtolds 12y ago
    Having a separate process for SSL termination is of course completely doable, but was a little more challenging. We use client-side certificates for application level authentication logic, so we would have needed to side-channel the authentication information from our reverse proxy to our app. Again, completely, totally doable, was just harder for our transliteration project [1]. Good advice. [1] https://www-staging.spacemonkey.com/blog/posts/go-space-monkey https://www-staging.spacemonkey.com/blog/posts/go-space-monk...
axaxs 12y ago
The problem seems to be that very few people understand timing attacks, and even fewer are capable or willing to fix it. This is evidenced by the fact I've seen this same argument made three or more times now, and all link to a completely technically void twitter post. Not that AGL isn't a brilliant man, mind you, but it seems people just link to the post with no real knowledge of whether the problem exists or what it means. I like go and have contributed where I can, but like most, my time is limited. You're a respected member of this community with a respect for go. Let's say we work to fix this by 1.4. Are you willing?

Xorlev 12y ago

Nice set of libraries. I wonder if they looked at log4go before writing their own logger, and if so what drove them to do so.

ominous_prime 12y ago
is there an active fork of log4go? The original hasn't been updated in 2 years, with a couple serious issues which have been open at least as long. I would also love a flexible logging package (that isn't glog ;).
- mratzloff 12y ago
  I use seelog (https://github.com/cihub/seelog https://github.com/cihub/seelog) and have been pretty happy with it.

kbd 12y ago

In their errors library - http://godoc.org/github.com/SpaceMonkeyGo/errors http://godoc.org/github.com/SpaceMonkeyGo/errors - they write: > It doesn't take long during Go development before you may find yourself wondering where an error came from. In other languages, as soon as an error is raised, a stack trace is captured and is displayed as part of the language's error handling. Go error types are simply basic values, and no such magic happens to tell you what line or what stack an error came from. I was thinking the other day that Go's "if err != nil { return err }" is just the explicit version of what's done implicitly with exceptions, except the 'err' doesn't remember where it came from. This lack of information seems like a pretty big inconvenience. How do Go developers typically deal with this problem?

nknighthb 12y ago
I'm not sure what kind of inconvenience it seems like. If you do extensive logging (which you should), and you set Llongfile or Lshortfile (which you should), you will at least know where the error bubbled up into your own code. And you can always get a stacktrace from runtime/debug.PrintStack(), if you want, though I've only had real cause to use it once or twice. (In the event of a panic, you do automatically get a stack trace, though most of the time when that happens I barely have cause to glance at it.) Idiomatic Go is not so deeply layered as some Java stacks I've seen. It is more horizontal than vertical. If you call a function in somebody else's library and get an error back, the source of the error is probably not ten layers further down.
grey-area 12y ago
If you're logging errors, you can use the runtime package to find out where a logging statement was called. So you can log on error and record the line, function name etc: pc, _, line, _ := runtime.Caller(1) name := runtime.FuncForPC(pc).Name() Not that you need this much, but it is there if you need to return information on exactly where an error came from when logged, just put that in your logging function and whenever you log it'll trace where the log was called from, or you could store the info in your error object if you wish. log4go has some examples of this sort of thing I think. Typically I've only used that when there was an intractable bug though, normally you're not going to need it if your error objects are specific enough, it's easy to find where they came from and store whatever info you want in them, which I assume is what they're doing here.

tagrun 12y ago

> Go TLS does not support hardware accelerated crypto. Go uses AES-NI instructions on amd64 when available http://tip.golang.org/src/pkg/crypto/aes/asm_amd64.s http://tip.golang.org/src/pkg/crypto/aes/asm_amd64.s > Go TLS does not have wide support for backwards compatibility with less secure and older protocol versions (a good thing, until you have no other option). LibreSSL people beg to differ about this "feature". > Go TLS does not give much control over the certificate validation process, which makes it difficult or impossible to add additional checks and validations. What exactly is this about? > Oh, and Go TLS is vulnerable to timing attacks. A mitigation is in WIP https://codereview.appspot.com/94850043 https://codereview.appspot.com/94850043

jtolds 12y ago
AES-NI doesn't work on ARM, which is what we have. >> Go TLS does not have wide support for backwards compatibility with less secure and older protocol versions (a good thing, until you have no other option). > LibreSSL people beg to differ about this "feature". Agreed agreed! Don't use old protocols! >> Go TLS does not give much control over the certificate validation process, which makes it difficult or impossible to add additional checks and validations. > What exactly is this about? OpenSSL allows you to specify your own validation callback, to add additional checks to the certificate validation step. The only certificate validation option Go's TLS gives you is either InsecureSkipVerify or not. >> Oh, and Go TLS is vulnerable to timing attacks. > A mitigation is in WIP https://codereview.appspot.com/94850043 https://codereview.appspot.com/94850043 awesome!
- tagrun 12y ago
  > AES-NI doesn't work on ARM, which is what we have. I'm afraid this falls into scope of the low-priority issue 4299 then: https://code.google.com/p/go/issues/detail?id=4299 https://code.google.com/p/go/issues/detail?id=4299 > OpenSSL allows you to specify your own validation callback, to add additional checks to the certificate validation step. The only certificate validation option Go's TLS gives you is either InsecureSkipVerify or not. I'm not sure about the necessity or the details of the particular use case, but if it is general enough, you can file an issue.