5 ms·

Tagged Logging in Go

66 points by GarethX 12y ago

radimm 12y ago
similar structured logger https://github.com/Sirupsen/logrus https://github.com/Sirupsen/logrus
- pquerna 12y ago
  I've used `logrus` in a couple projects and been pretty happy with it. Easy to get rolling, easy to make chained loggers, and can do JSON in production or pretty colors when in dev environment.
  - wtf_is_up 12y ago
    The hooks are nice too. I'm using the Slack hook to push notifications. Works like a charm.
  - girishso 12y ago
    Ditto for me. I have created a desktop app and use 'logrus' for logging locally and have created a 'hook' to send logs to the server. Works pretty neat!
- rakoo 12y ago
  It seems to me FogCreek's tagged logging is a simpler version of logging than what logrus provides and what should actually always be the case: you don't log strings, you log structs with multiple fields and possibly multiple values for each field. It's much more versatile and seems to be enough for most if not all use cases, as logstash shows (really, at some points it starts being much easier to just log a JSON dict and stop fiddling with fancy regexps for analytics)
- deleted 12y ago
  [deleted]
- kiyoto 12y ago
  Interesting. This can be used with Fluentd as well with the JSONFormatter (https://www.fluentd.org https://www.fluentd.org)
tveita 12y ago
SLF4J has two ways to do "tagged logging": markers that are passed per-call, and a thread-local mapped diagnostic context (MDC). The latter is useful because you can set it for an entire request and have it included even for code that isn't explicitly aware of e.g. accounts. Unfortunately Go doesn't have thread-local storage, so that feature may not be feasable. (Except through ugly hacks: https://github.com/jtolds/gls https://github.com/jtolds/gls)
jzelinskie 12y ago
Wouldn't the signature of the tagged logging function be better if it was something like: DebugWithTags(message string, tags ...string) This way you could easily add tags without having to explicitly allocate a []string whenever you wanted to change them.
- pythonist 12y ago
  +1 :)
- fizzbatter 12y ago
  Perhaps it depends on how much content you're putting into the message, normally? Eg, big multi-line strings, followed by more args at the end, is not a syntax i personally prefer. With that said, i really do wish Go supported `DebugWithTags(tags ...string, message string)` syntax. I've had so many cases where i wanted to use something similar.
  - kornish 12y ago
    For the record, if the last argument of a Golang function is variadic, you can still pass an array to it - you just have to transform the array into variadic args by appending `...` to it. So adopting the type signature above could give you the best of both worlds - the ability to use one-off tag combinations inline, but also the ability to pass predefined arrays of tags.
  - fizzbatter 12y ago
    I hadn't actually seen their Code, it looks like they actually use `DebugWithTags([]string, string, ...interface{})`. So the `...interface{}` is the blocker here, as it appears in every logger that they have. Though, i would agree that the order should be switched. Eg, `DebugWithTags(string, []string, ...interface{})`
- blakecaldwell 12y ago
  It's more consistent with the printf-style counterparts: func (logger *LoggerImpl) DebugWithTagsf(tags []string, fmt string, args ...interface{}) { logger.logwithformat(DEBUG, tags, fmt, args...) }
Omie6541 12y ago
Please correct me if I have got this wrong or missed some code piece but you are pushing log records to a buffered channel (incomingChannel) without making sure that it has some space left. When that buffer of 2048 gets full, all the next requests are going to get blocked. Even if you call .log() in the background, there will be number of goroutines waiting to get their turn which will lead the system to a state where you have no idea what is pending and where. here's a smaller version of the problem, notice how fillFoo() will have to wait to push to the buffer. https://gist.github.com/Omie/52add99f6685dcb340a1 https://gist.github.com/Omie/52add99f6685dcb340a1 there's also a similar problem (how they solved it) described here: http://pivotallabs.com/a-concurrent-ring-buffer-for-go/ http://pivotallabs.com/a-concurrent-ring-buffer-for-go/
inconshreveable 12y ago
Did you evaluate log15? And if so, what did you find lacking about it? https://godoc.org/gopkg.in/inconshreveable/log15.v2 https://godoc.org/gopkg.in/inconshreveable/log15.v2 https://github.com/inconshreveable/log15 https://github.com/inconshreveable/log15

66 points by GarethX 12y ago

radimm 12y ago

similar structured logger https://github.com/Sirupsen/logrus https://github.com/Sirupsen/logrus

pquerna 12y ago
I've used `logrus` in a couple projects and been pretty happy with it. Easy to get rolling, easy to make chained loggers, and can do JSON in production or pretty colors when in dev environment.
- wtf_is_up 12y ago
  The hooks are nice too. I'm using the Slack hook to push notifications. Works like a charm.
- girishso 12y ago
  Ditto for me. I have created a desktop app and use 'logrus' for logging locally and have created a 'hook' to send logs to the server. Works pretty neat!
rakoo 12y ago
It seems to me FogCreek's tagged logging is a simpler version of logging than what logrus provides and what should actually always be the case: you don't log strings, you log structs with multiple fields and possibly multiple values for each field. It's much more versatile and seems to be enough for most if not all use cases, as logstash shows (really, at some points it starts being much easier to just log a JSON dict and stop fiddling with fancy regexps for analytics)
deleted 12y ago
[deleted]
kiyoto 12y ago
Interesting. This can be used with Fluentd as well with the JSONFormatter (https://www.fluentd.org https://www.fluentd.org)

tveita 12y ago

SLF4J has two ways to do "tagged logging": markers that are passed per-call, and a thread-local mapped diagnostic context (MDC). The latter is useful because you can set it for an entire request and have it included even for code that isn't explicitly aware of e.g. accounts. Unfortunately Go doesn't have thread-local storage, so that feature may not be feasable. (Except through ugly hacks: https://github.com/jtolds/gls https://github.com/jtolds/gls)

jzelinskie 12y ago

Wouldn't the signature of the tagged logging function be better if it was something like: DebugWithTags(message string, tags ...string) This way you could easily add tags without having to explicitly allocate a []string whenever you wanted to change them.

pythonist 12y ago
+1 :)
fizzbatter 12y ago
Perhaps it depends on how much content you're putting into the message, normally? Eg, big multi-line strings, followed by more args at the end, is not a syntax i personally prefer. With that said, i really do wish Go supported `DebugWithTags(tags ...string, message string)` syntax. I've had so many cases where i wanted to use something similar.
- kornish 12y ago
  For the record, if the last argument of a Golang function is variadic, you can still pass an array to it - you just have to transform the array into variadic args by appending `...` to it. So adopting the type signature above could give you the best of both worlds - the ability to use one-off tag combinations inline, but also the ability to pass predefined arrays of tags.
- fizzbatter 12y ago
  I hadn't actually seen their Code, it looks like they actually use `DebugWithTags([]string, string, ...interface{})`. So the `...interface{}` is the blocker here, as it appears in every logger that they have. Though, i would agree that the order should be switched. Eg, `DebugWithTags(string, []string, ...interface{})`
blakecaldwell 12y ago
It's more consistent with the printf-style counterparts: func (logger *LoggerImpl) DebugWithTagsf(tags []string, fmt string, args ...interface{}) { logger.logwithformat(DEBUG, tags, fmt, args...) }

Omie6541 12y ago

Please correct me if I have got this wrong or missed some code piece but you are pushing log records to a buffered channel (incomingChannel) without making sure that it has some space left. When that buffer of 2048 gets full, all the next requests are going to get blocked. Even if you call .log() in the background, there will be number of goroutines waiting to get their turn which will lead the system to a state where you have no idea what is pending and where. here's a smaller version of the problem, notice how fillFoo() will have to wait to push to the buffer. https://gist.github.com/Omie/52add99f6685dcb340a1 https://gist.github.com/Omie/52add99f6685dcb340a1 there's also a similar problem (how they solved it) described here: http://pivotallabs.com/a-concurrent-ring-buffer-for-go/ http://pivotallabs.com/a-concurrent-ring-buffer-for-go/

inconshreveable 12y ago

Did you evaluate log15? And if so, what did you find lacking about it? https://godoc.org/gopkg.in/inconshreveable/log15.v2 https://godoc.org/gopkg.in/inconshreveable/log15.v2 https://github.com/inconshreveable/log15 https://github.com/inconshreveable/log15