4 ms·

Every time I come across a logging package, I don't understand what the point of them are. Finally in the linked issue, I see this: > For many programs that ar

Every time I come across a logging package, I don't understand what the point of them are. Finally in the linked issue, I see this: > For many programs that aren't run directly by person, like servers, logging is the main way for developers to observe the detailed behavior of the system, and often the first place they go to debug it. and it makes sense. I don't deal with servers at all, other than a couple I have made just to host some basic sites or programs locally. I think Go team might be overlooking the fact that a big chunk of the userbase doesnt understand, and has no need for a logging package. Couple that with the giant API proposed, along with a poor design decision (kvs ...any), and the large amount of bikeshedding that already happened in this issue, I really hope they think hard before just shoving this into "std".

atsjie 4y ago
The majority of Golang devs actually do work on web services, API's, or libs/frameworks that would benefit from a standardized structured logging interface. Source: https://go.dev/blog/survey2022-q2-results https://go.dev/blog/survey2022-q2-results (scroll almost to the bottom) For the audience of this proposal it's probably fairly evident what the advantages of standardization would be. Also you argue quite strongly about the poor API design decisions being made; while you similarly admit not having worked with such tools or with web services in Golang general. That's odd.
- svnpenn 4y ago
  I don't work with servers much, but I do have extensive experience in Go. I know a bad API when I see one. strings.Replacer [1] uses the same syntax, and several others have commented in the discussion about it being a bad idea. 1. https://godocs.io/strings#Replacer https://godocs.io/strings#Replacer
  - xani_ 4y ago
    From what I've skimmed the most problems seem to be "golang's type system won't allow expressing all of that on compile time so it will be costly" and I don't think that can be sensibly fixed in logging package > I know a bad API when I see one. strings.Replacer [1] uses the same syntax, and several others have commented in the discussion about it being a bad idea. What's wrong with it ? Never used it. But I think the complaints has been about wasting time for runtime type handling and Replacer doesn't have that problem.
    - Joker_vD 4y ago
      The main performance problem is with "kvs ...any" parameter in the Logger's functions because it allocates lots of stuff: the array itself, and then the wrappers around the values. I've tried to use something like this in a central loop of a CPU-emulator to trace every instruction executed and IIRC it slowed things about 3x times down. The only thing you can do to escape this cost is if logging.Level() >= logging.DEBUG { logging.Logger.Debug(...) } but it's ugly and can't be abstracted away. And yes, I've put several of such guards in the several hot paths of some actual production code base and seen about 25% speed up after that edit.
    - jen20 4y ago
      > What's wrong with it ? It relies on lining up pairs of values in a way that cannot be checked by the compiler rather than having dedicated functions that ensure they are kept together. This is the fundamental difference between bad logging libraries (like klog, hclog etc) and good logging libraries (like zerolog).
      - xani_ 4y ago
        I guess simplest fix for that particular one would be accepting `map[string]string` but that won't exactly work for the any type, and you can't easily add more. Maybe `...map[string]interface{}` but we're back to "the overhead is kinda shit"
kaz-inc 4y ago
I also didn't understand the need for a logging package for a while, but I think it makes sense for go because they're focusing on an audience that writes server applications.
- xani_ 4y ago
  It's less about package and more about interface. One "good enough" in stdlib means libraries could just require passing a interface matching it. Then no matter which logging package you use, as long as it fulfils the interface of stdlib logger it can be used with any library or other component you pulled. So with simple app you can just have simple logger that dumps everything on stderr, complex app can have whatever complex logging logic it needs, but lib "just using stdlib logger" doesn't care about that. And as a lib author, again, you can just use the stdlib one and most likely cover all your use cases.
xani_ 4y ago
Develop bigger app and you will understand. But one example - having an logging interface that is not shit (the current builtin is so garbage it might as well not exist at all) allows for packages to adopt it as default method and you magically can just fed same logger to every dep you pull with no loss of features or having to write wrapper. Contexted logger means you can just fed ["plugin":pluginname] tagged one to every plugin and get logs pointing you to the place where it happened. The structured logging part is mostly to avoid woes of then having to parse it back to structured format via bunch of regexpes that will inevitably break next release. If you still don't get it, here is a regexp to parse HAProxy logs: .*haproxy\[(\d+)]: (.+?):(\d+) \[(.+?)\] (.+?)(|[\~]) (.+?)\/(.+?) ([\-\d]+)\/([\-\d]+)\/([\-\d]+)\/([\-\d]+)\/([\-\d]+) ([\-\d]+) ([\-\d]+) (\S+) (\S+) (\S)(\S)(\S)(\S) ([\-\d]+)\/([\-\d]+)\/([\-\d]+)\/([\-\d]+)\/([\-\d]+) ([\-\d]+)\/([\-\d]+)(| \{.*\}) (".*)([\n|\s]*?)$`) > I think Go team might be overlooking the fact that a big chunk of the userbase doesnt understand, and has no need for a logging package How does that proposal impede them ? if you don't need any of the extras you'd be able to just `log.Info"
tptacek 4y ago
It's not going into std; it's going into /x.
- marcus_holmes 4y ago
  effectively the same place at this point. There's so many "standard" libs in /x that trying to constraint to "just the standard lib" has to include /x to be actually useful.
  - tptacek 4y ago
    They're not at all effectively the same place, and much of what's in /x competes with third-party packages, which are on a somewhat level playing field in that both /x and third-party code have to be explicitly brought in to a project as a dep.
    - marcus_holmes 4y ago
      meh, I'm always going to prefer an /x package to a 3rd-party package.
  - philosopher1234 4y ago
    Such as?
    - marcus_holmes 4y ago
      goimports, godoc and the other tools (https://pkg.go.dev/golang.org/x/tools https://pkg.go.dev/golang.org/x/tools - you basically can't use Go properly without these) all the crypto stuff (https://pkg.go.dev/golang.org/x/crypto https://pkg.go.dev/golang.org/x/crypto) if you're doing anything with crypto then this is needed others are really useful (see index at https://pkg.go.dev/golang.org/x https://pkg.go.dev/golang.org/x) - I've imported the image, sync and text packages it's a lot of the "missing bits" from the standard lib. I get why they're in /x but the distinction is moot in most cases.

atsjie 4y ago

The majority of Golang devs actually do work on web services, API's, or libs/frameworks that would benefit from a standardized structured logging interface. Source: https://go.dev/blog/survey2022-q2-results https://go.dev/blog/survey2022-q2-results (scroll almost to the bottom) For the audience of this proposal it's probably fairly evident what the advantages of standardization would be. Also you argue quite strongly about the poor API design decisions being made; while you similarly admit not having worked with such tools or with web services in Golang general. That's odd.

svnpenn 4y ago
I don't work with servers much, but I do have extensive experience in Go. I know a bad API when I see one. strings.Replacer [1] uses the same syntax, and several others have commented in the discussion about it being a bad idea. 1. https://godocs.io/strings#Replacer https://godocs.io/strings#Replacer
- xani_ 4y ago
  From what I've skimmed the most problems seem to be "golang's type system won't allow expressing all of that on compile time so it will be costly" and I don't think that can be sensibly fixed in logging package > I know a bad API when I see one. strings.Replacer [1] uses the same syntax, and several others have commented in the discussion about it being a bad idea. What's wrong with it ? Never used it. But I think the complaints has been about wasting time for runtime type handling and Replacer doesn't have that problem.
  - Joker_vD 4y ago
    The main performance problem is with "kvs ...any" parameter in the Logger's functions because it allocates lots of stuff: the array itself, and then the wrappers around the values. I've tried to use something like this in a central loop of a CPU-emulator to trace every instruction executed and IIRC it slowed things about 3x times down. The only thing you can do to escape this cost is if logging.Level() >= logging.DEBUG { logging.Logger.Debug(...) } but it's ugly and can't be abstracted away. And yes, I've put several of such guards in the several hot paths of some actual production code base and seen about 25% speed up after that edit.
  - jen20 4y ago
    > What's wrong with it ? It relies on lining up pairs of values in a way that cannot be checked by the compiler rather than having dedicated functions that ensure they are kept together. This is the fundamental difference between bad logging libraries (like klog, hclog etc) and good logging libraries (like zerolog).
    - xani_ 4y ago
      I guess simplest fix for that particular one would be accepting `map[string]string` but that won't exactly work for the any type, and you can't easily add more. Maybe `...map[string]interface{}` but we're back to "the overhead is kinda shit"

kaz-inc 4y ago

I also didn't understand the need for a logging package for a while, but I think it makes sense for go because they're focusing on an audience that writes server applications.

xani_ 4y ago
It's less about package and more about interface. One "good enough" in stdlib means libraries could just require passing a interface matching it. Then no matter which logging package you use, as long as it fulfils the interface of stdlib logger it can be used with any library or other component you pulled. So with simple app you can just have simple logger that dumps everything on stderr, complex app can have whatever complex logging logic it needs, but lib "just using stdlib logger" doesn't care about that. And as a lib author, again, you can just use the stdlib one and most likely cover all your use cases.

xani_ 4y ago

Develop bigger app and you will understand. But one example - having an logging interface that is not shit (the current builtin is so garbage it might as well not exist at all) allows for packages to adopt it as default method and you magically can just fed same logger to every dep you pull with no loss of features or having to write wrapper. Contexted logger means you can just fed ["plugin":pluginname] tagged one to every plugin and get logs pointing you to the place where it happened. The structured logging part is mostly to avoid woes of then having to parse it back to structured format via bunch of regexpes that will inevitably break next release. If you still don't get it, here is a regexp to parse HAProxy logs: .*haproxy\[(\d+)]: (.+?):(\d+) \[(.+?)\] (.+?)(|[\~]) (.+?)\/(.+?) ([\-\d]+)\/([\-\d]+)\/([\-\d]+)\/([\-\d]+)\/([\-\d]+) ([\-\d]+) ([\-\d]+) (\S+) (\S+) (\S)(\S)(\S)(\S) ([\-\d]+)\/([\-\d]+)\/([\-\d]+)\/([\-\d]+)\/([\-\d]+) ([\-\d]+)\/([\-\d]+)(| \{.*\}) (".*)([\n|\s]*?)$`) > I think Go team might be overlooking the fact that a big chunk of the userbase doesnt understand, and has no need for a logging package How does that proposal impede them ? if you don't need any of the extras you'd be able to just `log.Info"

tptacek 4y ago

It's not going into std; it's going into /x.

marcus_holmes 4y ago
effectively the same place at this point. There's so many "standard" libs in /x that trying to constraint to "just the standard lib" has to include /x to be actually useful.
- tptacek 4y ago
  They're not at all effectively the same place, and much of what's in /x competes with third-party packages, which are on a somewhat level playing field in that both /x and third-party code have to be explicitly brought in to a project as a dep.
  - marcus_holmes 4y ago
    meh, I'm always going to prefer an /x package to a 3rd-party package.
- philosopher1234 4y ago
  Such as?
  - marcus_holmes 4y ago
    goimports, godoc and the other tools (https://pkg.go.dev/golang.org/x/tools https://pkg.go.dev/golang.org/x/tools - you basically can't use Go properly without these) all the crypto stuff (https://pkg.go.dev/golang.org/x/crypto https://pkg.go.dev/golang.org/x/crypto) if you're doing anything with crypto then this is needed others are really useful (see index at https://pkg.go.dev/golang.org/x https://pkg.go.dev/golang.org/x) - I've imported the image, sync and text packages it's a lot of the "missing bits" from the standard lib. I get why they're in /x but the distinction is moot in most cases.