4 ms·

I admit to doing this. Even many of the useful error messages that clearly indicate the fix are drowned out out by the mass of output. I've made this mistake

by starkd 4y ago

I admit to doing this. Even many of the useful error messages that clearly indicate the fix are drowned out out by the mass of output. I've made this mistake before, and I'll probably do so again.

chillfox 4y ago
I feel like this is a problem of overly chatty application logs + lack of formatting for errors. If the volume of drivel was lowered and errors were formatted with spacing and color to stand out, then they would be easier to focus on. So log errors to stderr, send it to a separate log file, and format it well (use multiple lines).
- marcosdumay 4y ago
  > So log errors to stderr, send it to a separate log file, and format it well (use multiple lines). Oh, for sure. Do never: - send errors to the same log you send normal activity. - default into logging things that aren't errors on the error log (make this possible to override if you want, but never the default). - log the errors there, but the necessary context on stdout so it appears correct on a terminal. (E.g. build tools that print entering into target in stdout; error in stderr; leaving target in stdout) - try to recover just to show a different error later.
  - randomswede 4y ago
    Oh, boy, on the "never log expected failures as errors" front, I once worked with a database system that used opportunistic transactions. Basically, each modification to a row carried effectively the original value of that trow with the update and if it failed, the API call triggered an error saying that the transaction failed. So if you did a "SET column=(column+1) WHERE rowid=unique", the client could basically do an automatic retry. But, it also logged each and every occurrence of this at "Error" severity, instead of at "Info" severity (it is, after all, expected to happen once in a while). And of course, once our code switched over to using this, the first few times every team member had to deal with a production issue, the immediate reaction was "oh, no, the data store is unhealthy! look at this mass of error logs, I can see one every few minutes!". Thankfully, after the first team member (me, as it happens) spent half an hour reading the relevant parts of the design and implementation docs, we could frequently short-cut a lengthy investigation by "oh, you think $DB is bad because you are seeing transaction failures? no, that's expected, see $URL".

by starkd 4y ago

I admit to doing this. Even many of the useful error messages that clearly indicate the fix are drowned out out by the mass of output. I've made this mistake before, and I'll probably do so again.

chillfox 4y ago

I feel like this is a problem of overly chatty application logs + lack of formatting for errors. If the volume of drivel was lowered and errors were formatted with spacing and color to stand out, then they would be easier to focus on. So log errors to stderr, send it to a separate log file, and format it well (use multiple lines).

marcosdumay 4y ago
> So log errors to stderr, send it to a separate log file, and format it well (use multiple lines). Oh, for sure. Do never: - send errors to the same log you send normal activity. - default into logging things that aren't errors on the error log (make this possible to override if you want, but never the default). - log the errors there, but the necessary context on stdout so it appears correct on a terminal. (E.g. build tools that print entering into target in stdout; error in stderr; leaving target in stdout) - try to recover just to show a different error later.
- randomswede 4y ago
  Oh, boy, on the "never log expected failures as errors" front, I once worked with a database system that used opportunistic transactions. Basically, each modification to a row carried effectively the original value of that trow with the update and if it failed, the API call triggered an error saying that the transaction failed. So if you did a "SET column=(column+1) WHERE rowid=unique", the client could basically do an automatic retry. But, it also logged each and every occurrence of this at "Error" severity, instead of at "Info" severity (it is, after all, expected to happen once in a while). And of course, once our code switched over to using this, the first few times every team member had to deal with a production issue, the immediate reaction was "oh, no, the data store is unhealthy! look at this mass of error logs, I can see one every few minutes!". Thankfully, after the first team member (me, as it happens) spent half an hour reading the relevant parts of the design and implementation docs, we could frequently short-cut a lengthy investigation by "oh, you think $DB is bad because you are seeing transaction failures? no, that's expected, see $URL".