3 ms·

No, error messages should tell the user what they need to do to accomplish their goal as concisely as possible.

barrkel 3y ago
Actionable error messages are a lot more work and are not always feasible. A library discovering a mismatch between a configuration setting and a value it received by API isn't in a position to make a decision on which is wrong: is it misconfigured, or is the value it received via API under the control of the user? The caller of the API isn't necessarily in a better place to make a decision. The higher up the stack you go, the more context you have on the action the user is trying to perform but the number of potential errors increases exponentially. It's usually not feasible to contextualize every possible error condition, particularly for things like configuration which are side-loaded while executing code and are a library implementation detail from the perspective of the application writer. Consider access denied to a file: is this a user mistake or a permission mistake? Should the user change the file permissions, or not try to access that file? Consider quota exceeded while writing to a file: should the user free up space, create a smaller file, or ask their administrator to give them more space? What if the quota was exceeded due to a configuration setting in the application specifying where to put scratch files? It gets very complicated very quickly, and even the simplest of actions - like writing a byte out to a file - have errors that require global knowledge to give good actionable feedback on.
- estebank 3y ago
  It's true that providing good actionable errors with useful context is hard and sometimes requires global knowledge. It still should be attempted. I see that if a project doesn't aim for that objective, then errors are the first thing to degrade and stay bad indefinitely. If even a modicum of effort is put to improve them, the project attracts users and contributors that appreciate them and try to improve them or at least hold the bar where it is. So, yes, it's an uphil battle, but one that needs to be fought. I keep looking around at well established projects that don't even try, and as a user they are incredibly frustrating, because improving the status quo wouldn't require introducing global analysis, it might be as simple as adding an extra sentence with some internal state, a description with a few keywords to Google for, or a link to the docs. I'm well aware of the effort needed to have great error messages. I'd be happy if the average error message was passable, instead of terrible. Part of the problem (in open source) is that developers write tools for themselves first, so they are intimately knowledgeable about the internal state of the application, so they provide enough context for themselves. So little more is given unless enough people complain and the original developer agrees to change. In that ecosystem we constantly interact with tools written like that, so that's "the bar" we're used to. All our tools end up taking the approach of "The user did something stupid, I don't know", so that's what we do in our tools. And I completely reject that framing. I take the approach of "The user did something that didn't met the applications, sometimes implicit, preconditions." and take if from there. Maybe what they did was wrong due to a misunderstanding caused by bad docs, maybe the user did something wrong because we're missing a feature implied by the rest of the features, maybe the user hasn't done a necessary configuration step. All of those things can be checked for! I can't think of any tool that isn't necessarily an error reporting tool. But few projects takes that into consideration and approaches it as "this is an error reporting tool first, and the actual thing it does is secondary". If the user cannot accomplish their task, it doesn't matter if it is the fastest tool in the West. "Syntax error on line 1" is very different to "missing WHERE condition after AND on line 1 column 45". A lot of the examples you have have alternative courses of action. Great, enumerate them in order of likelihood. Don't even claim to be right, just throw your users a bone. If you're right with any of them, you just saved them time and anguish. If you're wrong in all cases you mention, then you have a bug in your output. You can even get away with "these two things might be happening, but there might be other problems, can't help you more if so". "IO error", "couldn't to disk", "couldn't write to file", "couldn't write to file '/tmp/foo-bar'", "couldn't write to document metadata file '/tmp/foo-bar' because the file already exists and their permissions dont permit this user to write to it; you can change the path with '--metadata-path=/path/to/file" are all the same error, but I'd much rather get the last one, over the first one.
  - barrkel 3y ago
    The best is the enemy of the good. In practice, instead of describing the error, a policy for actionable error messages all too frequently results in something along the lines of Action failed. Go to https://example.com/foo?code=E123 for more details. It's "actionable" - there's a place you can go for more information - but there's little or no context, and the web page is frequently vague. You need to dive into logs to find out what goes wrong these days. I prefer an error message that isn't necessarily actionable, but tells me what the code was trying to do that failed, even if it's very low level. Error: Disk quota exceeded while trying to write to "D:\scratch\ABCEF.tmp" This gives me clues. I can recognize a directory I configured, or a filename I'm working with. I know which disk is the problem. Etc. And of course these two approaches can't be easily melded without transmitting config, detailed failure information including disk free space etc. back to home base. PS: I use Windows paths here because I noticed a significant degradation in error messages from Microsoft software in the early 2000s, which made it harder to fix problems locally without searching the internet for error codes and randomly trying out fixes from various forums. I also believe that an industry-wide pull-back from exceptions as an error transmission mechanism resulted in worse error messages. Without runtime polymorphism to encapsulate context from the point of failure, the higher level parts of the app no longer have enough information to even give a descriptive error message, the way a top-level UI exception handler used to.

Actionable error messages are a lot more work and are not always feasible. A library discovering a mismatch between a configuration setting and a value it received by API isn't in a position to make a decision on which is wrong: is it misconfigured, or is the value it received via API under the control of the user? The caller of the API isn't necessarily in a better place to make a decision. The higher up the stack you go, the more context you have on the action the user is trying to perform but the number of potential errors increases exponentially. It's usually not feasible to contextualize every possible error condition, particularly for things like configuration which are side-loaded while executing code and are a library implementation detail from the perspective of the application writer. Consider access denied to a file: is this a user mistake or a permission mistake? Should the user change the file permissions, or not try to access that file? Consider quota exceeded while writing to a file: should the user free up space, create a smaller file, or ask their administrator to give them more space? What if the quota was exceeded due to a configuration setting in the application specifying where to put scratch files? It gets very complicated very quickly, and even the simplest of actions - like writing a byte out to a file - have errors that require global knowledge to give good actionable feedback on.

estebank 3y ago
It's true that providing good actionable errors with useful context is hard and sometimes requires global knowledge. It still should be attempted. I see that if a project doesn't aim for that objective, then errors are the first thing to degrade and stay bad indefinitely. If even a modicum of effort is put to improve them, the project attracts users and contributors that appreciate them and try to improve them or at least hold the bar where it is. So, yes, it's an uphil battle, but one that needs to be fought. I keep looking around at well established projects that don't even try, and as a user they are incredibly frustrating, because improving the status quo wouldn't require introducing global analysis, it might be as simple as adding an extra sentence with some internal state, a description with a few keywords to Google for, or a link to the docs. I'm well aware of the effort needed to have great error messages. I'd be happy if the average error message was passable, instead of terrible. Part of the problem (in open source) is that developers write tools for themselves first, so they are intimately knowledgeable about the internal state of the application, so they provide enough context for themselves. So little more is given unless enough people complain and the original developer agrees to change. In that ecosystem we constantly interact with tools written like that, so that's "the bar" we're used to. All our tools end up taking the approach of "The user did something stupid, I don't know", so that's what we do in our tools. And I completely reject that framing. I take the approach of "The user did something that didn't met the applications, sometimes implicit, preconditions." and take if from there. Maybe what they did was wrong due to a misunderstanding caused by bad docs, maybe the user did something wrong because we're missing a feature implied by the rest of the features, maybe the user hasn't done a necessary configuration step. All of those things can be checked for! I can't think of any tool that isn't necessarily an error reporting tool. But few projects takes that into consideration and approaches it as "this is an error reporting tool first, and the actual thing it does is secondary". If the user cannot accomplish their task, it doesn't matter if it is the fastest tool in the West. "Syntax error on line 1" is very different to "missing WHERE condition after AND on line 1 column 45". A lot of the examples you have have alternative courses of action. Great, enumerate them in order of likelihood. Don't even claim to be right, just throw your users a bone. If you're right with any of them, you just saved them time and anguish. If you're wrong in all cases you mention, then you have a bug in your output. You can even get away with "these two things might be happening, but there might be other problems, can't help you more if so". "IO error", "couldn't to disk", "couldn't write to file", "couldn't write to file '/tmp/foo-bar'", "couldn't write to document metadata file '/tmp/foo-bar' because the file already exists and their permissions dont permit this user to write to it; you can change the path with '--metadata-path=/path/to/file" are all the same error, but I'd much rather get the last one, over the first one.
- barrkel 3y ago
  The best is the enemy of the good. In practice, instead of describing the error, a policy for actionable error messages all too frequently results in something along the lines of Action failed. Go to https://example.com/foo?code=E123 for more details. It's "actionable" - there's a place you can go for more information - but there's little or no context, and the web page is frequently vague. You need to dive into logs to find out what goes wrong these days. I prefer an error message that isn't necessarily actionable, but tells me what the code was trying to do that failed, even if it's very low level. Error: Disk quota exceeded while trying to write to "D:\scratch\ABCEF.tmp" This gives me clues. I can recognize a directory I configured, or a filename I'm working with. I know which disk is the problem. Etc. And of course these two approaches can't be easily melded without transmitting config, detailed failure information including disk free space etc. back to home base. PS: I use Windows paths here because I noticed a significant degradation in error messages from Microsoft software in the early 2000s, which made it harder to fix problems locally without searching the internet for error codes and randomly trying out fixes from various forums. I also believe that an industry-wide pull-back from exceptions as an error transmission mechanism resulted in worse error messages. Without runtime polymorphism to encapsulate context from the point of failure, the higher level parts of the app no longer have enough information to even give a descriptive error message, the way a top-level UI exception handler used to.