7 ms·

Generally speaking, if you're trying to write a reliable distributed system (and email is a massively distributed system), a good principle to follow is to retr

by growse 6y ago

Generally speaking, if you're trying to write a reliable distributed system (and email is a massively distributed system), a good principle to follow is to retry on failure, no matter the failure. Obviously there are edge cases, and obviously you don't just retry every 1ms forever, but to assume that an error that comes back from a system you've called once is both (a) the product of a 100% working non-faulty system and/or (b) not in any way a transitory issue or failure is naive. This type of thinking is how we end up with brittle systems. A cautious implementation might see an SMTP 500 and then may just choose to try it again once or twice a little later, to see exactly how 'permanent' that failure is. What the "but the RFC says!" people seem to forget is that the spec is only meaningful when the system is working correctly. Good engineering never assumes that.

throwaway201103 6y ago
This makes email delivery much more complicated than it already is, for no good reason. There are no guarantees when it comes to email delivery anyway.
- pmlnr 6y ago
  But there are. At least there supposed to be: a 2xx should be issued when the mail is written to storage, and not before.
bbarnett 6y ago
Email is extremely non-brittle, and everyone following the RFC is one main way it is able to be non-brittle. What is naive here, is that you think things are the way there are, for no reason. Mail is decades old, built upon millions of hours of work crafting software, RFC standards, and works the way it does, including bounces, to ensure stability. A 5xx error means 'perm failure'. There are a variety of 5xx class responses, from 'user account deleted' to 'no such domain'. It is, in fact, a working mail system, which responds with 5xx, or 4xx (temp fail) or 2xx (received OK) messages. A severely broken system is incapable of even responding. For a mail system to respond with 5xx, when it is internally broken, is 100% a configuration issue. Every MTA on the planet, is designed (eg, postfix, sendmail, etc) to respond with a 4xx TEMP fail message if something is borked. A milter gone bad. A library missing. A full disk. A config issue. An issue forking. Memory limits. All of it. By default, MTAs are designed to 4xx(tempfail) on those errors. Loads, and loads, and loads of work to ensure that. Code meticulously crafted. This is how good engineering works. This is how the RFC works. You either get no ACK for your SYN, because it is so borked, or you get a 4xx if it can run, but has a failure condition.. OR someone did something very, very wrong. I understand your angst, but the real problem isn't the RFC, or the sender not retrying. The problem is: - the MTA was up and running - it had an error condition on its back end - whomever set everything up, didn't taking into account internal failure conditions, and to respond with a 4xx if that was the case THAT is where the "good engineering" failed. Mail accounts are deactivated all the time. Domains are deactivated all the time. 5xx tells us this. Someone replying with 5xx, when they did not mean to, has a configuration issue on their end. That's all there is to it. I'll put this another way. What you want to do is make 5xx like 4xx, because you feel there should never be any way for a SMTP server to say "No, really, this email address doesn't exist.. don't bother trying again".
- growse 6y ago
  > Email is extremely non-brittle, and everyone following the RFC is one main way it is able to be non-brittle. What is naive here, is that you think things are the way there are, for no reason. > Mail is decades old, built upon millions of hours of work crafting software, RFC standards, and works the way it does, including bounces, to ensure stability. Same people argue that email is a hot mess exactly because it's decades old and comprised of a patchwork of standards. It's not exactly the poster child of how to do this sort of thing well. > For a mail system to respond with 5xx, when it is internally broken, is 100% a configuration issue. Every MTA on the planet, is designed (eg, postfix, sendmail, etc) to respond with a 4xx TEMP fail message if something is borked. A milter gone bad. A library missing. A full disk. A config issue. An issue forking. Memory limits. All of it. Configuration issues can be, and often are, temporary. Also, bugs are a thing. > THAT is where the "good engineering" failed. I agree with this. > What you want to do is make 5xx like 4xx, because you feel there should never be any way for a SMTP server to say "No, really, this email address doesn't exist.. don't bother trying again". Of course there's a semantic difference between "come back later" and "go away forever", I'm just arguing that a client that doesn't blindly trust what every connected system tells it is going to more often successfully achieve its goals than one which does.
  - jimmydorry 6y ago
    If you hosted a mail server, would you really want an ever increasing number of mail servers hitting your mail server for invalid email addressess for an indeterminate period of time? The 5xx error code you are returning can't be trusted, after all. That seems to be what you are advocating for.
  - bbarnett 6y ago
    Same people argue that email is a hot mess You're lumping together "email content" with "SMTP". SMTP isn't a "patchwork of standards", it has a very specific RFC for it. SMTP works very, very, very well. Configuration issues can be, and often are, temporary. Also, bugs are a thing. This isn't just "a bug" or "a config issue", this is an edge case bug or config issue. I've handled literally hundreds of thousands of mail servers in high avail production, with some of those being extremely high volume. What you're wanting to change the normal flow of operation, for an extremely rare edge case. Of course there's a semantic difference between "come back later" and "go away forever", I'm just arguing that a client that doesn't blindly trust what every connected system tells it is going to more often successfully achieve its goals than one which does. Not true here. In 99.999999% of cases when you get a 5xx response code, the correct thing to do is immediate bounce. This sort of mess, 5xx 'by accident', is insanely rare. And it's not 'go away forever', it's "this specific mail cannot be delivered, please return it to sender, so they can examine the issue and deal with it". It's "bring a human into the equation". How is this an issue? For something which is very, very rare.

by growse 6y ago

throwaway201103 6y ago

This makes email delivery much more complicated than it already is, for no good reason. There are no guarantees when it comes to email delivery anyway.

pmlnr 6y ago
But there are. At least there supposed to be: a 2xx should be issued when the mail is written to storage, and not before.

bbarnett 6y ago

Email is extremely non-brittle, and everyone following the RFC is one main way it is able to be non-brittle. What is naive here, is that you think things are the way there are, for no reason. Mail is decades old, built upon millions of hours of work crafting software, RFC standards, and works the way it does, including bounces, to ensure stability. A 5xx error means 'perm failure'. There are a variety of 5xx class responses, from 'user account deleted' to 'no such domain'. It is, in fact, a working mail system, which responds with 5xx, or 4xx (temp fail) or 2xx (received OK) messages. A severely broken system is incapable of even responding. For a mail system to respond with 5xx, when it is internally broken, is 100% a configuration issue. Every MTA on the planet, is designed (eg, postfix, sendmail, etc) to respond with a 4xx TEMP fail message if something is borked. A milter gone bad. A library missing. A full disk. A config issue. An issue forking. Memory limits. All of it. By default, MTAs are designed to 4xx(tempfail) on those errors. Loads, and loads, and loads of work to ensure that. Code meticulously crafted. This is how good engineering works. This is how the RFC works. You either get no ACK for your SYN, because it is so borked, or you get a 4xx if it can run, but has a failure condition.. OR someone did something very, very wrong. I understand your angst, but the real problem isn't the RFC, or the sender not retrying. The problem is: - the MTA was up and running - it had an error condition on its back end - whomever set everything up, didn't taking into account internal failure conditions, and to respond with a 4xx if that was the case THAT is where the "good engineering" failed. Mail accounts are deactivated all the time. Domains are deactivated all the time. 5xx tells us this. Someone replying with 5xx, when they did not mean to, has a configuration issue on their end. That's all there is to it. I'll put this another way. What you want to do is make 5xx like 4xx, because you feel there should never be any way for a SMTP server to say "No, really, this email address doesn't exist.. don't bother trying again".

growse 6y ago
> Email is extremely non-brittle, and everyone following the RFC is one main way it is able to be non-brittle. What is naive here, is that you think things are the way there are, for no reason. > Mail is decades old, built upon millions of hours of work crafting software, RFC standards, and works the way it does, including bounces, to ensure stability. Same people argue that email is a hot mess exactly because it's decades old and comprised of a patchwork of standards. It's not exactly the poster child of how to do this sort of thing well. > For a mail system to respond with 5xx, when it is internally broken, is 100% a configuration issue. Every MTA on the planet, is designed (eg, postfix, sendmail, etc) to respond with a 4xx TEMP fail message if something is borked. A milter gone bad. A library missing. A full disk. A config issue. An issue forking. Memory limits. All of it. Configuration issues can be, and often are, temporary. Also, bugs are a thing. > THAT is where the "good engineering" failed. I agree with this. > What you want to do is make 5xx like 4xx, because you feel there should never be any way for a SMTP server to say "No, really, this email address doesn't exist.. don't bother trying again". Of course there's a semantic difference between "come back later" and "go away forever", I'm just arguing that a client that doesn't blindly trust what every connected system tells it is going to more often successfully achieve its goals than one which does.
- jimmydorry 6y ago
  If you hosted a mail server, would you really want an ever increasing number of mail servers hitting your mail server for invalid email addressess for an indeterminate period of time? The 5xx error code you are returning can't be trusted, after all. That seems to be what you are advocating for.
- bbarnett 6y ago
  Same people argue that email is a hot mess You're lumping together "email content" with "SMTP". SMTP isn't a "patchwork of standards", it has a very specific RFC for it. SMTP works very, very, very well. Configuration issues can be, and often are, temporary. Also, bugs are a thing. This isn't just "a bug" or "a config issue", this is an edge case bug or config issue. I've handled literally hundreds of thousands of mail servers in high avail production, with some of those being extremely high volume. What you're wanting to change the normal flow of operation, for an extremely rare edge case. Of course there's a semantic difference between "come back later" and "go away forever", I'm just arguing that a client that doesn't blindly trust what every connected system tells it is going to more often successfully achieve its goals than one which does. Not true here. In 99.999999% of cases when you get a 5xx response code, the correct thing to do is immediate bounce. This sort of mess, 5xx 'by accident', is insanely rare. And it's not 'go away forever', it's "this specific mail cannot be delivered, please return it to sender, so they can examine the issue and deal with it". It's "bring a human into the equation". How is this an issue? For something which is very, very rare.