4 ms·

> "I don't want to" is not a valid excuse for the client. If you're implementing a server, "the client SHOULD but didn't" isn't a valid excuse to reject a clie

> "I don't want to" is not a valid excuse for the client. If you're implementing a server, "the client SHOULD but didn't" isn't a valid excuse to reject a client either. You can do it anyway, you might even have good reasons for it, but then you sure don't get to point at the RFC and call the client broken.

Arnt 8mo ago
Hearsay has it that the reason is spam. Spam messages are said to have massively higher chances of minor RFC violations when they arrive at the destination server.
- shadowgovt 8mo ago
  Most of the time, in my experience, when one encounters a situation like this in Internet tech (i.e. "why is this suggestion treated like a hard requirement?"), this is the answer: "because attackers found a way to exploit the lack of the suggestion's implementation in the wild, so it is now a hard requirement." The standards, to my observation, tend to lag the CVEs. Side-note: If someone has built a reverse-database that annotates RFCs with overriding CVEs that have invalidated or rendered harmful part of the spec, I'd love to put that in my toolbox. It'd be nice-to-have in the extreme if it hasn't been created yet.
  - atherton94027 8mo ago
    How is not having a message-id a security risk? It seems that Gmail is being pedantic for no reason
    - geocar 8mo ago
      > How is not having a message-id a security risk? CVE classify a lot of things that have nothing to do with security. Not having a Message-ID can cause problems for loop-detection (especially on busy netnews and mailing lists), and with reliable delivery status notification. Dealing with these things for clients who can't read the RFC wastes memory and time which can potentially deny legitimate users access to services > It seems that Gmail is being pedantic for no reason Now you know that feeling is just ignorance.
      - hsbauauvhabzb 8mo ago
        So add a message id at the first stop, or hard ban the sender server version until they confirm. A midway point that involves a doom switch is not a good option.
        geocar 8mo ago
        > So add a message id at the first stop That should have already happened. Google is not the "first stop". > hard ban the sender server version until they confirm SMTP clients do not announce their version. Also I don't work for you, stop telling me what to do. > A midway point that involves a doom switch is not a good option. No shit. That's almost certainly a big part of why Google blocks messages from being transited without a Message-ID.
      - atherton94027 8mo ago
        Well, gmail does not manage usenet groups and mailing lists. Delivery status notifications are considered best effort so it wouldn't make sense to block messages for that case. Additionally, Gmail adds its own message identifier on every message (g-msgid) because it knows that message ids can not be trusted to be unique. Finally just calling me ignorant is the cherry on top – please try to keep things civil on here.
        geocar 8mo ago
        > Well, [google] does not manage usenet groups and mailing lists. They do. Sort of. Google used to nntp, and manages the largest usenet archive; They still have one of the largest mailing list servers in the world, and they still perform distribution on those lists via SMTP email. They still have all of the problems associated with it, as do lots of other mail/news/list sites still do that are a fraction of Google's size. > Delivery status notifications are considered best effort so it wouldn't make sense to block messages for that case. Sure it does. You consider them best-effort, but that doesn't follow that I should consider them best-effort. For a simple example: Consider spam. In any event, if you keep sending me the same message without any evidence you can handle them, I'm not going to accept your messages either, because I don't know what else you aren't doing. That's part of the subtext of "SHOULD". Most big sites take this policy because it is internal nodes that will generate the delivery notification, but the edge nodes that are tasked with preventing loops. If the edge node adds a Message-ID based on the content, it'll waste CPU and possibly deny service; If the edge node naively adds a Message-ID like an MSA, the origin won't recognise it, and forwarded messages can loop or (if sent to a mailing list) be amplified. There also are other specific documented requirements related to Internet Mail that edge nodes not do this (e.g. RFC2821 § 6.3). However you seem to be assuming Google is blocking messages "for this case" which is a little presumptuous. Google is presumably trying to save themselves a headache of handling errors for people who aren't prepared to do anything about it, the most common of which is spam. And the use of Message-ID in this application is documented at least as early as RFC2635. > Additionally, Gmail adds its own message identifier on every message (g-msgid) because it knows that message ids can not be trusted to be unique. Without knowing what Google does with the g-msgid header, you are making a mistake to assume it is equivalent to the Message-ID header just because it has a similar name. You have no reason to believe this is true. > Finally just calling me ignorant is the cherry on top – please try to keep things civil on here. I am sorry you are offended to not know things, but you do not know this thing, and your characterising my actions will make it very difficult for you to learn something new and not be so ignorant in the future. Think hard exactly about what you want to happen here: Do you want Google (et al) to do something different? Do you want me to agree Google should? Who exactly are you trying to convince of what, and to what end? I am trying to tell you how to interpret the documentation of the Internet, in this case to be successful sending email. That's it. I am not likely to try and tell Google what to do in this case because of my own experiences running mail servers over the last 30 years, but listen: I am willing to be convinced. That's all I can do. If it's something else, I'm sorry I just don't understand.
    - shadowgovt 8mo ago
      Because in practice it showed up for a period of time as a common thing in spam-senders. They were trying to maximize throughput and minimize software maintenance costs, so they leave out things that the spec says are optional. But that makes "a commonly-implemented optional thing was left out" into a stronger spam signal. Is it still a strong spam signal? Hard to say. Sources disagree. But as with laws, heuristics, once added, are often sticky.
Veserv 8mo ago
You are describing MAY. “MAY This word, or the adjective "OPTIONAL", mean that an item is truly optional… An implementation which does not include a particular option MUST be prepared to interoperate with another implementation which does include the option, though perhaps with reduced functionality. In the same vein an implementation which does include a particular option MUST be prepared to interoperate with another implementation which does not include the option (except, of course, for the feature the option provides.)” Note how it explicitly calls out interoperation with implementations that do or do not implement MAY. As a exception that proves the rule, we can reasonably assume that not interoperating with a system ignoring a SHOULD rule is a correct implementation and it is the fault of whoever is not implementing SHOULD.
geocar 8mo ago
> isn't a valid excuse to reject a client either. Yes it absolutely is: https://www.rfc-editor.org/rfc/rfc2119 https://www.rfc-editor.org/rfc/rfc2119 is quite clear. 3. SHOULD This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course. If the client SHOULD do something and doesn't, and your server does not know why, you SHOULD disconnect and move on. If the server has considered fully the implications of not having a Message-ID header, then it MAY continue processing. In general, you will find most of the Internet specifications are labelled MUST if they are required for the protocol's own state-processing (i.e. as documented), while specifications are labelled SHOULD if they are required for application state-processing in some circumstances (i.e. other users of the protocol).
- hsbauauvhabzb 8mo ago
  Either the server must explicitly confirm to servers or the clients must accept everything. Otherwise message delivery is not guaranteed. In the context of an email protocol, this often is a silent failure which causes real-world problems. I don’t care what the protocol rfc says, the client arbitrarily rejecting an email from the server for some missing unimportant header (for deduction detection?) is silly.
  - behringer 8mo ago
    If it was unimportant it would be MAY.
    - hsbauauvhabzb 8mo ago
      Is the server somehow unable to inject an ID if the sender did not send one? Stop hiding behind policy and think for yourself.
      - geocar 8mo ago
        > Is the server somehow unable to inject an ID if the sender did not send one? Yes. https://www.rfc-editor.org/rfc/rfc2821#section-6.3 https://www.rfc-editor.org/rfc/rfc2821#section-6.3 refers to servers that do this and says very clearly: These changes MUST NOT be applied by an SMTP server that provides an intermediate relay function. That's Google in this situation. > Stop hiding behind policy and think for yourself. Sometimes you should think for yourself, but sometimes, and friend let me tell you this is one of those times, you should take some time to read all of the things that other people have thought about a subject, especially when that subject is as big and old as email. There is no good reason viva couldn't make a Message-ID, but there's a good reason to believe they can't handle delivery status notifications, and if they can't do that, they are causing bigger problems than just this.

Arnt 8mo ago

Hearsay has it that the reason is spam. Spam messages are said to have massively higher chances of minor RFC violations when they arrive at the destination server.

shadowgovt 8mo ago
Most of the time, in my experience, when one encounters a situation like this in Internet tech (i.e. "why is this suggestion treated like a hard requirement?"), this is the answer: "because attackers found a way to exploit the lack of the suggestion's implementation in the wild, so it is now a hard requirement." The standards, to my observation, tend to lag the CVEs. Side-note: If someone has built a reverse-database that annotates RFCs with overriding CVEs that have invalidated or rendered harmful part of the spec, I'd love to put that in my toolbox. It'd be nice-to-have in the extreme if it hasn't been created yet.
- atherton94027 8mo ago
  How is not having a message-id a security risk? It seems that Gmail is being pedantic for no reason
  - geocar 8mo ago
    > How is not having a message-id a security risk? CVE classify a lot of things that have nothing to do with security. Not having a Message-ID can cause problems for loop-detection (especially on busy netnews and mailing lists), and with reliable delivery status notification. Dealing with these things for clients who can't read the RFC wastes memory and time which can potentially deny legitimate users access to services > It seems that Gmail is being pedantic for no reason Now you know that feeling is just ignorance.
    - hsbauauvhabzb 8mo ago
      So add a message id at the first stop, or hard ban the sender server version until they confirm. A midway point that involves a doom switch is not a good option.
      - geocar 8mo ago
        > So add a message id at the first stop That should have already happened. Google is not the "first stop". > hard ban the sender server version until they confirm SMTP clients do not announce their version. Also I don't work for you, stop telling me what to do. > A midway point that involves a doom switch is not a good option. No shit. That's almost certainly a big part of why Google blocks messages from being transited without a Message-ID.
    - atherton94027 8mo ago
      Well, gmail does not manage usenet groups and mailing lists. Delivery status notifications are considered best effort so it wouldn't make sense to block messages for that case. Additionally, Gmail adds its own message identifier on every message (g-msgid) because it knows that message ids can not be trusted to be unique. Finally just calling me ignorant is the cherry on top – please try to keep things civil on here.
      - geocar 8mo ago
        > Well, [google] does not manage usenet groups and mailing lists. They do. Sort of. Google used to nntp, and manages the largest usenet archive; They still have one of the largest mailing list servers in the world, and they still perform distribution on those lists via SMTP email. They still have all of the problems associated with it, as do lots of other mail/news/list sites still do that are a fraction of Google's size. > Delivery status notifications are considered best effort so it wouldn't make sense to block messages for that case. Sure it does. You consider them best-effort, but that doesn't follow that I should consider them best-effort. For a simple example: Consider spam. In any event, if you keep sending me the same message without any evidence you can handle them, I'm not going to accept your messages either, because I don't know what else you aren't doing. That's part of the subtext of "SHOULD". Most big sites take this policy because it is internal nodes that will generate the delivery notification, but the edge nodes that are tasked with preventing loops. If the edge node adds a Message-ID based on the content, it'll waste CPU and possibly deny service; If the edge node naively adds a Message-ID like an MSA, the origin won't recognise it, and forwarded messages can loop or (if sent to a mailing list) be amplified. There also are other specific documented requirements related to Internet Mail that edge nodes not do this (e.g. RFC2821 § 6.3). However you seem to be assuming Google is blocking messages "for this case" which is a little presumptuous. Google is presumably trying to save themselves a headache of handling errors for people who aren't prepared to do anything about it, the most common of which is spam. And the use of Message-ID in this application is documented at least as early as RFC2635. > Additionally, Gmail adds its own message identifier on every message (g-msgid) because it knows that message ids can not be trusted to be unique. Without knowing what Google does with the g-msgid header, you are making a mistake to assume it is equivalent to the Message-ID header just because it has a similar name. You have no reason to believe this is true. > Finally just calling me ignorant is the cherry on top – please try to keep things civil on here. I am sorry you are offended to not know things, but you do not know this thing, and your characterising my actions will make it very difficult for you to learn something new and not be so ignorant in the future. Think hard exactly about what you want to happen here: Do you want Google (et al) to do something different? Do you want me to agree Google should? Who exactly are you trying to convince of what, and to what end? I am trying to tell you how to interpret the documentation of the Internet, in this case to be successful sending email. That's it. I am not likely to try and tell Google what to do in this case because of my own experiences running mail servers over the last 30 years, but listen: I am willing to be convinced. That's all I can do. If it's something else, I'm sorry I just don't understand.
  - shadowgovt 8mo ago
    Because in practice it showed up for a period of time as a common thing in spam-senders. They were trying to maximize throughput and minimize software maintenance costs, so they leave out things that the spec says are optional. But that makes "a commonly-implemented optional thing was left out" into a stronger spam signal. Is it still a strong spam signal? Hard to say. Sources disagree. But as with laws, heuristics, once added, are often sticky.

Veserv 8mo ago

You are describing MAY. “MAY This word, or the adjective "OPTIONAL", mean that an item is truly optional… An implementation which does not include a particular option MUST be prepared to interoperate with another implementation which does include the option, though perhaps with reduced functionality. In the same vein an implementation which does include a particular option MUST be prepared to interoperate with another implementation which does not include the option (except, of course, for the feature the option provides.)” Note how it explicitly calls out interoperation with implementations that do or do not implement MAY. As a exception that proves the rule, we can reasonably assume that not interoperating with a system ignoring a SHOULD rule is a correct implementation and it is the fault of whoever is not implementing SHOULD.

geocar 8mo ago

> isn't a valid excuse to reject a client either. Yes it absolutely is: https://www.rfc-editor.org/rfc/rfc2119 https://www.rfc-editor.org/rfc/rfc2119 is quite clear. 3. SHOULD This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course. If the client SHOULD do something and doesn't, and your server does not know why, you SHOULD disconnect and move on. If the server has considered fully the implications of not having a Message-ID header, then it MAY continue processing. In general, you will find most of the Internet specifications are labelled MUST if they are required for the protocol's own state-processing (i.e. as documented), while specifications are labelled SHOULD if they are required for application state-processing in some circumstances (i.e. other users of the protocol).

hsbauauvhabzb 8mo ago
Either the server must explicitly confirm to servers or the clients must accept everything. Otherwise message delivery is not guaranteed. In the context of an email protocol, this often is a silent failure which causes real-world problems. I don’t care what the protocol rfc says, the client arbitrarily rejecting an email from the server for some missing unimportant header (for deduction detection?) is silly.
- behringer 8mo ago
  If it was unimportant it would be MAY.
  - hsbauauvhabzb 8mo ago
    Is the server somehow unable to inject an ID if the sender did not send one? Stop hiding behind policy and think for yourself.
    - geocar 8mo ago
      > Is the server somehow unable to inject an ID if the sender did not send one? Yes. https://www.rfc-editor.org/rfc/rfc2821#section-6.3 https://www.rfc-editor.org/rfc/rfc2821#section-6.3 refers to servers that do this and says very clearly: These changes MUST NOT be applied by an SMTP server that provides an intermediate relay function. That's Google in this situation. > Stop hiding behind policy and think for yourself. Sometimes you should think for yourself, but sometimes, and friend let me tell you this is one of those times, you should take some time to read all of the things that other people have thought about a subject, especially when that subject is as big and old as email. There is no good reason viva couldn't make a Message-ID, but there's a good reason to believe they can't handle delivery status notifications, and if they can't do that, they are causing bigger problems than just this.