15 ms·
The curious case of the missing period
- ijuelz 2y agoThe night is dark and full of errors.
- 867-5309 2y ago>Every time an employee of our client needed to send out a document via email or needed to print a document that needed to be sent out by the postal services to the customer the employee would have to replace all the placeholders within the document ironically titled..
- jcpham2 2y agoAs soon as I started reading one of the first things that came to mind was the termination period in the smtp server spec. If you’ve ever had to troubleshoot an SMTP transport issue typing SMTP con Ds by hand is a familiar thing. Cool read.
- talkingtab 2y agoThis reminds me of an experience debugging a network protocol implementation - specifically AppleTalk NBP for other ancient people. I had coded everything but my packets were rejected (aka silently dropped) when real (aka Apple implementation) packets were not. I had a copy of the good and bad packets on the screen of my computer and had gone over them byte by byte to find the problem. And there was none. From start to finish they were exactly the same, with correct check sums etc. It was time to go home and I decided just to print the stupid things to look at later. As soon as I printed them, the error was clear. My version ran to two pages and the good implementation one page. I had not been careful to clear the buffer before sending the data (mbufs don't you know). This still cracks me up.
- the_real_tjaart 2y agoThanks for sharing!
- renewiltord 2y agoSuspected it was the 990 char limit per line but this is another one. I assume this is an old system? Despite many claims about “best practices”, there were definitely past platforms where these didn’t exist and a minimal implementation was safer just because you knew the scope of error. Of course if it was modern, different question.
- 256_ 2y agoSometimes I feel like I'm wasting my time by obsessively reading RFCs and specifications for the things I use. This made me feel better about that. And also much more smug. Other than the obvious moral that protocols should be implemented properly, the moral of the story is that all abstractions are leaky, and it will always be useful to understand the lower levels.
- pyuser583 2y agoFor female readers, the title might have a very different meaning.
- cat_plus_plus 2y agoCongratulation on upcoming new addition to your family!
- BrandonMarc 2y agoThis reads like a story from the daily wtf
- blueflow 2y agoYou can avoid this mess altogether by using the quoted-printable content encoding when generating emails.
- unilynx 2y agoHow so? Quoted printable doesn't require the dot to be encoded. Maybe you're thinking of base64 encoding?
- blueflow 2y agoDoesn't require, but comfortably allows you to do.
- unilynx 2y agoAgreed. But this is SMTP, I have no doubt there are gateways out there that will reencode the mail and put the dot back in the wrong place, eg in the name of wrapping all URLs behind a phishing-warning-page
- opello 2y agoI think the point here is that any subsequent implementation handling the message may well not be susceptible to the problem thus avoiding the erroneous behavior.
- int_19h 2y agoIt can handle it fine in that sense, but still unquote it before sending it over to the next node. Which might have the bug.
- marcosdumay 2y agoSo... SMTP client misses basic SMTP functionality that fits in the 20 lines summary. Good luck having it handle any of the SMTP craziness that isn't on the short introduction to the protocol.
- banish-m4 2y agoWhen you roll your own library duplicating something that likely already exists, you own it all. And that doesn't include all of the vendor-specific nonconforming edge-cases.
- marcosdumay 2y agoI guess when you roll your own library, you get an obligation to read the introduction of the 10 pages long standard. Or the usage example.
- evmar 2y ago> A portion of this code implemented a SMTP client. If I wanted to root cause this, the real problem is right there. Implementing protocols correctly is hard and bugs like in the post are common. A properly implemented SMTP client library, like one you would pull off the shelf, would accept text and encode it properly per the SMTP protocol, regardless of where the periods were in the input. The templating layer shouldn't be worrying about SMTP.
- TeMPOraL 2y agoThe real problem isn't the protocol, but the cowboy approach to interacting with it. It's not hard to "accept text and encode it properly per the SMTP protocol", you just need to realize you need to do it in the first place. There is a multitude of classes of errors and security vulnerabilities, including "SQL injection", XSS, and similar, that are all caused by the same mistake that this case of missing period was[0]: gluing strings together. For example, with SQL queries, the operation of binding values to a query template should happen in "SQL space", not in untyped string space. "SELECT * FROM foo WHERE foo.bar = " + $userData; is doing the dumb thing and writing directly to SQL's serialized format. In correct code (and correct thinking), "SELECT * FROM..." bit is not a string, it just looks like one. Same with HTML templating[1] - work with the document tree instead of its string representation, and you'll avoid dumb vulnerabilities. So, if you want to avoid missing dots in your e-mails, don't inject unstructured text into the middle of SMTP pipeline. Respect the abstraction level at which you work. See also: langsec. -- [0] - And therefore should be considered as a single class of errors, IMO. [1] - Templating systems themselves are thus a mistake belonging to this class, too - they're all about gluing string representations together, where the correct way is to work at the level of language/data structures represented by the text.
- mananaysiempre 2y agoFor what it’s worth, some markup-first template systems have tried to respect the target format’s structure—Genshi[1] and TAL[2] come to mind, and of course XSLT (see also SLAX[3]). I said “markup-first” so that the whole question isn’t trivialized by JSX. [1] https://genshi.edgewall.org/wiki/Documentation/xml-templates.html https://genshi.edgewall.org/wiki/Documentation/xml-templates... [2] https://zope.readthedocs.io/en/latest/zopebook/AppendixC.html https://zope.readthedocs.io/en/latest/zopebook/AppendixC.htm... [3] https://juniper.github.io/libslax/slax-manual.html https://juniper.github.io/libslax/slax-manual.html
- banish-m4 2y agoDot stuffing required SMTP https://www.rfc-editor.org/rfc/rfc5321#section-4.5.2 https://www.rfc-editor.org/rfc/rfc5321#section-4.5.2 Also in POP3 https://www.rfc-editor.org/rfc/rfc1939#page-8 https://www.rfc-editor.org/rfc/rfc1939#page-8
- jiveturkey 2y ago> This meant some customers received emails informing them their new premium was now $2700 instead of $27.00. there's a secondary issue here, why in the world would you auto split a monetary value across a numeric decimal indicator? why would you split lines at all for this use case?
- kccqzy 2y agoFrom the article: > The maximum total length of a text line including the <CRLF> is 1000
- HeatrayEnjoyer 2y agoI do not understand what you are asking. "$27.00" is a standard expense format.
- hunter2_ 2y agoI think GP was was using the phrase "auto split ... across [character]" in reference to characters that can cause line breaks for "word wrap" purposes in page layouts. For example, a normal space is a character that causes line breaks, but a non-breaking space (nbsp) is not. A hyphen, a tab, a zero-width space (zwsp), and several other characters are also generally used for line breaking. I think GP is saying that the decimal indicator -- the fourth character of "$27.00" -- should not be used for breaking. I think GP assumes that the problematic line breaking in TFA is akin to the type of "word wrap" page layout logic I've just explained; in reality the line breaking in TFA has nothing to do with that, it's simply breaking at 1000 octets (probably for reasons of buffer size, certainly not page layout) regardless of what character is in that position, so this whole thing is moot. GP needs to RTFA!
- Aloisius 2y agoIf the period was the 999th character in the line, it would split it to the next line since the maximum line length in SMTP is 1000 characters including CRLF.
- pwg 2y agoFrom TFA it is stated that they were doing the split of lines because of the "1000 octet" maximum line length requirement of the SMTP protocol. And, they also state that the period disappeared because it was placed at the start of the next line when the split occurred. From which one can deduce that they were doing the most basic "split" possible, splitting at the exact 1000 octet point, i.e. something like: if (length(line)>1000) then: line1=string_range(line,0,999) line2=string_range(line,1000,end) fi And if the period in 27.00 ended up exactly at offset 1000 in "line" then it got 'split' into line 2 as the first character of line2.
- jeffbee 2y agoIf a person has never heard of dot stuffing they're never going to believe what other horrors lie within the email space. Header folding, quoting in the local part, ipv6 literals, etc.
- blueflow 2y agoWhats the issue with ipv6 literals?
- jeffbee 2y agoEverything that is normally annoying with IPv6 literals, plus the fact that it's encapsulated as [IPV6:<the address>] and the address could take a dumb form such as ::1.2.3.4. But I mentioned it because it might initially seem to a neophyte in this field that the thing to the right of the rightmost @ sign is a name you can pass to your resolver. It might not be.
- kazinator 2y agoThe IPv6 notation claims : for separating digits. But that's already an established notation for port numbers: 10.1.2.3:456. Oops! The square brackets allow us to stick a port number on it: [ffff::0123:4567]:6301
- jeffbee 2y agoYes, but the RFCs permit 6v4 addresses such as ff:ee::aa:1.2.3.4
- jcranmer 2y agoNote that IPv4 address literals are encoded in square brackets (e.g., user@[127.0.0.1]), so it's really all IP address literals that could be a problem rather than specifically IPv6 literals.
- teddyh 2y agoI see two huge bad habits here. The first is the obvious one, as pointed out by many commenters here: Don’t implement standards haphazardly, if you even should do so yourself. Either give the implementations the necessary care and attention, or use a pre-made library. But the other thing is: Don’t vendor your dependencies. Those libraries you use need to be updated regularly and timely, and absolutely not “only as necessary”. If updates lag behind or are avoided entirely, bugs like this can be huge problems even when the upstream code has been fixed, for people who thought that they should update only when they, themselves, see a problem or need.
- the_real_tjaart 2y ago> Don’t implement standards haphazardly, if you even should do so yourself. Either give the implementations the necessary care and attention, or use a pre-made library. I agree 100%.
- ryandrake 2y agoFamous last words at >50% of the companies I've worked: "Just implement as much of standard X as you need to ship an MVP of feature Y!"
- rrr_oh_man 2y agoCan you share a story?
- maxbond 2y agoI had a very difficult to track down bug that ended up losing us a very big fish client, that came down to parsing a binary file with an ad-hoc parser that looked for the index of a header. They didn't realize that after the header was some metadata, so a small amount of metadata was interpreted as data. I fixed it by writing a proper ad-hoc parser that actually worked on a header-by-header level. But the damage was done and we had looked like buffoons to the client. Was very hard for the team. There was a shouting match over it, some hard feelings. The code was written in the spirit GP alludes to by an enthusiastic executive who wanted to help lighten the load. I should've rejected the PR but was intimidated to reject the exec's code (not an engineering reason for an engineering decision!). The exec was a good data scientist but not as strong a coder as me, and parsing binary files is one of my specialties. Friends don't let friends parse using "indexOf()".
- nytesky 2y agoWasn’t this an episode of Silicon Valley?
- m3kw9 2y agoFor a second I thought this was the health forum
- kazinator 2y agoBut the line "We are happy to welcome you to our family." is not anywhere near the line limit. There is something else going on here, like perhaps the whole thing actually being an HTML MIME attachment, perhaps? IN which it is like ... lots of text ... <br>We are happy to welcome you to our family.<br> or whatever. But if you blindly split HTML into lines, it will break tags.
- canucker2016 2y agoand probably the vast majority of the people receiving those broken HTML emails will realize there's something wrong with the email's formatting, chalk up the problem to a company that can't be bothered to correctly write a legible email and lower the company's competence rating, and go on to the next email. the company is just blindly unaware to their minor problem.
- dylan604 2y agorarely do I hold the sins of an incompetent marketing department against the company itself. otherwise, there'd be no company left deemed as competent because all marketing departments are incompetent.
- Aloisius 2y agoQuoted-printable encoding with soft breaks would allow blind splitting of lines without breaking HTML tags. While quoted-printable is supposed to have a max line limit of 78 characters including CRLF rather than 1000, email clients tend to be permissive.
- the_real_tjaart 2y agoThis was just an example, sorry for not providing a real example that had the exact character count.
- davidmurdoch 2y agoLoved reading this. This is now one of my new favorite bug hunt stories!
- rrr_oh_man 2y agoI found it quite mild compared to the 500 mile email https://www.ibiblio.org/harris/500milemail.html https://www.ibiblio.org/harris/500milemail.html
- the_real_tjaart 2y agoThanks for sharing!
- davidmurdoch 2y agoThat's my favorite.
- dankwizard 2y ago[flagged]
- davidmurdoch 2y agoYou disagree that I loved the article? Or that it's one of my new favorites? (I'm only messing with you as you criticized the author's use of language then replied to me with imprecision.)
- the_real_tjaart 2y agoI am glad you enjoyed it.
- kazinator 2y agoWhy would a cron job that sends e-mails need to implement its own SMTP client??? You just use the mail program from mailutils or whatever. Just from a point of view of deliverability, developing bare bones SMTP interaction over a socket is a nonstarter. You can't just connect to random mail exchange hosts directly and send mail these days. A solution has to be capable of connecting to a specific SMTP forwarding host (e.g. provided by your ISP). For that, you need to implement connections over TLS, with authentication and all. Also, a slightly ironic thing is that cron already knows how to send mail. The output of a cron job is mailed to the owner. Some crons let that mail address be overriden with a MAILTO variable in the crontab or some such thing.
- ummonk 2y agoJust another example of Zawinski's law, no?
- onlyrealcuzzo 2y ago> Zawinski's Law captures common market pressure on software solutions, stating that “every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.”
- sjf 2y agoThe modern version would seem to be every platform expands until it includes instant messaging.
- easyThrowaway 2y agoI believe that's outdated already, the next...uh, expansion layer would be some sort of generative AI features. In other words, every other platform expands until it can summarize emails.
- tantalor 2y agoIM is just fancy email.
- jayceedenton 2y ago> A portion of this code implemented a SMTP client. What the...
- eschneider 2y agoI guessed the cause of the problem (well, "leading period") from the description, but that's because I've experienced a lot of pain in my life...
- xmjw 2y agoHoly shit. This is one of the 2 bugs in my career I never solved. (That I knew about, and lost sleep over, etc…)
- tuck1s 2y agoThe need for dot-stuffing is a side-effect of email's use of the period as part of the termination sequence CR LF . CR LF. Mishandling of that sequence has led to other recently discussed bugs. https://smtpsmuggling.com/ https://smtpsmuggling.com/
- bhaney 2y agoThis made me realize I have the opposite problem. Now I have to go update the toy SMTP server that I ended up implementing in a perl script so it handles SMTP clients double-dotting a line.
- readthenotes1 2y agoAll this about a missing period and nothing about the consistently missing commas. In fact I would say leaving off punctuation on one line paragraphs would be more consistent.
- aftoprokrustes 2y agoI will not comment on the technical part, as others already did it better than I could, but it just reminded me of an anecdote that reminds of the importance of such trivial things as a period at the end of a sentence: In Germany, where I work, it is usual at the end of employement to ask for a letter of recommendation ("Zeugnis") that lists the tasks performed, and how good the employee was. It is an important document, as it will typcally be required when applying for jobs. Obviously, no employee would accept a document explicitly stating "this guy is a lazy bastard, do not hire him", so there is a "Zeugnissprache", a "secret code" to disguise this information as praise. One part of this code is that a missing period in the last sentence means "please ignore everything said here, this guy is horrible". How do I know? I let a lawyer check my Zeugnis after my last employment, and (I assume out of lack of care, as all my performance reviews were positive) the last sentence was missing the period.
- meisenhus 2y ago[dead]
- mwigdahl 2y agoDid you check your performance reviews to see if they were missing periods also?
- pests 2y agoWhy would they need to keep it secret if its a review and internal-only?
- lisper 2y ago> One part of this code is that a missing period in the last sentence means "please ignore everything said here, this guy is horrible". Gee, what could possibly go wrong?
- Biganon 2y agoSecret codes being used in recommandation letters are an urban legend. HR people have no incentive to create a secret code for them and their potential rivals, let alone teach it to new HR people while also keeping it secret. This legend comes from the fact that HR people cannot be too explicit about the fact that you've been a pain in the ass (you could probably sue if it's too transparent), so if they have nothing positive to say they will commend your punctuality or something equally as mundane. It's not secret codes, it's like... "bless their heart", but in HR talk. Plausible deniability if you want to sue, I guess. "But it's a good thing, your honor! They were always on time!"
- irrational 2y ago> Seeing that the SMTP client code was borrowed from a previous project we thought it good to let our other teams know about this bug in case they needed to patch it as well. They thanked us and we called it a day. As soon as I read the above, I knew the below would be the result. > It seems one of our other teams haven't gotten around to patching this bug in their code.
- bufordtwain 2y agoWhat about the missing comma at the end of the first line? :)
- the_real_tjaart 2y agoWhoops, :)
- deleted 2y ago[deleted]
- readyman 2y agoThe title sounds like a pregnancy scare
- billy99k 2y ago[flagged]
- vsuperpower2020 2y agoI'm sorry to hear that but this is not really the time or the place for personal stories.
- billy99k 2y agoIt was a joke. Lighten up.
- bruce343434 2y agoAs someone who configures email servers for a living (among other things): email needs to be replaced, frankly. This is a stupid protocol with even stupider file formats. What is the reason for such a hard coded line limit? It's just a stream of bytes... Not to mention all the weird bandaids on top of bandaids to try to get sender verification and tamper proof emails working. That alongside the complete lack of end to end encryption. It's just an incredibly unpleasant tech stack from top to bottom, through and through. The amount of moving parts/pieces of running software needing to cooperate just right to even function as a simple outgoing-only mail server is too damn high.
- vardump 2y agoYou’re not wrong, but how would you suggest to accomplish this? Replacing whole email infrastructure seems nearly impossible.
- bruce343434 2y agoMy hope is that people move away from email as they get tired by it, and towards more convenient instant messaging platforms which pretty much get the feature set right (attachments, encryption, blocking/spam provisions, provenance, "stories" and special group chat modes for broadcasting) and for which open protocols arise ((are being forced by the eu)): https://www.theverge.com/2024/2/6/24063705/whatsapp-interoperability-plans-eu-dma https://www.theverge.com/2024/2/6/24063705/whatsapp-interope... In the future, I hope that email will be like fax, or at least treated like http in comparison to instant messaging (https).
- linsomniac 2y agoShow of hands: Who knew exactly where this was going as soon as "SMTP" was mentioned?
- barryrandall 2y agoI first encountered this when I was writing a WAP[1] mail client in ColdFusion[2]. I must have read the entire SMTP spec 20 times before I spotted my mistake. [1] https://en.wikipedia.org/wiki/Wireless_Application_Protocol https://en.wikipedia.org/wiki/Wireless_Application_Protocol [2] https://en.wikipedia.org/wiki/ColdFusion_Markup_Language https://en.wikipedia.org/wiki/ColdFusion_Markup_Language
- layer8 2y agoAs soon as "missing period" was mentioned I suspected it would be about SMTP.
- carimura 2y agoGood story. Reading this reminds me of all the "curious cases of....." that I've solved (or in some cases not solved) over my career and how that feeling of triumph is so deeply tied to why I got into computers in the first place. The pure joy of unraveling the mysteries of engineering... like forgetting a semi-colon somewhere in a 50k LOC Perl backend.
- the_real_tjaart 2y ago> how that feeling of triumph is so deeply tied to why I got into computers in the first place I share your sentiment, thank you for reading.
- sethammons 2y agoI recognized the problem on sight; we solved this exact issue but since it was MTA software we knew about periods being special. Unfortunately, people are often solving problems that many others have solved. Maybe AI will allow the lines to be connected or solve for known edge cases like a dot in the smtp data
- genewitch 2y agoyou know, now that you mention this, i think i have heard of this exact problem before, perhaps from the late 90s; the "full stop on a line by itself" itched a bit, probably for that reason as well.
- camel_gopher 2y agoI have two kids. The case is not curious at all.
- userbinator 2y agoI suspect a lot of people are no longer being taught these fundamental protocols by manual interaction with a terminal, since that's what SMTP seems to have been originally intended for; and as someone who actually made use of that for a nontrivial amount of time, the "single dot on a line" to end a message has been permanently etched into my memory. Relatedly, escaping somehow seems to be a foreign concept for a lot of programmers, who wouldn't ever see the above situation and ask themselves "but what if I want to send an email with a line containing a single dot?" yet another large group of them finds it perfectly logical and easy to understand.
- edanm 2y ago> I suspect a lot of people are no longer being taught these fundamental protocols by manual interaction with a terminal, I'd be surprised if any significant portion of software developers learned anything like that in the last 30 years, at least.
- Maxion 2y agoI mean, if we did that for everything in webdev, It'd take 30 years of training before you'd get to do your first PR!
- Learner100 2y ago- The SMTP client they implemented could insert a newline such that a line was comprised of only a single period. - The SMTP client spec says that an additional period would be added here. - The SMTP server spec says that it would remove this additional period, bringing us back to one period. I don’t get how this led to there being no period at all. Am I missing something?
- roer 2y agoThe spec says that an additional period should be added on the client, but their implementation did not do that.
- Learner100 2y agoAh, thanks!
- EVa5I7bHFq9mnYK 2y agoGiven that the words Night and Club appear in the picture, the title looked mildly intriguing ..
- davidwritesbugs 2y agoAs I'm implementing an NNTP server based on RFC specs I knew instantly what was happening here without RFA. Dot stuffing, yea 80s protocols baby.
- edweis 2y agoI like the simplicity of the email membership price increase.
- croes 2y ago>the first character is a period and there are other characters on the line, the first character is deleted. Why is it implemented that way? If a single period means end of mail then more than a period means it's mail data. Why deleting the period in the first place? Couldn't they store one byte to check the next?
- wruza 2y agoCause you have to send a line with a single period somehow. . Ends body .. One period <-- ... Two periods .A A ..A .A A A Anyway that’s stupid and only helps if you compose your email right in a tcp session.
- layer8 2y agoSMTP = Sporadically Missing Trailing Period ;-)
- 256_ 2y ago/SMTP/SMLP/ SMLP = Sporadically Missing Leading Period (-;