7 ms·
Email: Explained from first principles
- t3estabc 4y ago[dead]
- marcopicentini 4y agoGreat project. Why not improving the existing Wikipedia page for Internet, email etc..? Wikipedia is more likely to stay online forever and indexed better
- KasparEtter 4y agoThis is a valid question. I'm the author of the blog and I've asked this myself in the past. The main reasons why I haven't done this so far are - artistic freedom (regarding the form/structure, the style/tone, the interactivity), - the process (I prefer not to have long discussions and unexpected revisions; but I must admit that I should collect actual experience in this regard), - and the lack of attribution (part of it is certainly vanity, but I'd also like to make a living out of this at some point).
- philistine 4y agoThe content is CC BY 4.0; in a sense it's not that you're not contributing to Wikipedia, you're simply at the very first step of delegating that task!
- hilbert42 4y agoThis is the best simplified description of email and the email system I've read. It's simple to understand and fully comprehensive—with the exception of RFCs of course (but often they aren't easy to understand). A highly recommended read/reference whether you're an email neophyte or one who builds email clients or server software.
- riffic 4y agowhat is a "first principle" and why does it matter here in an explanation of email?
- owlglass 4y agoIt's an approach to reasoning and thinking used in philosophy and science [1]. It could be applied to any concept. The author's articles and those by Bartosz Ciechanowski [2] are good examples of first-principles explanations of various concepts. [1] https://en.wikipedia.org/wiki/First_principle https://en.wikipedia.org/wiki/First_principle [2] https://ciechanow.ski/ https://ciechanow.ski/
- macintux 4y agoI once gave a talk about Erlang starting with the core idea that = is an assertion of truth and building (most of) the rest of the language from that. Found it a very rewarding approach.
- schoen 4y agoIs there any way for the public to see this talk or learn more about this?
- macintux 4y agoHere’s the Midwest.io talk. Shame the conference didn’t last. https://youtu.be/E18shi1qIHU https://youtu.be/E18shi1qIHU
- benatkin 4y agoThat's why I don't like it when languages use = for key: value pairs like TOML. There seems to be a big meaning of "=" that people forget.
- KasparEtter 4y agoI chose the name of the blog to mean analytic, reductionist reasoning (in contrast to associative, vague reasoning). In the meantime, I also like the following framing, which I wanted to add to the front page of the blog for some time: https://www.cold-takes.com/minimal-trust-investigations/ https://www.cold-takes.com/minimal-trust-investigations/
- jabroni_salad 4y agoI don't have any kind of pedigree with email but this page has been very helpful to me over the past year. Every now and then something that seems weird or esoteric makes it to the ticket board and having this explain how email in general is supposed to work as opposed to anything vendor-specific let me take cases that the rest of the consultancy passed up on. I also really like the website's software. I wish all the technical writing I have to deal with was presented in this way.
- hit8run 4y agoI read the rfcs whenever I need I need some more understanding. They are quite good and readable.
- a-dub 4y agonicely done! especially liked the clear discussion on the nuances involving smtp envelopes vs. message headers and the concise discussions of extensions like spf, dkim and rbl.
- albertgoeswoof 4y agoIf you’re going to learn anything, email is a good bet. It’s barely changed in 30 years, and I suspect it will not change much for a long time. I built my own transactional email provider (https://mailpace.com https://mailpace.com), I wish I had found this link before I started. I would also recommend reading the email RFCs, they’re not that difficult to understand and the history explains a lot. Email aside we can learn a lot from this kind of distributed, decentralised system design in the future.
- klabb3 4y ago> Email aside we can learn a lot from this kind of distributed, decentralised system design in the future. Nit for other people curious about email: email isn’t decentralized by the popularized use of the term, both in theory and more so in practice. Email relies on DNS which is federated. That doesn’t mean that it’s worse, federation is a proven-to-work trade off which provides many of the benefits of decentralization, (arguably the most important parts). As for practice, email relies on opaque reputation systems to prevent abuse, which acts as a centralizing force. I’m sure you know (better than me) that the barrier of entry to sending email on your own is quite a bit above zero.
- codetrotter 4y ago> Email relies on DNS which is federated To a great extent, yes. If I give my public email address to someone and expect them to be able to send me email from anywhere at anytime, they will almost certainly have to rely on the DNS that exists like we know it. But the situation can be slightly different in various ways: - If our mail servers have globally routable IP-addresses permanently assigned to them, and are set up to accept mails where the IP address is used directly, we could use the IP addresses instead of domain names. But that's still similar to the situation with domain names, since it's still ICANN at the top for that too. - In a LAN or a VPN we could exchange email between hosts using either DNS that is completely under our control, or we could use mDNS on it which is also under our control, or in the case of either really small networks or a short time span we could even rely on the IP addresses our mail servers have on said LAN or VPN and send mail and again that would be completely under our own control as well. - Since Tor offers a base for TCP connections, every TCP based protocol can operate on top of it. So if we both use Tor connected email servers then we could exchange emails that way. Our email clients could connect to our email servers over clearnet without having to be aware of Tor, and we could use email addresses like bob@fastrcl5totos3vekjbqcmgpnias5qytxnaj7gpxtxhubdcnfrkapqad.onion and alice@mysitenameyx4fi3l6x2gyzmtmgxjyqyorj9qsb5r543izcwymle.onion In these ways, e-mail could still be regarded as decentralized. As well as in a simpler way that as opposed to for example Facebook where there is one company that owns the whole system and oversees and decides who gets to have an account on the system or not, you and I can always find another e-mail provider to give us a new account and we can communicate with others from those new accounts. And in fact, if we were willing to do it, we could change how we set up our e-mail servers so that we have the server accept mail from any other e-mail server but with the condition that the mail was encrypted with a known PGP key. Now imagine that you and I set up our mail servers like this and we exchange PGP keys. Then even if I lose access to my original email account that I had told you about, I could mail you from a different account with another provider using the same PGP key, and your server would recognise that the mail was from me and accept it and then you could respond back. But what if we both lose access to our accounts at the same time? If we plan ahead we could make not just one account each but a number of accounts on a number of different mail servers that we both tell each other about. That would give us quite a lot of wiggle room.
- layer8 4y agoI love such thorough expositions, although it isn’t quite as thorough as I would like. Two examples that caught my attention in the parts I read: 1. It doesn’t mention the old convention for writing an email address with display name as john.doe@example.net (John Doe) instead of: John Doe <john.doe@example.net> 2. It states that subject prefixes like “Re:” have no technical relevance. That’s not entirely true, because email clients recognize existing prefixes when replying/forwarding, in order to not add a redundant one. There are several issues here: - Localized email software sometimes uses a different prefix than “Re”, based on the local language. This is an issue when having an email thread between email clients who don’t recognize each other’s local-language prefixes. (Arguably, it would be better for everyone to stick with “Re” regardless of language.) - Some email clients have the convention of adding a count to the “Re”, e.g. “Re[2]:”, “Re[3]:”, and so on. When you write an email client, you may want to consider recognizing those. Due to the variety in subject prefixes, some email clients allow users to configure a regex.
- codetrotter 4y ago> This is an issue when having an email thread between email clients who don’t recognize each other’s local-language prefixes. https://en.wikipedia.org/wiki/List_of_email_subject_abbreviations#Abbreviations_in_other_languages https://en.wikipedia.org/wiki/List_of_email_subject_abbrevia... RE:SV:RE:FWD:AW:FWD:AW:FWD:REF:RIF:Verbale della riunione, venerdì 7 :p But yaeh, I think the big email clients are good at recognizing these across different languages.
- hummus_bae 4y ago[dead]
- sbuk 4y agoEmail clients don’t really have a spec to follow other than parsing messages that are received via POP or more likely IMAP these days (not forgetting MAPI either). Adding FWD or RE to a subject field doesn’t appear anywhere in the email message RFCs, starting with 822. As such they are undocumented conventions or extensions to the specs, and as the article points out technically irrelevant - no MDAs, MTAs or MUAs that do not “support” features that utilise these conventions will fail to parse a message.
- denton-scratch 4y agoOw, it seems heavy going! I mean, it's good stuff; detailed, well-organized, and accurate (as far as I can see). But in my mind, it just shouldn't be such hard work to understand. The title's clear: "from first principles". I can't complain! I'm only about a quarter of the way through; I need a rest now. I may edit later.
- jeffrallen 4y agoIn order to check that document's detail level, I went and searched for "envelope". Understanding the difference between envelope To and the To header is critical to thinking about email routing. (Thank you Arnold, for teaching me that.)
- tuhinnair 4y agoI think this explanation does a great job of showing just how large the bottom of the Email iceberg is. When I first worked with Email, I was overwhelmed by the work required to handle all the variation. I didn't even try to run my own mail server, I was just trying to parse and store email in a structured form. What I thought would be a quick evening's work turned into a full week. While building Hypermail [0], I actually resorted to using GPT-3 with some of the parsing. It did extremely well in handling variations in email reply formats (email clients all have their own way of doing things). [0] - https://www.idiotlamborghini.com/articles/the_hypermail_experiment https://www.idiotlamborghini.com/articles/the_hypermail_expe...
- agumonkey 4y ago> What I thought would be a quick evening's work turned into a full week makes me wanna write a book series titled like this. so often i overlook details and time required is 5-10x more :) also makes me wanna find a slightly deterministic way to asses this more precisely. i was trying to do vague dimensional analysis and see how many parts and relations there would be for any given ideas.
- ggm 4y agoEHLO Richie, Lionel is it 8-bit clean me you're looking for?
- shon 4y agoThis is how everything used to be taught. I was one of the first CCNPs and we learned networking from wire signaling up. 7 layers, starting with the physical. First principles is good.
- kuharich 4y agoPast comments: https://news.ycombinator.com/item?id=27086608 https://news.ycombinator.com/item?id=27086608
- dieselgate 4y agoI'm interested in this topic but didn't have the attention span to read the whole thing. One part of this website I found interesting is the light/dark toggle. Clicking on the button sends an http response and the page is loaded with the newly selected theme. Only mentioning this because it's something I've thought about implementation-wise but never actually coded out because there seems like other options that don't utilize an http request. Just cool to see it in the wild, though
- japanman425 4y ago[dead]
- peter_retief 4y agoI have always wanted to create an email based forms application using subdomains as variables and identifiers. My early experience was with Lotus Notes which is in essence an email application. Maybe one day if I get the motivation...
- Sai_ 4y ago> _using subdomains as variables and identifiers._ Can you expand on this? I think formspree.io does some version of this, if I understand your ask correctly.
- peter_retief 4y agoI will have a look thanks. As for my vaporware, the first obvious subdomains would be for company->location->department->person. For variables I would use ->xxx->uuid->yyy to create indexed threads for high speed lookups. Just an idea, but I do love mail.
- andris9 4y agoWell, I for one, hope that email stays as complicated as described in the post. Otherwise my project that simplifies access to email accounts (https://emailengine.app https://emailengine.app) would get no traction :D
- Oreko 4y agoSCRAM seems incredibly insecure. Does anyone have information on how many servers support SCRAM?
- KasparEtter 4y ago> SCRAM seems incredibly insecure. Why do you think so? > Does anyone have information on how many servers support SCRAM? I'd be interested in this as well. :-)