5 ms·
Executive summary: "Webization of mailbox access for developers who don't understand anything outside of the web bubble".
by kazinator 3y ago
Executive summary: "Webization of mailbox access for developers who don't understand anything outside of the web bubble".
- londons_explore 3y agoHave you tried to send SMTP email lately? Most residential ISP's block SMTP ports and all the encrypted variants of it. If you want something that works for everyone, it has to be port 443 and webby. And any mobile push notifications better be via Google/Apple notification services, because modern OS's don't let applications leave a hanging TCP connection open anymore.
- deleted 3y ago[deleted]
- kazinator 3y agoSMTP and IMAP are different things. Destination e-mail domains to which you want to send mail have SMTP servers. To send them mail, you have to contact their SMTP server somehow. If not directly, then through an intermediary. > If you want something that works for everyone, it has to be port 443 and webby. That doesn't follow. SMTP is already authenticated and SSL-protected (or can be). The port number isn't the problem, it's the spam. Suppose that I run an alternative mail server, so that you can send me mail over HTTPS on 443. If you're a residential subscriber, I'm still blocking you the same way. Only, it's harder now. I have only one static IP address, and have to serve web requests on it on port 443. So I can't block you at the IP level; I have to let you access my web site (open to all) but block your use of the specific API. That's more costly in terms of system resources; I have to let you connect, and do the SSL handshake and submit requests, instead of turfing your packets at the IP layer. > modern OS's don't let applications leave a hanging TCP connection open anymore That has been the case since BSD Unix introduced sockets. When the application quits, the connections close. (At best, there is a graceful shutdown in the background, where the protocol stack exchanges the final FIN segments.)
- School-Cotton 3y agoWhy are you trying to run an MTA on a residential network? I suspect you are confused about how things work. The standard SMTP port (25) is for communication among MTAs and yes, residential networks block it (as they should) as any home PC sending on it is 99.999999999% likely to be part of a botnet. Sending mail from a home PC to the outside world is done by submitting it to an MTA on the SMTP Submission port (587) which your email provider should have given you credentials to. Home ISPs don't block port 587 AFAIK.
- chrismorgan 3y ago> 99.999999999% Eleven nines is very excessive. Frankly even six nines would still feel excessive, though I’m sorry to say that five might not be any more. Fifteen years ago, I think even three nines would have been excessive. Legitimate outgoing SMTP was fairly common in those days, and botnet SMTP probably proportionally rarer. You know what, even one nine might have been excessive less than twenty years ago.
- kazinator 3y agoYou misunderstand the state of the e-mail world. Running an MTA on a residential network for receiving is completely fine. Sending mail directly from a residential network by connecting to the given mail domain is a nonstarter. Hasn't worked in twenty years or more. That doesn't require a MTA; only a MUA. Your mail program (MUA) like Thunderbird or Outlook, or the mail utility in a Unix-like OS, can perfectly well work without a configured SMTP server. You send to bob@example.com. The mail program will do a MX type DNS query for example.com and find out that the mail.example.com is the mail host for that domain. It will resolve that again with an A query to get an IP address, and connect to port 25 of that host to send the mail. In a world without spammers, that's how things would work. In a world with spammers, mail servers block connections from random addresses like residential IP addresses. Users (who want to send mail from home) have to resort to using forwarding hosts: their mail client is configured to make an SMTP connection not directly to a target mail domain, but to a configured host, like something run by their ISP or some commercial SMTP provider.
- gsich 3y agoNo, they don't. 465 and 587 should always be open. If not, complain, you have crippled service you pay for. Other MX might reject you on port 25 due to a residential IP, but that is not an ISP block.
- carlos22 3y agoThat's a bit harsh given all the problems SMTP and IMAP have. They are very dated and have some exotic "features". There is not a single mail client that gets everything right!
- deleted 3y ago[deleted]
- kazinator 3y ago> They are very dated The words "they", "are", "very" and "dated" are even older, so don't use them if you want to be hip.
- eschaton 3y agoIn other words, “they don’t sit atop the currently-fashionable implementation techniques that are the only thing webshits happen to know.” Well, no shit, they were designed before HTML and HTTP, not just JavaScript, and there’s zero need to actually shove them into that mold because they already exist and are fine as they are in terms of protocol-level features. If you can’t deal with that maybe you shouldn’t be trying to do development at that level. Meanwhile, all that energy spent reinventing the protocol layer to be maximally aesthetic to people for whom all the world is JavaScript running in a browser could have been used to actually improve the applications running over the protocol.
- mildred593 3y agoJMAP really solves many problems and allows you to do many things that are just not possible with IMAP. Also, IMAP has many ways to (badly) perform the same task, not all supported by all servers or all clients. MIME parsing is not easy to do and there is not always quality libraries available for your language... JMAP solves all of this.
- kazinator 3y agoJMAP doesn't solve any problems that I have. That's because I'm not a monolithic mail company trying to take a slice out of Gmail's pie.
- chrismorgan 3y agoTwo lines from a song I and rjbs wrote but never quite published (set to the tune of Major-General’s song): > Its JSON-centred roots and HTTP are conventional, > so special parsers are not needed: this is quite intentional. SMTP and IMAP are a pain and problematic to deal with for various reasons. Building on popular web tech primitives makes everyone’s lives much easier, even if you’re not making a webmail client.
- kazinator 3y agoSMTP and IMAP are a solved problem. They are problematic for only one kind of entity: someone who has delusions of becoming the next gmail. Everyone else can just set up an open source SMTP and IMAP setup using existing programs that work, without writing a single line of parsing code. I don't trust this "open standard" bullshit. When the only thing running JMAP is Fastmail, it's a Fastmail-specific protocol. If you want developers to interface with it, you can't hide it: you have to document what the requests and responses look like. The "open standard" rhetoric has good optics; even more so with the IETF rubber stamp. The underlying idea behind JMAP is that the future consists of only large mail operators who somehow forward mail among themselves. Routing to SMTP servers is gone, and so is self-hosting, mostly. If you want your own mail domain, you can do that, but to operate it, your instance must connect over some web protocol to an e-mail monopolist. The IETF may have signed off on this, but I look forward to what the EFF have to say.
- chrismorgan 3y agoSpeaking as one who has drunk the JMAP kool-aid (I worked for Fastmail for a few years): you sound like you’ve never tried to write an email client, or done much with email in general. Because the conclusions you’re coming to are completely back-to-front. Suppose you want to write a client that can do email sending (SMTP), email reading (IMAP), contact book (CardDAV) and calendar (CalDAV) interactions. SMTP/IMAP autodiscovery is limited at best, and CardDAV/CalDAV autodiscovery nonexistent, I think, so new users will have to find the details for several things, which is a big hurdle—or else you’ll have to recognise and special-case individual providers, which basically means Google and Microsoft get a massive unfair advantage. By contrast, JMAP handles autodiscovery of all this: type in an email address, and it finds the JMAP endpoint, which handles all of the stuff. (I will admit that this is currently more aspirational than real: to begin with it requires a DNS SRV record and/or a /.well-known/jmap HTTP response; and the means of authentication is deliberately unspecified at this stage, so we kinda need to wait around to see what providers settle on for authentication, with some kind of OAuth arrangement probably most likely.) And then perhaps your client is webmail; if so, you simply can’t use IMAP or SMTP. I guess you could try talking them over WebSocket or something like that (still requiring a proxy of some kind, even if a pass-through one), but that’d be painful; instead, everyone rolls their own thing, and so there’s never any webmail cross-compatibility. In JMAP, by contrast, the only thing you can’t do from a browser is looking up DNS SRV records. Everything else works, so that you can run your own webmail client that talks directly to the origin server. This allows you to disentangle the provision of service and software, which is very desirable for decentralisation. Well, now you want your client to produce notifications when new messages come in. If you’re writing for a desktop platform, fine: you can keep your IMAP connection open. But if you’re on a mobile platform, you probably can’t do that, and certainly can’t reliably, so… you’re stuck needing some kind of proxy and server component again. No good. But with JMAP, you set up a push subscription and it works just like any other app, without needing to keep a connection open. So there are three simple and concrete advantages of JMAP over the legacy protocols. JMAP actively bolsters decentralisation, in email providers, in domain names, in contact book and calendar support, in webmail, and in mobile notifications. There are other functional advantages too. JMAP enables some moderately complex batching that makes some sorts of realistic operations vastly faster than IMAP due to being able to skip many round trips. And JMAP’s pleasantness to work with directly should not be understated: no one wants to work with SMTP or IMAP directly because they’re painful protocols, and so of course all of that stuff tends to get extracted into a single library that protects you from them; by contrast, working in raw JMAP is pleasant and easy, which can genuinely open you up to doing things that would have been too painful to do through your IMAP library. And if you use JMAP, your mail client can skip worrying about MIME parsing (which is landmine-filled territory) if you choose. You can write a useful and complex standalone email workflow with nothing more than HTTP and JSON libraries. > The underlying idea behind JMAP is that the future consists of only large mail operators who somehow forward mail among themselves. Routing to SMTP servers is gone, and so is self-hosting, mostly. This is all completely wrong: JMAP is for client–server communications and doesn’t replace server–server communications, which still use SMTP. JMAP does not prop up email monopolists. > If you want developers to interface with it, you can't hide it: you have to document what the requests and responses look like. I have no idea what this is about. The spec shows examples. > The "open standard" rhetoric has good optics; even more so with the IETF rubber stamp. There was no IETF rubber stamp (which I take you to be using as a pejorative). There was collaborative process with people from a variety of companies who took what Fastmail had designed for their own use and wanted to share, and made it considerably better.
- eschaton 3y agoAmen, all of the actually-interesting features could have been done within the framework that MRC (RIP) laid down for IMAP, without going wholly into “screw line oriented text protocols, gotta be Web 2.0!” bullshit.
- kazinator 3y agoJSON is not "modern". * curly braces are not modern. * quotes around strings are not modern, and neither are backslash escapes like \n * commas and colons to separate items are not modern; square brackets for arrays are not modern * JSON's printed floating-point format which looks like 123.56E+78 can be seen in computing-related documents from the late 1950's. * HTTPS isn't new, and sending formatted data over web transport as an API is soon going to be approaching thirty; remember SOAP from the 1990's? The SOAP was handled by so many hands it shrank down to JSON.