17 ms·

JMAP – a modern email open standard

1030 points by tambourine_man 3y ago

tiffanyh 3y ago
Fastmail.com continues to delivery great innovations and open sources much of their work. They are the company largely behind this, check them out. (I'm not affiliated in any way to them)
- foxtacles 3y ago
  Fully agree. I'm also a very happy Fastmail customer of over 10 years now; I could never go back to gmail
  - stavros 3y ago
    Same here, whenever I have to open Gmail for work I'm annoyed.
    - arp242 3y ago
      You can connect FastMail to gmail and never look at gmail. Your employer may not approve if you hillary your emails like that though, but that's what I did at a previous employer where this wasn't an issue.
      - stavros 3y ago
        I could, but I don't want to look at work emails outside working hours.
        foxtacles 3y ago
        Have you considered setting up a rule using the Snooze feature[0]? Using this it should be possible to delay all incoming work email until a time that is suitable for you. [0]: https://www.fastmail.com/blog/fastmail-snooze/ https://www.fastmail.com/blog/fastmail-snooze/ Edit: If you want your work emails to arrive only during work hours and otherwise delay them, you can use a Sieve condition in your email rule similar to this: anyof (date :value "lt" "date" "hour" "09", date :value "ge" "date" "hour" "17")
        stavros 3y ago
        Well that's very interesting, thank you!
      - hackernewds 3y ago
        I like your usage of Hillary as a verb. I don't understand the affinity against Gmail though. With the fastmail forwarding service are you still not using Gmail spam service and have Gmail access to your email anyways (not that I'd believe they do unless wearing a tinfoil hat)
        arp242 3y ago
        I just don't care much for gmail's UI; that's all there's to it. And having all email in one place is more convenient for me regardless – saves having to open/check another service. You're still using gmail's spam checking; it just uses the API (or IMAP? I forgot) to sync the emails to FastMail.
      - psankar 3y ago
        > If you hillary your emails LOL
  - dbrunton 3y ago
    Came here (and used Fastmail for password recovery) to say just this. I actually forget what it’s like to sell advertising on my correspondence!
    - gersg 3y ago
      If you don't enable "smart" features in gmail you don't see ads either.
  - TylerE 3y ago
    I tried, but fm’s spam filtering is just hot garbage. Both false negatives and (more troublesome) false positives. It’s like stepping back in time 20 years when soamassassin with default config is as good as it gets.
    - deleted 3y ago
      [deleted]
    - UberFly 3y ago
      I experience the opposite of this, but I guess everyone's use case is different. They do use SpamAssasin and it seems pretty effective.
    - moozeek 3y ago
      I have the exact opposite experience and find Fastnail's spam detection quite superior. There is the occasional spam.email getting through - maybe 1 per day - despite the fact that I have quite a number of custom domains with catch-all email addresses. At the same time I really never have to check my spam folder for false positive, everything in there is literal spam so I forget checking it for weeks on end. Whereas on n Gmail I always need to check the spam folder, Google completely overdoes the filtering it and a lot of wanted emails end up in spam.
    - chrismorgan 3y ago
      I think I’m up to two false positives after six years, neither of which was in any way important (in fact, I kinda agreed with its judgement), and I’m up to 175 false negatives in the last 5⅓ years (of which 32 are from the last twelve months, which suggests a fairly steady rate across the whole time; it comes to about one every eleven days). Before that, I used Gmail (on the same address, for about six years), and my memory is that its false positive rates were around one every month or two, and some of them actually mattered; and its false negative rates varied: sometimes it’d go for a few months with none, then it might let through one or two per day for a few weeks. Certainly less consistent. In the last year I’ve also had to use Gmail via a couple of organisations on addresses with very low volume (less than one per day) which have never received any spam, and both have had at least two false positives.
    - 3y ago
  - ndr 3y ago
    I wish their android app supported offline access. It's a huge PITA when you're offline (say in a venue with no cover) and need to pull a ticket out of your inbox. I'm a customer and can't believe they've been in business for so long. I guess this feature will never come.
    - dogmatism 3y ago
      That’s what imap clients are for
      - the_pwner224 3y ago
        Unfortunately all the FOSS email clients for Android have horrible UI/UX.
        tristan957 3y ago
        Even K9 (now Thunderbird)?
        zilti 3y ago
        K9 is great
        synecdoche 3y ago
        Could FairEmail be of interest? https://email.faircode.eu/ https://email.faircode.eu/
  - qingcharles 3y ago
    20 years here. One of the only companies I don't have a complaint about.
- dheera 3y ago
  How are they funded, and are they profitable? I've been using Google Apps for email for 10+ years and I don't want my e-mail provider to die at the whim of some Sand Hill investor's emotions at a bay area house party.
  - scblock 3y ago
    You pay them.
    - dheera 3y ago
      Is that enough for their company to be alive? Many startups offer paid services but it isn't enough to cover their own costs and salaries in the first several years; everything depends on investors willing to fund them through that period.
      - Semaphor 3y ago
        FWIW, they are not a startup, they are 24 years old and (after being bought by opera and then buying themselves out again) fully independent [0]. They are also probably HN’s most beloved company, but that just as a side ;) [0]: https://en.wikipedia.org/wiki/Fastmail https://en.wikipedia.org/wiki/Fastmail
        deleted 3y ago
        [deleted]
        hackernewds 3y ago
        > (after being bought by opera and then buying themselves out again) that history does not assign to me confidence in them as custodian of my emails
        Semaphor 3y ago
        This was in 2010, not the opera you know today.
- taftster 3y ago
  Would love to use them. But the pricing is too much. I need a custom domain (or three). And I have a family of 6 people. So that's $5 / user / month = $30 / month = $360 / year. Unfortunately, that's not justifiable for me. And thus why I'm stuck using Gmail with a custom domain.
  - WordSkill 3y ago
    "You can build an account where you mix and match the plan level to each user. For example, you can build a Fastmail Family by adding additional Standard and Basic plans for adults and children." - https://www.fastmail.com/pricing/ https://www.fastmail.com/pricing/ Only one of your accounts needs to be Standard tier ($50/y) to allow up to 100 custom domains. The other 5 accounts can be Basic tier ($30/y) and still use as many custom domains as you give them access to. Total undiscounted cost for your family would be $200/y. Use the 25% 1Password discount to reduce your total to $150/y. You have the option to extend that 25% discount up to 3yrs and receive an additional discount for paying in advance. Your total for your first 3 years would be $420, then $560 for each 3 years thereafter. Alternatively, you could consider allowing all your family members use the same Standard account ($105 for the first 3yrs) but with multiple email addresses. Use labels to surface each address for the relevant family member. In my case, I created a family@surname address for any family business, and a name@surname address for each family member. Each family member has set their Fastmail app to focus on their own address. Yes, technically, they can see each other's email, but it hasn't been a problem. I presume they use throwaway gmail accounts for stuff they want to keep totally private. We share a specific custom domain for our masked emails and love that feature, especially the 1Password integration although that has some rough edges.
  - kstrauser 3y ago
    Apple has an incredibly cheap option here. If you subscribe to iCloud+ (https://www.apple.com/icloud/ https://www.apple.com/icloud/), starting at $1/month, you can share one or more custom email domains with up to 6 people total. I've been using this for a year or so and my family's been happy with it.
    - taftster 3y ago
- mannycalavera42 3y ago
  it's a shame they don't have (yet) a EU hosting :(
- pazimzadeh 3y ago
  Infomaniak is good too https://www.infomaniak.com/en/hosting/service-mail/ https://www.infomaniak.com/en/hosting/service-mail/
kazinator 3y ago
Executive summary: "Webization of mailbox access for developers who don't understand anything outside of the web bubble".
- londons_explore 3y ago
  Have you tried to send SMTP email lately? Most residential ISP's block SMTP ports and all the encrypted variants of it. If you want something that works for everyone, it has to be port 443 and webby. And any mobile push notifications better be via Google/Apple notification services, because modern OS's don't let applications leave a hanging TCP connection open anymore.
  - deleted 3y ago
    [deleted]
  - kazinator 3y ago
    SMTP and IMAP are different things. Destination e-mail domains to which you want to send mail have SMTP servers. To send them mail, you have to contact their SMTP server somehow. If not directly, then through an intermediary. > If you want something that works for everyone, it has to be port 443 and webby. That doesn't follow. SMTP is already authenticated and SSL-protected (or can be). The port number isn't the problem, it's the spam. Suppose that I run an alternative mail server, so that you can send me mail over HTTPS on 443. If you're a residential subscriber, I'm still blocking you the same way. Only, it's harder now. I have only one static IP address, and have to serve web requests on it on port 443. So I can't block you at the IP level; I have to let you access my web site (open to all) but block your use of the specific API. That's more costly in terms of system resources; I have to let you connect, and do the SSL handshake and submit requests, instead of turfing your packets at the IP layer. > modern OS's don't let applications leave a hanging TCP connection open anymore That has been the case since BSD Unix introduced sockets. When the application quits, the connections close. (At best, there is a graceful shutdown in the background, where the protocol stack exchanges the final FIN segments.)
  - School-Cotton 3y ago
    Why are you trying to run an MTA on a residential network? I suspect you are confused about how things work. The standard SMTP port (25) is for communication among MTAs and yes, residential networks block it (as they should) as any home PC sending on it is 99.999999999% likely to be part of a botnet. Sending mail from a home PC to the outside world is done by submitting it to an MTA on the SMTP Submission port (587) which your email provider should have given you credentials to. Home ISPs don't block port 587 AFAIK.
jeffbee 3y ago
The fact that JMAP has ~zero adoption outside Fastmail itself contradicts the "much needed" part of this title.
- Nextgrid 3y ago
  The problem is that email has been taken over by a handful of providers, all of which have "growth & engagement" as their business model and would rather not implement any open protocol at all. IMAP remains supported for backwards-compatibility, but I'm sure its shortcomings are seen as a feature to push users towards the official clients so there is no incentive to implement another open protocol to address them.
  - stonogo 3y ago
    This is a problem, but the other problem is the tools to implement the service. You can use Cyrus, Apache James, or Stalwart, and that's it. These tools do not integrate with anything; in the case of Stalwart (the only JMAP server not in "experimental" status) you can't just install a JMAP server attached to an existing email service; you have to hand it complete control of everything, even unto the on-disk storage of mail. This is a complete non-starter for any email service outside of hobby/personal hosting. In other words, even if someone wanted to make JMAP available to their users, they'd have to tear the whole thing down and rebuild it in JMAP's image, and the tools available to do so are not fully-featured or integratable enough to do so.
    - StalwartLabs 3y ago
      > These tools do not integrate with anything; in the case of Stalwart (the only JMAP server not in "experimental" status) you can't just install a JMAP server attached to an existing email service; you have to hand it complete control of everything, even unto the on-disk storage of mail. This point you mention is being improved. The next release of Stalwart JMAP (expected in one or two months) will delegate user management to either a SQL database or an LDAP directory. Messages will be stored in either Maildir or MinIO/S3. And all other information will be in either SQLite or FoundationDB. All these features were already implemented except MinIO/S3. Development progress can be tracked at https://github.com/stalwartlabs/mail-server/tree/main/crates/jmap https://github.com/stalwartlabs/mail-server/tree/main/crates...
deleted 3y ago
[deleted]
ashton314 3y ago
Does anyone know of a JMAP -> Maildir client? I'd love to replace mbsync with something faster.
- rockorager 3y ago
  There aren’t any that go direct to maildir. There is mujmap that syncs to notmuch. I wasn’t able to get it to work the way I wanted, and wrote my own that I’d be happy to share. I’ve been using it for around a month with no issues but that’s certainly not much confidence yet! The advantage of going straight to notmuch is that the data model is so similar to JMAP that synchronizing is just that much easier. Both ends can give you a state and all emails which have changed from that state. Then it’s just a matter of updating the tags on those to each end or uploading/downloading the email if it’s new. I get something like 1.7kB round trip data usage when there are no updates (really it’s 0kB save for the keepalive when used in daemon mode) and ~100ms for that round trip. This is in comparison to several megabytes for a single mbsync poll (even when there are 0 changes) and ~5 seconds
feldrim 3y ago
I see that the specs are defined in 2019 excluding the Websockets spec in 2020. So over 3 years, these specs are implemented almost only by Fastmail. It feels like it is yet another standard [0], which could not be adopted by neither commercial nor open source email servers and clients. 0. https://xkcd.com/927/ https://xkcd.com/927/
- bewaretheirs 3y ago
  It's available as an alternate protocol in the open-source Cyrus IMAP server: https://www.cyrusimap.org/3.4/imap/developer/jmap.html https://www.cyrusimap.org/3.4/imap/developer/jmap.html
  - calvinmorrison 3y ago
    which is what FastMail runs internally
- commoner 3y ago
  While the xkcd comic mentions 14-15 competing standards, there are only 3 email client standards in wide use: POP/SMTP, IMAP/SMTP, and Microsoft Exchange ActiveSync. POP is wholly surpassed by IMAP in functionality and ActiveSync is proprietary, so we're left with IMAP/SMTP as the only viable universal option. I would really like to see an open email client standard that is more reliable and performant than IMAP/SMTP, and JMAP could potentially be it. It doesn't seem unreasonable to have 2 competing standards instead of 1 when IMAP hasn't seen any improvements in 20 years.
  - calvinmorrison 3y ago
    well the other nice benefit is that since JMAP operates by talking JSON via HTTP over TLS you get to skip a lot of the absolutely bonkers annoying cert stuff, SMTP port issues, etc.
  - pmontra 3y ago
    POP3 is perfect for people like me that download their mail and organize it in local folders on their computer. I'm using Thunderbird and mail filters to move messages to their folder. I backup remotely with duplicity.
dathinab 3y ago
The problem is that it still needs a conversion to classical mail (just now on the server instead of client side) which comes with the problem of braking signing. I'm not quite up to date wrt. JMAP but the last time I checked (quite a while ago) their approach to "fixing" that problem was security wise fundamentally broken. Through then most mails are not signed anyway. So for many use-cases this isn't an issue.
- lxgr 3y ago
  What do you mean by "classical mail"?
  - dathinab 3y ago
    The normal MIME based mail format with all it's pain in header encoding, multi part bodies, language and encoding annotations etc. EDIT: Note to be confused with Media-Types which are a (very) small part of the MIME specification and are sometimes called MIME Types. Due to their use in http they got their own spec later on (and are also redefined in the HTTP spec instead of referring to the mime spec, funnily their re-definition in the HTTP spec doesn't 100% match the specification in the MIME spec, luckily only in "don't do this every" parts).
mkenyon 3y ago
I love JMAP. It's what allowed me and my team (at 1Password) to easily add support for Masked Emails, where we randomly generate your email address in addition to your password. Our own Madeline Hanley wrote about that experience, if you'd like to see what it's like to work with JMAP: https://blog.1password.com/making-masked-email-with-jmap/ https://blog.1password.com/making-masked-email-with-jmap/
- ilyt 3y ago
  How JMAP made that better ? Last time we did something like that new email address was just "add entry to a Dovecot database", fully protocol agnostic
  - calvinmorrison 3y ago
    because the masked email API matches the rest of the JMAP API, so it's easy to extend and support. Also, 1Password isn't running Dovecot, Fastmail is. So they're asking a 3rd party is a fairly standards conformant way to go and create a new email alias
- saberworks 3y ago
  I use Fastmail and 1Password, and I do used the masked email functionality. There is a massive gap, though, that makes it very difficult to rely on masked email. When I sign up for a new account somewhere, I can choose to create a masked email. If I later need to email that company (not reply via an existing email message), I must first somehow look up what email I used for that company, go into fastmail settings and create a "sender identity", remember the email address again, and then when I compose, remember to choose the sender identity that is associated with the masked email address I created for that specific company. If I forget or mess up any of those steps, I end up sending from my primary account or some other masked email address associated with some other company. Multiply that by dozens or even a hundred companies! It would be really nice if adding a masked email automatically created a "sending identity." Further, it would be nice if the masked email account had a nickname that included the domain I was on when I created the account. That way if I need to send an email to support@nike.com, I can use the filter and type "nike.com" and get the correct masked email address.
gcanyon 3y ago
Sounds like the big claim here is that it is faster. That's definitely a good thing, but what I want is a protocol where: 1. Every time I give out my email address, behind the scenes a single-source permission/token is created. 2. Thus, that one source is able to send me email, from the email address they specified at the time. 3. But if they hand over the permission/token to anyone else, email from that new source will automatically bounce/be marked as suspect/flagged (my choice in configuration). 4. Even if the original authorized party tries to email me from a different address at the same domain, that email will be handled as in (3). 5. Even from the same email address, the configuration can specify limits, e.g. only 5 emails total, only 3 per week, expire after 10 days, etc. Again, configurable. Oh, and 6. Of course this means unsubscription becomes a local configuration thing that doesn't require the sender's participation. I know current email protocols allow none of that, but if we're proposing new protocols, that's what I want out of it. Faster is better too.
- Hooray_Darakian 3y ago
  > Every time I give out my email address, behind the scenes a single-source permission/token is created. 2. Thus, that one source is able to send me email, from the email address they specified at the time. 3. But if they hand over the permission/token to anyone else, email from that new source will automatically bounce/be marked as suspect/flagged (my choice in configuration). Do you mean hand out in a digital sense or on a business card? I'm not sure you can support the business card use case if you implement that.
  - all2 3y ago
    I assume that business cards would have a unique email/end point to allow for this case, kind of like how gmail and others allow you to add +<stuff> to the end of your email address. So you'd have a set of "business card" emails + rules that would allow only the first sender to send emails to that address (for example).
    - jrky 3y ago
      >So you'd have a set of "business card" emails + rules that would allow only the first sender to send emails to that address (for example). Simple but genius! I love it :-)
asim 3y ago
Sendgrid has an inbound email webhook which I've used quite effectively. What's great is that you can dynamically map your inboxes and provide any address for your domain. https://docs.sendgrid.com/for-developers/parsing-email/setting-up-the-inbound-parse-webhook https://docs.sendgrid.com/for-developers/parsing-email/setti...
mortenjorck 3y ago
The standard looks great, and the IETF publication is a big step, but it’s still caught in the middle of a chicken-and-egg problem between email service providers and OEMs. Even worse, only Apple would have any incentive at all to natively support JMAP on mobile, as Google would prefer you just use Gmail. It would be the exact same problem as trying to get Apple to adopt RCS. I’d love to see JMAP take off, but I don’t see a way forward in the current market.
thecosas 3y ago
Would love to see both Microsoft (for exchange especially, but also their free accounts) and Apple implement JMAP. That said, I know they don't have many incentives to do so and that makes me sad.
nailer 3y ago
Just want to say Fastmail is the best. If you find GMail laggy and other providers to have weak spam filters, give your money to the Fastmail people.
kitsunesoba 3y ago
I really hope that JMAP starts getting some traction, because I've toyed with writing an email client of my own while it's certainly possible to do so, my hope of writing one that people other than myself might actually want to use is quite low. There's so many little nuances and gotchas with IMAP that make it a real bear to work with, plus you have the whole mess of multipart encoding which you might have to roll your own for since not all languages have a suitable library. By contrast HTTP JSON-based APIs are incredibly well-trodden and practically every language has a high quality JSON (de)serializer, many as part of their standard libraries. JMAP would lower the bar for writing email clients dramatically.
- solarkraft 3y ago
  I'm in the same boat! I dislike the current state of E-Mail clients and while there's an uncountable amount of servers, I can't really find any clients.
mlhpdx 3y ago
> Every time a change occurs to data within the user, the modseq is incremented by one and the new value is associated with the changes. This isn’t what I’d call modern, and isn’t much fun. I’d really like to see a email access API standard that gives way on lock-oriented and session-oriented concepts.
- chrismorgan 3y ago
  You’re quoting https://jmap.io/server.html#modification-sequences https://jmap.io/server.html#modification-sequences, about a server implementation detail. Clients don’t get exposed to it directly, but deal with opaque state strings given by the server (which might leak this implementation detail to some extent, e.g. if you just stringify the numbers directly, but a client would be wrong to use it that way, and no harm can come if it does anyway). Back to the server considerations: note that this isn’t the only way it can be implemented, but it’s the simplest and most efficient approach. For the design as a whole, there are very good reasons for it all being done the way it is done. The thing to realise to begin with is that JMAP isn’t actually about email: it’s an object synchronisation protocol (JMAP core, RFC 8620), that just so happens to have email as its first motivating use case (JMAP for Mail, RFC 8621). It’s definitely different in shape from what web developers are used to; but if you want actual object synchronisation (synchronising stuff for offline use, efficiently fetching new and updated records, whether for the entire collection or for queries (which includes searches and things like single-folder views), that kind of thing), it’s almost the simplest principled design possible. When people implement any form of synchronisation or offline support atop their REST or GraphQL or whatever APIs, they’ll either do something very similar to what JMAP does, or build on some other suitable primitive like Merkle trees, or (and this is what normally happens) be unreliable in ways that make inconsistency likely to occur, and data loss distinctly possible. > I’d really like to see a email access API standard that gives way on lock-oriented and session-oriented concepts. I’m not sure what you mean. Could you expand?
danpalmer 3y ago
I have a hard time believing that JMAP is good specifically because it's yet to grow much further than Fastmail. I love that it's open, and I'm a long term happy paying customer of Fastmail, but as an engineer I'd be skeptical of JMAP. I don't have the expertise to know if JMAP solves the necessary problems, and there are probably not that many engineers who do as deep IMAP/POP knowledge is somewhat rare at this point.
- chrismorgan 3y ago
  JMAP was initially developed at Fastmail, but it changed shape quite a bit during the standardisation process at IETF, with significant input from multiple other providers. The end result was definitely the better for it. (In the end, RFC 8620 has two authors listed—one from Fastmail, one from Oracle.) JMAP’s good stuff. (Source/bias: I worked on Fastmail’s webmail from 2017–2020.)
  - danpalmer 3y ago
    Oh that's really good to know, I didn't realise it had that much outside input. In that case I'd feel a lot more comfortable betting on it.
- ocdtrekkie 3y ago
  The problem is Gmail. Google is a bad actor who significantly prefers clients have to support a proprietary API, even though JMAP takes all of the proprietary hacks Gmail built and standardizes them. (Things like snoozed emails, labels, etc. cannot be implemented with IMAP, but can with JMAP.) Supporting JMAP would mean clients that have hacky issues syncing things around labels and snoozed emails could just work, without having to use a proprietary Google API to do it. That alone would be a huge benefit to JMAP being not just widely used, but specifically used by Gmail. The problem is Gmail compatibility has been used as a club to beat down competing OS platforms before, and Google has a strong vested interest in requiring clients use their proprietary APIs and features. Building walled gardens is the core way Google extracts value. It's amazing how the people who have the hardest time believing in JMAP all work or have worked for the company that is most strongly vested in it's failure.
awill 3y ago
First they need to get massive email providers to offer JMAP support, and then get clients to support it. Surely this is simply too big a ship to steer. Google is not going to support this, and isn't that >50% of the market?
- solarkraft 3y ago
  They just need an IMAP/SMTP <-> JMAP bridge. This allows (small) providers to add support without much effort and there's suddenly a demand for clients (and vice versa good clients will increase demand for JMAP support). The website lists jmap-proxy (https://github.com/jmapio/jmap-perl https://github.com/jmapio/jmap-perl) Its age, technology choice and lack of documentation aren't exactly confidence inspiring, however. Stalwart (https://github.com/stalwartlabs/imap-server/ https://github.com/stalwartlabs/imap-server/) seems to include a proxy feature (for both, though it's non-obvious to discover) and be better maintained. A functioning proxy is make or break for me. If it works I can jump right into making a client, if it doesn't there's no point.
  - mildred593 3y ago
    The problem with the proxy is that to be efficient, I believe the proxy will need to store all IMAP messages in an indexed form on the proxy-side. The problem with IMAP is that some requests are not possible efficiently, and the real breakthrough of JMAP is that those requests are now possible and clients are now allowed to be fast. For that though, the server-side JMAP needs to access the indexed mail store directly.
  - awill 3y ago
    but a proxy doesn't have any benefit over regular IMAP. It's just a stop-gap.
    - solarkraft 3y ago
      It has all the benefit. I don't want to have to worry about IMAP and that's what it allows me to do - with not only theoretical browser compatibility, but a complete complete client library fully ready.
Xeoncross 3y ago
So bummed. Encryption is missing again. > Users expect to be able to search their whole archive, so either you need all the data in the client, or the server needs to have access to the data... ...JMAP is therefore not introducing any new measures to address end-to-end encryption You don't need to transfer gigs of emails to a client and manually search character-by-character through a massive trove of data. That isn't how search functions. You can use probabilistic filters (like bloomfilters) so a client can actually run a search on thousands of emails using only a few hundred kilobytes (or maybe megabytes) of actual encrypted data. Basically, less than a modern webpage. Each time a new email is downloaded, parse it, add it to an index segment, encrypt it, and store it back on the email server. Different clients can reuse the same indexes.
- eviks 3y ago
  Exactly, the explanation doesn't hold water, and it's a pity such and important issue is sidestepped in this way
- Szpadel 3y ago
  from technical point of view this sounds interesting, do you have any example of such implementation for text search anywhere? I can see multiple challenges in implementing something like this and I would like to educate myself
  - Xeoncross 3y ago
    https://www.stavros.io/posts/bloom-filter-search-engine/ https://www.stavros.io/posts/bloom-filter-search-engine/
    - Szpadel 3y ago
      Either I didn't understand something or this method would still require server to have way to decrypt data to create that index? Or this could be computed on client side but that's not much different than fetching all emails locally
chillbill 3y ago
I would really like to see encrypted email integrated with this and with Fastmail.
gumby 3y ago
Where are the clients though? Until Apple and Microsoft support it in their default clients, who will deploy it on the server?
singpolyma3 3y ago
It's so hard to remember that JMAP isn't an April fool's joke...
- eschaton 3y ago
  I sincerely thought it was when I first heard of it. At least Microsoft had an excuse with the wacky variations of the Exchange protocol: They built a Big Thing before Internet email “won” and thus have a bunch of legacy to carry forward. This is just a “solution” looking for a problem.
  - chrismorgan 3y ago
    Take a look through https://news.ycombinator.com/item?id=36134733 https://news.ycombinator.com/item?id=36134733 to see some of the concrete problems JMAP solves.
deleted 3y ago
[deleted]
cdme 3y ago
I love what Fastmail does for open source and the email ecosystem writ large — there's even a dedicated macOS client for JMAP https://swiftmail.io https://swiftmail.io (curious when we might see Apple adopt it).
mathfailure 3y ago
> The protocol is stateless > Clients can efficiently fetch updates from their current state a-la QRESYNC. This can be implemented effectively using the MODSEQ data already in modern IMAP servers, or by using a transaction log data structure. The server can always indicate to the client if it cannot calculate updates from a particular client state (for example, because it is too old). Does not compute.
- chrismorgan 3y ago
  The protocol is stateless, just as HTTP is stateless¹. This means that every request is handled independently. Pagination is like “please give me page one for the query ‘foo’” followed by “please give me page two for the query ‘foo’”—so if you lost your connection in the middle, you’re fine. By contrast, IMAP is stateful, and pagination can be more like “I’m starting a query ‘foo’, please give me the results” followed by “please give me the next page”, with the result that a dropped connection can leave you stranded high and dry, having to do everything over again if it’s even possible to. (In practice, now-widely-adopted extensions have been developed to avoid this happening. And really this whole paragraph is a simplification of reality.) —⁂— ¹ Well, HTTP is stateless, but in the HTTP/2 and HTTP/3 wire formats, the HPACK and QPACK header compression schemes use connection-level state. But developers talk in terms of HTTP requests, not HTTP/2 or HTTP/3 frames, so it’s OK and HTTP is still stateless.
dang 3y ago
Related: JMAP: It’s Like IMAP but Not Really (2019) - https://news.ycombinator.com/item?id=32978005 https://news.ycombinator.com/item?id=32978005 - Sept 2022 (131 comments) How and why we built Masked Email with JMAP, an open API standard - https://news.ycombinator.com/item?id=29188639 https://news.ycombinator.com/item?id=29188639 - Nov 2021 (4 comments) Implementing 'focus and reply' for Fastmail with JMAP - https://news.ycombinator.com/item?id=24207506 https://news.ycombinator.com/item?id=24207506 - Aug 2020 (47 comments) JMAP: Modern Mail Standard - https://news.ycombinator.com/item?id=21599666 https://news.ycombinator.com/item?id=21599666 - Nov 2019 (17 comments) Making email more modern with JMAP - https://news.ycombinator.com/item?id=20720630 https://news.ycombinator.com/item?id=20720630 - Aug 2019 (166 comments) The JSON Meta Application Protocol (JMAP) - https://news.ycombinator.com/item?id=20477212 https://news.ycombinator.com/item?id=20477212 - July 2019 (96 comments) JMAP: A modern, open email protocol - https://news.ycombinator.com/item?id=19839104 https://news.ycombinator.com/item?id=19839104 - May 2019 (115 comments) JMAP: Like IMAP but Not Really - https://news.ycombinator.com/item?id=18996200 https://news.ycombinator.com/item?id=18996200 - Jan 2019 (228 comments) JMAP is on the home straight - https://news.ycombinator.com/item?id=18766709 https://news.ycombinator.com/item?id=18766709 - Dec 2018 (50 comments) JMAP – an IMAP replacement - https://news.ycombinator.com/item?id=14734091 https://news.ycombinator.com/item?id=14734091 - July 2017 (20 comments) JMAP: A better way to email (2014) - https://news.ycombinator.com/item?id=14283659 https://news.ycombinator.com/item?id=14283659 - May 2017 (51 comments) Progress on JMAP, better standard for synchronising mail, calendars and contacts - https://news.ycombinator.com/item?id=10781894 https://news.ycombinator.com/item?id=10781894 - Dec 2015 (35 comments) An open source JMAP proxy, JavaScript library and webmail demo - https://news.ycombinator.com/item?id=10038547 https://news.ycombinator.com/item?id=10038547 - Aug 2015 (18 comments) JMAP – a better way to email - https://news.ycombinator.com/item?id=8785894 https://news.ycombinator.com/item?id=8785894 - Dec 2014 (115 comments) JSON Mail Access Protocol Specification (JMAP) - https://news.ycombinator.com/item?id=7141152 https://news.ycombinator.com/item?id=7141152 - Jan 2014 (61 comments) JSON Mail Access Protocol Specification (JMAP) - https://news.ycombinator.com/item?id=7141028 https://news.ycombinator.com/item?id=7141028 - Jan 2014 (1 comment)
einpoklum 3y ago
email is a simple textual protocol. FastMail is not offering an alternative to that, but an "alternative to proprietary email APIs that only work with Gmail". Well, I don't use GMail (and neither should you! All your email is mined for advertising and shared occasionally with the US government.) - so it's not clear to me why I need an alternative to that stuff.
- csb6 3y ago
  > so it's not clear to me why I need an alternative to that stuff. It seems like you don't need it. However, a lot of people access their email across multiple devices and want everything to be kept in sync/efficient to fetch. Currently IMAP is the standard for this, but JMAP is attempting to provide a less complicated replacement.
- jeltz 3y ago
  Nothing about IMAP is simple. JMAP was created partially due to IMAP being so complicated to implement and use.
crispinb 3y ago
Does anyone other than Fastmail use it?
karmanyaahm 3y ago
Having WebPush style push notifications is a huge advantage of JMAP. Right now, 3rd party mobile email clients are stuck to polling every few minutes or keeping a connection open. I'm looking forward to more JMAP adoption, especially on FOSS Android platforms.
- amluto 3y ago
  The _Fastmail app_, which one might assume is the flagship client, doesn’t work at all offline. So I’m unimpressed. Every other non-Webmail client I’ve ever used, whether POP3, IMAP, Exchange, or proprietary, worked at least to a reasonable extent offline.
doodlesdev 3y ago
> A lot of the optimisations for efficient client-server sync require the server to be able to read the message. If everything were encrypted, the server would basically be a dumb blob store. This is particularly bad for mobile, where you only want to sync partial information. Users expect to be able to search their whole archive, so either you need all the data in the client, or the server needs to have access to the data. > JMAP is therefore not introducing any new measures to address end-to-end encryption. The best advice is probably to run your own "JMAP server" on trusted hardware; otherwise you need to sync the entire multi-gigabyte mail spool to all your devices. JMAP is also simple enough that you could run the server on multiple machines with an underlying replication protocol over encrypted links and have that do your smarts. They lost a huge opportunity. Encryption at rest of emails and E2EE should've been how we built these protocols from the start, here we get a chance to try it and... no, just "self-host" instead. I enjoy self-hosting stuff, but it's just not what normal people will be doing, and they deserve privacy too. Also, self-hosting email is possibly the most complicated self-hosting task you can think of due to deliverability issues. My biggest pain point with mail providers that support encryption, such as Proton or Tutanota, is that I have to run a "bridge" application that hosts the IMAP server on my machine, since otherwise there's no way to get the encrypted payloads into my mail client. Otherwise, I will have to use their proprietary client (proprietary in the sense it implements their own protocols). This sucks, and it could be solved at the protocol level with JMAP. I expected this to be a top priority of the Fastmail team when developing this, but unfortunately it's not.
- danielheath 3y ago
  > They lost a huge opportunity. Encryption at rest of emails and E2EE To implement E2EE for email would involve replacing literally every part of the tech stack at literally every provider. You can't lose an opportunity you never had.
  - drdaeman 3y ago
    I don't think so. I mean, I can't imagine what exactly needs to be replaced, and I'm not really getting what that GP quote is talking about, unless it's email metadata (encryption of which is impossible in practice) or full-text search (I always forget about it, as I don't really use it myself) First, at-rest encryption is perfectly doable with most typical software setups (Postfix/Exim/Dovecot/Cyrus/etc), and I think should be not so hard to achieve with proprietary systems as long as they a) capable of dealing with MIME messages (don't reprocess emails for storage) and b) have milter-like capabilities in their local delivery pipeline. You just throw in a milter that encrypts messages to a user-provided key (such as S/MIME or - hey, don't throw anything at me - PGP public key, or whatever MUA may support), and rely on the mail user agent (aka mail client) to be able to decrypt this mail. Second, I'm not sure how E2EE is even in the picture. It - by definition - should be all happening on the endpoints, so I'm not sure how providers (let alone protocols they use) even fit in the picture. And if anything, it's not on IMAP but on SMTP. But even then, SMTP/LMTP are delivering MIME-encapsulated data (that typically happens to be HTML or text pieces), and they don't really care about what sort of data is in there. This significantly hinders anti-spam capabilities, though.
kovac 3y ago
I'm not sure if I understand the claimed benefit of jmap being an open standard. Isn't imap already an open standard? If so, if Gmail adopted jmap and added proprietary APIs to it, wouldn't we end up where we are now?
- TimTheTinker 3y ago
  The benefits are that (a) JMAP is superior to IMAP in ways that really affect end-users of email clients, and (b) as opposed to other high-performance email protocols, JMAP is an open standard. > if Gmail adopted jmap and added proprietary APIs to it, wouldn't we end up where we are now? The promise is that vendors would be able to use JMAP out of the box.
  - catiopatio 3y ago
    > in ways that really affect end-users of email clients If you don’t mind enumerating what you have in mind, what specifically?
    - chrismorgan 3y ago
      I worked through a few in https://news.ycombinator.com/item?id=36134733 https://news.ycombinator.com/item?id=36134733.
- buzzert 3y ago
  IMAP is a really difficult and antiquated protocol to work with also. It's responses are in a non-regular grammar (as opposed to JSON), and IMAP server implementations vary wildly. As a result, properly functioning IMAP clients/libraries are really few and far-between.
userbinator 3y ago
JSON? At least it's not XML.
- juped 3y ago
  JSON makes me realize how good we actually had it with XML.
btown 3y ago
I was hopeful that https://www.nylas.com/ https://www.nylas.com/ would be the de-facto "adapter" placing a common API surface on top of the major providers and dragging them into a modern-API world. They even had an email client of their own as a proof of concept (forked by one of the original authors as https://github.com/Foundry376/Mailspring https://github.com/Foundry376/Mailspring - and its reusable core https://github.com/Foundry376/Mailspring-Sync https://github.com/Foundry376/Mailspring-Sync may be interesting to many here). But they've pivoted towards making their API only available behind B2B contracts and opaque pricing, and primarily used for corporate email monitoring and CRM use cases - perhaps because security and privacy considerations are nontrivial. I'm still rooting for them but it's a shadow of what it could have been.
alooPotato 3y ago
The query syntax is interesting. A json schema defining an order of operations to perform server side. Looks kind of gnarly tbh. At that point, seems simpler to just treat email data as a bunch of sql tables (i.e. lists, messages, threads, mailboxes, etc) and have the clients issue SQL statements to the server. Would be so much easier to create and read the requests and server implementers could lean on all the SQL tooling available.
- mildred593 3y ago
  SQLite have a bunch of features that would allow limiting what you can do in a request, and if you limit the access to a database by the user that puts e-mails in that database this is easily doable. I started looking at implementing that
xvilka 3y ago
Worth checking once implemented in Mutt and Thunderbird.
amluto 3y ago
I’m mystified by one thing about JMAP: JMAP is supposed to support offline clients, but the Fastmail iOS app has no offline support whatsoever. What gives?
- mildred593 3y ago
  Offline is just a matter stable identifiers on the server. It's the client that is responsible for caching the e-mail content on the device. Also, the fact that JMAP makes it convenient to only download part of a message, and let the server do all of the message parsing does not encourage the client developer to do much processing client-side and keep e-mails blob in cache to be processed offline.
akulbe 3y ago
Three questions. 1. How hard would it be to get 19 years worth of Gmail to Fastmail? 2. I have 4 different domains worth of mail all coming in to one mailbox? Would this be an insane cost on Fastmail? 3. For those of you who have already switched from Gmail -> Fastmail, is there anything you miss after the move?
AnonC 3y ago
More than a year ago, I had posted an "Ask HN: Where is JMAP (jmap.io) implemented or being implemented?" [1] but it didn't get any engagement. The situation doesn't seem to have changed much in the meantime. Fastmail (the main developer) seems to be the only one deploying it on a large scale. [1]: https://news.ycombinator.com/item?id=30088483 https://news.ycombinator.com/item?id=30088483
bullen 3y ago
SMTP is the main protocol, how you store things is not supposed to be a standard?
ceving 3y ago
No end-to-end encryption. Just another useless example how to do the same in another way.
thomasfl 3y ago
Looking forward to see all kinds of chat, messaging and discussions systems being implemented on top of JMAP. Next generation messaging will hopefully be more open.
rcarmo 3y ago
I see this popping up every year (sometimes multiple times a year) and, as someone who used to be a postmaster and did _a lot_ of IMAP tweaking, I have to say that even though IMAP is old and gnarly and increasingly out of favor (I just can't use it on Google anymore, only on outlook.com and my own archive), JMAP seems to pretty much have failed adoption-wise -- I don't see any relevant e-mail clients supporting it, no webmail front-ends claim to use it, no other providers besides FastMail seem to support it, and even old-style ISPs (of which there are still a few) don't seem to have any interest in it (source: I used to work in telco-run ISPs until some years ago, and still have friends doing that). Are there any third-party implementations of servers or clients with significant adoption?
- aidenn0 3y ago
  Only servers I'm aware of that support JMAP are Cyrus and James. Clients do seem to be thin on the ground right now, with https://github.com/jmapio/jmap-demo-webmail https://github.com/jmapio/jmap-demo-webmail still being the one to use (as far as I know).
- marcus_holmes 3y ago
  I'm a Fastmail customer, and speaking to them for my last startup, they were adamant that we couldn't use them for sending transactional emails to our customers. So I'm a bit confused why they're letting 1Password do this. Are we allowed to use JMAP for sending transactional emails via Fastmail now? If not, then JMAP seems to be just a method for creating new Fastmail email clients, because no-one else supports it and Fastmail doesn't allow anything but conventional end-user activity. Or is this wrong? If I wanted to use JMAP for my next startup, is there a transactional email provider who supports it?
  - Spivak 3y ago
    There's software like https://github.com/jmapio/jmap-perl https://github.com/jmapio/jmap-perl you can use in front of your chosen provider but that seems like more headache than it's worth unless your imap implementation/lib is incredibly complicated. Doing something like this is essentially a bet that your provider will eventually implement jmap.
deknos 3y ago
Does thunderbird and evolution support this nowadays?
mildred593 3y ago
JMAP is great but is not nearly enough. it would still need some improvements... on the protocols and on the available servers. I experienced slowdowns with Cyrus (which is the most advanced implementation I believe) on complex requests. Also, the great idea behind JMAP is to chain requests so a series of tasks is performed on the server and the client then get the end result. This works great for the majority of cases but the replacement syntax (to replace results from a previous request in a following request) is not powerful enough in some cases (when trying to reuse complex responses in follow-up requests) The other issue is that there is no way to easily groups e-mails from a given contact. There are JMAP extensions for CardDAV and CalDAV that can be of some use but it's difficult to bridge them with the Email spec and get all e-mails from a given contact for instance. Last, it lacks editing features like appending a custom header to existing e-mails. The fact that JMAP abstract away all of the MIME format is great, but makes it difficult to just add a single header to an email.

1030 points by tambourine_man 3y ago

Fastmail.com continues to delivery great innovations and open sources much of their work. They are the company largely behind this, check them out. (I'm not affiliated in any way to them)

foxtacles 3y ago
Fully agree. I'm also a very happy Fastmail customer of over 10 years now; I could never go back to gmail
- stavros 3y ago
  Same here, whenever I have to open Gmail for work I'm annoyed.
  - arp242 3y ago
    You can connect FastMail to gmail and never look at gmail. Your employer may not approve if you hillary your emails like that though, but that's what I did at a previous employer where this wasn't an issue.
    - stavros 3y ago
      I could, but I don't want to look at work emails outside working hours.
      - foxtacles 3y ago
        Have you considered setting up a rule using the Snooze feature[0]? Using this it should be possible to delay all incoming work email until a time that is suitable for you. [0]: https://www.fastmail.com/blog/fastmail-snooze/ https://www.fastmail.com/blog/fastmail-snooze/ Edit: If you want your work emails to arrive only during work hours and otherwise delay them, you can use a Sieve condition in your email rule similar to this: anyof (date :value "lt" "date" "hour" "09", date :value "ge" "date" "hour" "17")
        stavros 3y ago
        Well that's very interesting, thank you!
    - hackernewds 3y ago
      I like your usage of Hillary as a verb. I don't understand the affinity against Gmail though. With the fastmail forwarding service are you still not using Gmail spam service and have Gmail access to your email anyways (not that I'd believe they do unless wearing a tinfoil hat)
      - arp242 3y ago
        I just don't care much for gmail's UI; that's all there's to it. And having all email in one place is more convenient for me regardless – saves having to open/check another service. You're still using gmail's spam checking; it just uses the API (or IMAP? I forgot) to sync the emails to FastMail.
    - psankar 3y ago
      > If you hillary your emails LOL
- dbrunton 3y ago
  Came here (and used Fastmail for password recovery) to say just this. I actually forget what it’s like to sell advertising on my correspondence!
  - gersg 3y ago
    If you don't enable "smart" features in gmail you don't see ads either.
- TylerE 3y ago
  I tried, but fm’s spam filtering is just hot garbage. Both false negatives and (more troublesome) false positives. It’s like stepping back in time 20 years when soamassassin with default config is as good as it gets.
  - deleted 3y ago
    [deleted]
  - UberFly 3y ago
    I experience the opposite of this, but I guess everyone's use case is different. They do use SpamAssasin and it seems pretty effective.
  - moozeek 3y ago
    I have the exact opposite experience and find Fastnail's spam detection quite superior. There is the occasional spam.email getting through - maybe 1 per day - despite the fact that I have quite a number of custom domains with catch-all email addresses. At the same time I really never have to check my spam folder for false positive, everything in there is literal spam so I forget checking it for weeks on end. Whereas on n Gmail I always need to check the spam folder, Google completely overdoes the filtering it and a lot of wanted emails end up in spam.
  - chrismorgan 3y ago
    I think I’m up to two false positives after six years, neither of which was in any way important (in fact, I kinda agreed with its judgement), and I’m up to 175 false negatives in the last 5⅓ years (of which 32 are from the last twelve months, which suggests a fairly steady rate across the whole time; it comes to about one every eleven days). Before that, I used Gmail (on the same address, for about six years), and my memory is that its false positive rates were around one every month or two, and some of them actually mattered; and its false negative rates varied: sometimes it’d go for a few months with none, then it might let through one or two per day for a few weeks. Certainly less consistent. In the last year I’ve also had to use Gmail via a couple of organisations on addresses with very low volume (less than one per day) which have never received any spam, and both have had at least two false positives.
  - 3y ago
- ndr 3y ago
  I wish their android app supported offline access. It's a huge PITA when you're offline (say in a venue with no cover) and need to pull a ticket out of your inbox. I'm a customer and can't believe they've been in business for so long. I guess this feature will never come.
  - dogmatism 3y ago
    That’s what imap clients are for
    - the_pwner224 3y ago
      Unfortunately all the FOSS email clients for Android have horrible UI/UX.
      - tristan957 3y ago
        Even K9 (now Thunderbird)?
      - zilti 3y ago
        K9 is great
      - synecdoche 3y ago
        Could FairEmail be of interest? https://email.faircode.eu/ https://email.faircode.eu/
- qingcharles 3y ago
  20 years here. One of the only companies I don't have a complaint about.
dheera 3y ago
How are they funded, and are they profitable? I've been using Google Apps for email for 10+ years and I don't want my e-mail provider to die at the whim of some Sand Hill investor's emotions at a bay area house party.
- scblock 3y ago
  You pay them.
  - dheera 3y ago
    Is that enough for their company to be alive? Many startups offer paid services but it isn't enough to cover their own costs and salaries in the first several years; everything depends on investors willing to fund them through that period.
    - Semaphor 3y ago
      FWIW, they are not a startup, they are 24 years old and (after being bought by opera and then buying themselves out again) fully independent [0]. They are also probably HN’s most beloved company, but that just as a side ;) [0]: https://en.wikipedia.org/wiki/Fastmail https://en.wikipedia.org/wiki/Fastmail
      - deleted 3y ago
        [deleted]
      - hackernewds 3y ago
        > (after being bought by opera and then buying themselves out again) that history does not assign to me confidence in them as custodian of my emails
        Semaphor 3y ago
        This was in 2010, not the opera you know today.
taftster 3y ago
Would love to use them. But the pricing is too much. I need a custom domain (or three). And I have a family of 6 people. So that's $5 / user / month = $30 / month = $360 / year. Unfortunately, that's not justifiable for me. And thus why I'm stuck using Gmail with a custom domain.
- WordSkill 3y ago
  "You can build an account where you mix and match the plan level to each user. For example, you can build a Fastmail Family by adding additional Standard and Basic plans for adults and children." - https://www.fastmail.com/pricing/ https://www.fastmail.com/pricing/ Only one of your accounts needs to be Standard tier ($50/y) to allow up to 100 custom domains. The other 5 accounts can be Basic tier ($30/y) and still use as many custom domains as you give them access to. Total undiscounted cost for your family would be $200/y. Use the 25% 1Password discount to reduce your total to $150/y. You have the option to extend that 25% discount up to 3yrs and receive an additional discount for paying in advance. Your total for your first 3 years would be $420, then $560 for each 3 years thereafter. Alternatively, you could consider allowing all your family members use the same Standard account ($105 for the first 3yrs) but with multiple email addresses. Use labels to surface each address for the relevant family member. In my case, I created a family@surname address for any family business, and a name@surname address for each family member. Each family member has set their Fastmail app to focus on their own address. Yes, technically, they can see each other's email, but it hasn't been a problem. I presume they use throwaway gmail accounts for stuff they want to keep totally private. We share a specific custom domain for our masked emails and love that feature, especially the 1Password integration although that has some rough edges.
- kstrauser 3y ago
  Apple has an incredibly cheap option here. If you subscribe to iCloud+ (https://www.apple.com/icloud/ https://www.apple.com/icloud/), starting at $1/month, you can share one or more custom email domains with up to 6 people total. I've been using this for a year or so and my family's been happy with it.
  - taftster 3y ago
mannycalavera42 3y ago
it's a shame they don't have (yet) a EU hosting :(
pazimzadeh 3y ago
Infomaniak is good too https://www.infomaniak.com/en/hosting/service-mail/ https://www.infomaniak.com/en/hosting/service-mail/

kazinator 3y ago

Executive summary: "Webization of mailbox access for developers who don't understand anything outside of the web bubble".

londons_explore 3y ago
Have you tried to send SMTP email lately? Most residential ISP's block SMTP ports and all the encrypted variants of it. If you want something that works for everyone, it has to be port 443 and webby. And any mobile push notifications better be via Google/Apple notification services, because modern OS's don't let applications leave a hanging TCP connection open anymore.
- deleted 3y ago
  [deleted]
- kazinator 3y ago
  SMTP and IMAP are different things. Destination e-mail domains to which you want to send mail have SMTP servers. To send them mail, you have to contact their SMTP server somehow. If not directly, then through an intermediary. > If you want something that works for everyone, it has to be port 443 and webby. That doesn't follow. SMTP is already authenticated and SSL-protected (or can be). The port number isn't the problem, it's the spam. Suppose that I run an alternative mail server, so that you can send me mail over HTTPS on 443. If you're a residential subscriber, I'm still blocking you the same way. Only, it's harder now. I have only one static IP address, and have to serve web requests on it on port 443. So I can't block you at the IP level; I have to let you access my web site (open to all) but block your use of the specific API. That's more costly in terms of system resources; I have to let you connect, and do the SSL handshake and submit requests, instead of turfing your packets at the IP layer. > modern OS's don't let applications leave a hanging TCP connection open anymore That has been the case since BSD Unix introduced sockets. When the application quits, the connections close. (At best, there is a graceful shutdown in the background, where the protocol stack exchanges the final FIN segments.)
- School-Cotton 3y ago
  Why are you trying to run an MTA on a residential network? I suspect you are confused about how things work. The standard SMTP port (25) is for communication among MTAs and yes, residential networks block it (as they should) as any home PC sending on it is 99.999999999% likely to be part of a botnet. Sending mail from a home PC to the outside world is done by submitting it to an MTA on the SMTP Submission port (587) which your email provider should have given you credentials to. Home ISPs don't block port 587 AFAIK.

jeffbee 3y ago

The fact that JMAP has ~zero adoption outside Fastmail itself contradicts the "much needed" part of this title.

Nextgrid 3y ago
The problem is that email has been taken over by a handful of providers, all of which have "growth & engagement" as their business model and would rather not implement any open protocol at all. IMAP remains supported for backwards-compatibility, but I'm sure its shortcomings are seen as a feature to push users towards the official clients so there is no incentive to implement another open protocol to address them.
- stonogo 3y ago
  This is a problem, but the other problem is the tools to implement the service. You can use Cyrus, Apache James, or Stalwart, and that's it. These tools do not integrate with anything; in the case of Stalwart (the only JMAP server not in "experimental" status) you can't just install a JMAP server attached to an existing email service; you have to hand it complete control of everything, even unto the on-disk storage of mail. This is a complete non-starter for any email service outside of hobby/personal hosting. In other words, even if someone wanted to make JMAP available to their users, they'd have to tear the whole thing down and rebuild it in JMAP's image, and the tools available to do so are not fully-featured or integratable enough to do so.
  - StalwartLabs 3y ago
    > These tools do not integrate with anything; in the case of Stalwart (the only JMAP server not in "experimental" status) you can't just install a JMAP server attached to an existing email service; you have to hand it complete control of everything, even unto the on-disk storage of mail. This point you mention is being improved. The next release of Stalwart JMAP (expected in one or two months) will delegate user management to either a SQL database or an LDAP directory. Messages will be stored in either Maildir or MinIO/S3. And all other information will be in either SQLite or FoundationDB. All these features were already implemented except MinIO/S3. Development progress can be tracked at https://github.com/stalwartlabs/mail-server/tree/main/crates/jmap https://github.com/stalwartlabs/mail-server/tree/main/crates...

deleted 3y ago

[deleted]

ashton314 3y ago

Does anyone know of a JMAP -> Maildir client? I'd love to replace mbsync with something faster.

rockorager 3y ago
There aren’t any that go direct to maildir. There is mujmap that syncs to notmuch. I wasn’t able to get it to work the way I wanted, and wrote my own that I’d be happy to share. I’ve been using it for around a month with no issues but that’s certainly not much confidence yet! The advantage of going straight to notmuch is that the data model is so similar to JMAP that synchronizing is just that much easier. Both ends can give you a state and all emails which have changed from that state. Then it’s just a matter of updating the tags on those to each end or uploading/downloading the email if it’s new. I get something like 1.7kB round trip data usage when there are no updates (really it’s 0kB save for the keepalive when used in daemon mode) and ~100ms for that round trip. This is in comparison to several megabytes for a single mbsync poll (even when there are 0 changes) and ~5 seconds

feldrim 3y ago

I see that the specs are defined in 2019 excluding the Websockets spec in 2020. So over 3 years, these specs are implemented almost only by Fastmail. It feels like it is yet another standard [0], which could not be adopted by neither commercial nor open source email servers and clients. 0. https://xkcd.com/927/ https://xkcd.com/927/

bewaretheirs 3y ago
It's available as an alternate protocol in the open-source Cyrus IMAP server: https://www.cyrusimap.org/3.4/imap/developer/jmap.html https://www.cyrusimap.org/3.4/imap/developer/jmap.html
- calvinmorrison 3y ago
  which is what FastMail runs internally
commoner 3y ago
While the xkcd comic mentions 14-15 competing standards, there are only 3 email client standards in wide use: POP/SMTP, IMAP/SMTP, and Microsoft Exchange ActiveSync. POP is wholly surpassed by IMAP in functionality and ActiveSync is proprietary, so we're left with IMAP/SMTP as the only viable universal option. I would really like to see an open email client standard that is more reliable and performant than IMAP/SMTP, and JMAP could potentially be it. It doesn't seem unreasonable to have 2 competing standards instead of 1 when IMAP hasn't seen any improvements in 20 years.
- calvinmorrison 3y ago
  well the other nice benefit is that since JMAP operates by talking JSON via HTTP over TLS you get to skip a lot of the absolutely bonkers annoying cert stuff, SMTP port issues, etc.
- pmontra 3y ago
  POP3 is perfect for people like me that download their mail and organize it in local folders on their computer. I'm using Thunderbird and mail filters to move messages to their folder. I backup remotely with duplicity.

dathinab 3y ago

The problem is that it still needs a conversion to classical mail (just now on the server instead of client side) which comes with the problem of braking signing. I'm not quite up to date wrt. JMAP but the last time I checked (quite a while ago) their approach to "fixing" that problem was security wise fundamentally broken. Through then most mails are not signed anyway. So for many use-cases this isn't an issue.

lxgr 3y ago
What do you mean by "classical mail"?
- dathinab 3y ago
  The normal MIME based mail format with all it's pain in header encoding, multi part bodies, language and encoding annotations etc. EDIT: Note to be confused with Media-Types which are a (very) small part of the MIME specification and are sometimes called MIME Types. Due to their use in http they got their own spec later on (and are also redefined in the HTTP spec instead of referring to the mime spec, funnily their re-definition in the HTTP spec doesn't 100% match the specification in the MIME spec, luckily only in "don't do this every" parts).

mkenyon 3y ago

I love JMAP. It's what allowed me and my team (at 1Password) to easily add support for Masked Emails, where we randomly generate your email address in addition to your password. Our own Madeline Hanley wrote about that experience, if you'd like to see what it's like to work with JMAP: https://blog.1password.com/making-masked-email-with-jmap/ https://blog.1password.com/making-masked-email-with-jmap/

ilyt 3y ago
How JMAP made that better ? Last time we did something like that new email address was just "add entry to a Dovecot database", fully protocol agnostic
- calvinmorrison 3y ago
  because the masked email API matches the rest of the JMAP API, so it's easy to extend and support. Also, 1Password isn't running Dovecot, Fastmail is. So they're asking a 3rd party is a fairly standards conformant way to go and create a new email alias
saberworks 3y ago
I use Fastmail and 1Password, and I do used the masked email functionality. There is a massive gap, though, that makes it very difficult to rely on masked email. When I sign up for a new account somewhere, I can choose to create a masked email. If I later need to email that company (not reply via an existing email message), I must first somehow look up what email I used for that company, go into fastmail settings and create a "sender identity", remember the email address again, and then when I compose, remember to choose the sender identity that is associated with the masked email address I created for that specific company. If I forget or mess up any of those steps, I end up sending from my primary account or some other masked email address associated with some other company. Multiply that by dozens or even a hundred companies! It would be really nice if adding a masked email automatically created a "sending identity." Further, it would be nice if the masked email account had a nickname that included the domain I was on when I created the account. That way if I need to send an email to support@nike.com, I can use the filter and type "nike.com" and get the correct masked email address.

gcanyon 3y ago

Sounds like the big claim here is that it is faster. That's definitely a good thing, but what I want is a protocol where: 1. Every time I give out my email address, behind the scenes a single-source permission/token is created. 2. Thus, that one source is able to send me email, from the email address they specified at the time. 3. But if they hand over the permission/token to anyone else, email from that new source will automatically bounce/be marked as suspect/flagged (my choice in configuration). 4. Even if the original authorized party tries to email me from a different address at the same domain, that email will be handled as in (3). 5. Even from the same email address, the configuration can specify limits, e.g. only 5 emails total, only 3 per week, expire after 10 days, etc. Again, configurable. Oh, and 6. Of course this means unsubscription becomes a local configuration thing that doesn't require the sender's participation. I know current email protocols allow none of that, but if we're proposing new protocols, that's what I want out of it. Faster is better too.

Hooray_Darakian 3y ago
> Every time I give out my email address, behind the scenes a single-source permission/token is created. 2. Thus, that one source is able to send me email, from the email address they specified at the time. 3. But if they hand over the permission/token to anyone else, email from that new source will automatically bounce/be marked as suspect/flagged (my choice in configuration). Do you mean hand out in a digital sense or on a business card? I'm not sure you can support the business card use case if you implement that.
- all2 3y ago
  I assume that business cards would have a unique email/end point to allow for this case, kind of like how gmail and others allow you to add +<stuff> to the end of your email address. So you'd have a set of "business card" emails + rules that would allow only the first sender to send emails to that address (for example).
  - jrky 3y ago
    >So you'd have a set of "business card" emails + rules that would allow only the first sender to send emails to that address (for example). Simple but genius! I love it :-)

asim 3y ago

Sendgrid has an inbound email webhook which I've used quite effectively. What's great is that you can dynamically map your inboxes and provide any address for your domain. https://docs.sendgrid.com/for-developers/parsing-email/setting-up-the-inbound-parse-webhook https://docs.sendgrid.com/for-developers/parsing-email/setti...

mortenjorck 3y ago

The standard looks great, and the IETF publication is a big step, but it’s still caught in the middle of a chicken-and-egg problem between email service providers and OEMs. Even worse, only Apple would have any incentive at all to natively support JMAP on mobile, as Google would prefer you just use Gmail. It would be the exact same problem as trying to get Apple to adopt RCS. I’d love to see JMAP take off, but I don’t see a way forward in the current market.

thecosas 3y ago

Would love to see both Microsoft (for exchange especially, but also their free accounts) and Apple implement JMAP. That said, I know they don't have many incentives to do so and that makes me sad.

nailer 3y ago

Just want to say Fastmail is the best. If you find GMail laggy and other providers to have weak spam filters, give your money to the Fastmail people.

kitsunesoba 3y ago

I really hope that JMAP starts getting some traction, because I've toyed with writing an email client of my own while it's certainly possible to do so, my hope of writing one that people other than myself might actually want to use is quite low. There's so many little nuances and gotchas with IMAP that make it a real bear to work with, plus you have the whole mess of multipart encoding which you might have to roll your own for since not all languages have a suitable library. By contrast HTTP JSON-based APIs are incredibly well-trodden and practically every language has a high quality JSON (de)serializer, many as part of their standard libraries. JMAP would lower the bar for writing email clients dramatically.

solarkraft 3y ago
I'm in the same boat! I dislike the current state of E-Mail clients and while there's an uncountable amount of servers, I can't really find any clients.

mlhpdx 3y ago

> Every time a change occurs to data within the user, the modseq is incremented by one and the new value is associated with the changes. This isn’t what I’d call modern, and isn’t much fun. I’d really like to see a email access API standard that gives way on lock-oriented and session-oriented concepts.

chrismorgan 3y ago
You’re quoting https://jmap.io/server.html#modification-sequences https://jmap.io/server.html#modification-sequences, about a server implementation detail. Clients don’t get exposed to it directly, but deal with opaque state strings given by the server (which might leak this implementation detail to some extent, e.g. if you just stringify the numbers directly, but a client would be wrong to use it that way, and no harm can come if it does anyway). Back to the server considerations: note that this isn’t the only way it can be implemented, but it’s the simplest and most efficient approach. For the design as a whole, there are very good reasons for it all being done the way it is done. The thing to realise to begin with is that JMAP isn’t actually about email: it’s an object synchronisation protocol (JMAP core, RFC 8620), that just so happens to have email as its first motivating use case (JMAP for Mail, RFC 8621). It’s definitely different in shape from what web developers are used to; but if you want actual object synchronisation (synchronising stuff for offline use, efficiently fetching new and updated records, whether for the entire collection or for queries (which includes searches and things like single-folder views), that kind of thing), it’s almost the simplest principled design possible. When people implement any form of synchronisation or offline support atop their REST or GraphQL or whatever APIs, they’ll either do something very similar to what JMAP does, or build on some other suitable primitive like Merkle trees, or (and this is what normally happens) be unreliable in ways that make inconsistency likely to occur, and data loss distinctly possible. > I’d really like to see a email access API standard that gives way on lock-oriented and session-oriented concepts. I’m not sure what you mean. Could you expand?

danpalmer 3y ago

I have a hard time believing that JMAP is good specifically because it's yet to grow much further than Fastmail. I love that it's open, and I'm a long term happy paying customer of Fastmail, but as an engineer I'd be skeptical of JMAP. I don't have the expertise to know if JMAP solves the necessary problems, and there are probably not that many engineers who do as deep IMAP/POP knowledge is somewhat rare at this point.

chrismorgan 3y ago
JMAP was initially developed at Fastmail, but it changed shape quite a bit during the standardisation process at IETF, with significant input from multiple other providers. The end result was definitely the better for it. (In the end, RFC 8620 has two authors listed—one from Fastmail, one from Oracle.) JMAP’s good stuff. (Source/bias: I worked on Fastmail’s webmail from 2017–2020.)
- danpalmer 3y ago
  Oh that's really good to know, I didn't realise it had that much outside input. In that case I'd feel a lot more comfortable betting on it.
ocdtrekkie 3y ago
The problem is Gmail. Google is a bad actor who significantly prefers clients have to support a proprietary API, even though JMAP takes all of the proprietary hacks Gmail built and standardizes them. (Things like snoozed emails, labels, etc. cannot be implemented with IMAP, but can with JMAP.) Supporting JMAP would mean clients that have hacky issues syncing things around labels and snoozed emails could just work, without having to use a proprietary Google API to do it. That alone would be a huge benefit to JMAP being not just widely used, but specifically used by Gmail. The problem is Gmail compatibility has been used as a club to beat down competing OS platforms before, and Google has a strong vested interest in requiring clients use their proprietary APIs and features. Building walled gardens is the core way Google extracts value. It's amazing how the people who have the hardest time believing in JMAP all work or have worked for the company that is most strongly vested in it's failure.

awill 3y ago

First they need to get massive email providers to offer JMAP support, and then get clients to support it. Surely this is simply too big a ship to steer. Google is not going to support this, and isn't that >50% of the market?

solarkraft 3y ago
They just need an IMAP/SMTP <-> JMAP bridge. This allows (small) providers to add support without much effort and there's suddenly a demand for clients (and vice versa good clients will increase demand for JMAP support). The website lists jmap-proxy (https://github.com/jmapio/jmap-perl https://github.com/jmapio/jmap-perl) Its age, technology choice and lack of documentation aren't exactly confidence inspiring, however. Stalwart (https://github.com/stalwartlabs/imap-server/ https://github.com/stalwartlabs/imap-server/) seems to include a proxy feature (for both, though it's non-obvious to discover) and be better maintained. A functioning proxy is make or break for me. If it works I can jump right into making a client, if it doesn't there's no point.
- mildred593 3y ago
  The problem with the proxy is that to be efficient, I believe the proxy will need to store all IMAP messages in an indexed form on the proxy-side. The problem with IMAP is that some requests are not possible efficiently, and the real breakthrough of JMAP is that those requests are now possible and clients are now allowed to be fast. For that though, the server-side JMAP needs to access the indexed mail store directly.
- awill 3y ago
  but a proxy doesn't have any benefit over regular IMAP. It's just a stop-gap.
  - solarkraft 3y ago
    It has all the benefit. I don't want to have to worry about IMAP and that's what it allows me to do - with not only theoretical browser compatibility, but a complete complete client library fully ready.

Xeoncross 3y ago

So bummed. Encryption is missing again. > Users expect to be able to search their whole archive, so either you need all the data in the client, or the server needs to have access to the data... ...JMAP is therefore not introducing any new measures to address end-to-end encryption You don't need to transfer gigs of emails to a client and manually search character-by-character through a massive trove of data. That isn't how search functions. You can use probabilistic filters (like bloomfilters) so a client can actually run a search on thousands of emails using only a few hundred kilobytes (or maybe megabytes) of actual encrypted data. Basically, less than a modern webpage. Each time a new email is downloaded, parse it, add it to an index segment, encrypt it, and store it back on the email server. Different clients can reuse the same indexes.

eviks 3y ago
Exactly, the explanation doesn't hold water, and it's a pity such and important issue is sidestepped in this way
Szpadel 3y ago
from technical point of view this sounds interesting, do you have any example of such implementation for text search anywhere? I can see multiple challenges in implementing something like this and I would like to educate myself
- Xeoncross 3y ago
  https://www.stavros.io/posts/bloom-filter-search-engine/ https://www.stavros.io/posts/bloom-filter-search-engine/
  - Szpadel 3y ago
    Either I didn't understand something or this method would still require server to have way to decrypt data to create that index? Or this could be computed on client side but that's not much different than fetching all emails locally

chillbill 3y ago

I would really like to see encrypted email integrated with this and with Fastmail.

gumby 3y ago

Where are the clients though? Until Apple and Microsoft support it in their default clients, who will deploy it on the server?

singpolyma3 3y ago

It's so hard to remember that JMAP isn't an April fool's joke...

eschaton 3y ago
I sincerely thought it was when I first heard of it. At least Microsoft had an excuse with the wacky variations of the Exchange protocol: They built a Big Thing before Internet email “won” and thus have a bunch of legacy to carry forward. This is just a “solution” looking for a problem.
- chrismorgan 3y ago
  Take a look through https://news.ycombinator.com/item?id=36134733 https://news.ycombinator.com/item?id=36134733 to see some of the concrete problems JMAP solves.

deleted 3y ago

[deleted]

cdme 3y ago

I love what Fastmail does for open source and the email ecosystem writ large — there's even a dedicated macOS client for JMAP https://swiftmail.io https://swiftmail.io (curious when we might see Apple adopt it).

mathfailure 3y ago

> The protocol is stateless > Clients can efficiently fetch updates from their current state a-la QRESYNC. This can be implemented effectively using the MODSEQ data already in modern IMAP servers, or by using a transaction log data structure. The server can always indicate to the client if it cannot calculate updates from a particular client state (for example, because it is too old). Does not compute.

chrismorgan 3y ago
The protocol is stateless, just as HTTP is stateless¹. This means that every request is handled independently. Pagination is like “please give me page one for the query ‘foo’” followed by “please give me page two for the query ‘foo’”—so if you lost your connection in the middle, you’re fine. By contrast, IMAP is stateful, and pagination can be more like “I’m starting a query ‘foo’, please give me the results” followed by “please give me the next page”, with the result that a dropped connection can leave you stranded high and dry, having to do everything over again if it’s even possible to. (In practice, now-widely-adopted extensions have been developed to avoid this happening. And really this whole paragraph is a simplification of reality.) —⁂— ¹ Well, HTTP is stateless, but in the HTTP/2 and HTTP/3 wire formats, the HPACK and QPACK header compression schemes use connection-level state. But developers talk in terms of HTTP requests, not HTTP/2 or HTTP/3 frames, so it’s OK and HTTP is still stateless.

dang 3y ago

Related: JMAP: It’s Like IMAP but Not Really (2019) - https://news.ycombinator.com/item?id=32978005 https://news.ycombinator.com/item?id=32978005 - Sept 2022 (131 comments) How and why we built Masked Email with JMAP, an open API standard - https://news.ycombinator.com/item?id=29188639 https://news.ycombinator.com/item?id=29188639 - Nov 2021 (4 comments) Implementing 'focus and reply' for Fastmail with JMAP - https://news.ycombinator.com/item?id=24207506 https://news.ycombinator.com/item?id=24207506 - Aug 2020 (47 comments) JMAP: Modern Mail Standard - https://news.ycombinator.com/item?id=21599666 https://news.ycombinator.com/item?id=21599666 - Nov 2019 (17 comments) Making email more modern with JMAP - https://news.ycombinator.com/item?id=20720630 https://news.ycombinator.com/item?id=20720630 - Aug 2019 (166 comments) The JSON Meta Application Protocol (JMAP) - https://news.ycombinator.com/item?id=20477212 https://news.ycombinator.com/item?id=20477212 - July 2019 (96 comments) JMAP: A modern, open email protocol - https://news.ycombinator.com/item?id=19839104 https://news.ycombinator.com/item?id=19839104 - May 2019 (115 comments) JMAP: Like IMAP but Not Really - https://news.ycombinator.com/item?id=18996200 https://news.ycombinator.com/item?id=18996200 - Jan 2019 (228 comments) JMAP is on the home straight - https://news.ycombinator.com/item?id=18766709 https://news.ycombinator.com/item?id=18766709 - Dec 2018 (50 comments) JMAP – an IMAP replacement - https://news.ycombinator.com/item?id=14734091 https://news.ycombinator.com/item?id=14734091 - July 2017 (20 comments) JMAP: A better way to email (2014) - https://news.ycombinator.com/item?id=14283659 https://news.ycombinator.com/item?id=14283659 - May 2017 (51 comments) Progress on JMAP, better standard for synchronising mail, calendars and contacts - https://news.ycombinator.com/item?id=10781894 https://news.ycombinator.com/item?id=10781894 - Dec 2015 (35 comments) An open source JMAP proxy, JavaScript library and webmail demo - https://news.ycombinator.com/item?id=10038547 https://news.ycombinator.com/item?id=10038547 - Aug 2015 (18 comments) JMAP – a better way to email - https://news.ycombinator.com/item?id=8785894 https://news.ycombinator.com/item?id=8785894 - Dec 2014 (115 comments) JSON Mail Access Protocol Specification (JMAP) - https://news.ycombinator.com/item?id=7141152 https://news.ycombinator.com/item?id=7141152 - Jan 2014 (61 comments) JSON Mail Access Protocol Specification (JMAP) - https://news.ycombinator.com/item?id=7141028 https://news.ycombinator.com/item?id=7141028 - Jan 2014 (1 comment)

einpoklum 3y ago

email is a simple textual protocol. FastMail is not offering an alternative to that, but an "alternative to proprietary email APIs that only work with Gmail". Well, I don't use GMail (and neither should you! All your email is mined for advertising and shared occasionally with the US government.) - so it's not clear to me why I need an alternative to that stuff.

csb6 3y ago
> so it's not clear to me why I need an alternative to that stuff. It seems like you don't need it. However, a lot of people access their email across multiple devices and want everything to be kept in sync/efficient to fetch. Currently IMAP is the standard for this, but JMAP is attempting to provide a less complicated replacement.
jeltz 3y ago
Nothing about IMAP is simple. JMAP was created partially due to IMAP being so complicated to implement and use.

crispinb 3y ago

Does anyone other than Fastmail use it?

karmanyaahm 3y ago

Having WebPush style push notifications is a huge advantage of JMAP. Right now, 3rd party mobile email clients are stuck to polling every few minutes or keeping a connection open. I'm looking forward to more JMAP adoption, especially on FOSS Android platforms.

amluto 3y ago
The _Fastmail app_, which one might assume is the flagship client, doesn’t work at all offline. So I’m unimpressed. Every other non-Webmail client I’ve ever used, whether POP3, IMAP, Exchange, or proprietary, worked at least to a reasonable extent offline.

doodlesdev 3y ago

> A lot of the optimisations for efficient client-server sync require the server to be able to read the message. If everything were encrypted, the server would basically be a dumb blob store. This is particularly bad for mobile, where you only want to sync partial information. Users expect to be able to search their whole archive, so either you need all the data in the client, or the server needs to have access to the data. > JMAP is therefore not introducing any new measures to address end-to-end encryption. The best advice is probably to run your own "JMAP server" on trusted hardware; otherwise you need to sync the entire multi-gigabyte mail spool to all your devices. JMAP is also simple enough that you could run the server on multiple machines with an underlying replication protocol over encrypted links and have that do your smarts. They lost a huge opportunity. Encryption at rest of emails and E2EE should've been how we built these protocols from the start, here we get a chance to try it and... no, just "self-host" instead. I enjoy self-hosting stuff, but it's just not what normal people will be doing, and they deserve privacy too. Also, self-hosting email is possibly the most complicated self-hosting task you can think of due to deliverability issues. My biggest pain point with mail providers that support encryption, such as Proton or Tutanota, is that I have to run a "bridge" application that hosts the IMAP server on my machine, since otherwise there's no way to get the encrypted payloads into my mail client. Otherwise, I will have to use their proprietary client (proprietary in the sense it implements their own protocols). This sucks, and it could be solved at the protocol level with JMAP. I expected this to be a top priority of the Fastmail team when developing this, but unfortunately it's not.

danielheath 3y ago
> They lost a huge opportunity. Encryption at rest of emails and E2EE To implement E2EE for email would involve replacing literally every part of the tech stack at literally every provider. You can't lose an opportunity you never had.
- drdaeman 3y ago
  I don't think so. I mean, I can't imagine what exactly needs to be replaced, and I'm not really getting what that GP quote is talking about, unless it's email metadata (encryption of which is impossible in practice) or full-text search (I always forget about it, as I don't really use it myself) First, at-rest encryption is perfectly doable with most typical software setups (Postfix/Exim/Dovecot/Cyrus/etc), and I think should be not so hard to achieve with proprietary systems as long as they a) capable of dealing with MIME messages (don't reprocess emails for storage) and b) have milter-like capabilities in their local delivery pipeline. You just throw in a milter that encrypts messages to a user-provided key (such as S/MIME or - hey, don't throw anything at me - PGP public key, or whatever MUA may support), and rely on the mail user agent (aka mail client) to be able to decrypt this mail. Second, I'm not sure how E2EE is even in the picture. It - by definition - should be all happening on the endpoints, so I'm not sure how providers (let alone protocols they use) even fit in the picture. And if anything, it's not on IMAP but on SMTP. But even then, SMTP/LMTP are delivering MIME-encapsulated data (that typically happens to be HTML or text pieces), and they don't really care about what sort of data is in there. This significantly hinders anti-spam capabilities, though.

kovac 3y ago

I'm not sure if I understand the claimed benefit of jmap being an open standard. Isn't imap already an open standard? If so, if Gmail adopted jmap and added proprietary APIs to it, wouldn't we end up where we are now?

TimTheTinker 3y ago
The benefits are that (a) JMAP is superior to IMAP in ways that really affect end-users of email clients, and (b) as opposed to other high-performance email protocols, JMAP is an open standard. > if Gmail adopted jmap and added proprietary APIs to it, wouldn't we end up where we are now? The promise is that vendors would be able to use JMAP out of the box.
- catiopatio 3y ago
  > in ways that really affect end-users of email clients If you don’t mind enumerating what you have in mind, what specifically?
  - chrismorgan 3y ago
    I worked through a few in https://news.ycombinator.com/item?id=36134733 https://news.ycombinator.com/item?id=36134733.
buzzert 3y ago
IMAP is a really difficult and antiquated protocol to work with also. It's responses are in a non-regular grammar (as opposed to JSON), and IMAP server implementations vary wildly. As a result, properly functioning IMAP clients/libraries are really few and far-between.

userbinator 3y ago

JSON? At least it's not XML.

juped 3y ago
JSON makes me realize how good we actually had it with XML.

btown 3y ago

I was hopeful that https://www.nylas.com/ https://www.nylas.com/ would be the de-facto "adapter" placing a common API surface on top of the major providers and dragging them into a modern-API world. They even had an email client of their own as a proof of concept (forked by one of the original authors as https://github.com/Foundry376/Mailspring https://github.com/Foundry376/Mailspring - and its reusable core https://github.com/Foundry376/Mailspring-Sync https://github.com/Foundry376/Mailspring-Sync may be interesting to many here). But they've pivoted towards making their API only available behind B2B contracts and opaque pricing, and primarily used for corporate email monitoring and CRM use cases - perhaps because security and privacy considerations are nontrivial. I'm still rooting for them but it's a shadow of what it could have been.

alooPotato 3y ago

The query syntax is interesting. A json schema defining an order of operations to perform server side. Looks kind of gnarly tbh. At that point, seems simpler to just treat email data as a bunch of sql tables (i.e. lists, messages, threads, mailboxes, etc) and have the clients issue SQL statements to the server. Would be so much easier to create and read the requests and server implementers could lean on all the SQL tooling available.

mildred593 3y ago
SQLite have a bunch of features that would allow limiting what you can do in a request, and if you limit the access to a database by the user that puts e-mails in that database this is easily doable. I started looking at implementing that

xvilka 3y ago

Worth checking once implemented in Mutt and Thunderbird.

amluto 3y ago

I’m mystified by one thing about JMAP: JMAP is supposed to support offline clients, but the Fastmail iOS app has no offline support whatsoever. What gives?

mildred593 3y ago
Offline is just a matter stable identifiers on the server. It's the client that is responsible for caching the e-mail content on the device. Also, the fact that JMAP makes it convenient to only download part of a message, and let the server do all of the message parsing does not encourage the client developer to do much processing client-side and keep e-mails blob in cache to be processed offline.

akulbe 3y ago

Three questions. 1. How hard would it be to get 19 years worth of Gmail to Fastmail? 2. I have 4 different domains worth of mail all coming in to one mailbox? Would this be an insane cost on Fastmail? 3. For those of you who have already switched from Gmail -> Fastmail, is there anything you miss after the move?

AnonC 3y ago

More than a year ago, I had posted an "Ask HN: Where is JMAP (jmap.io) implemented or being implemented?" [1] but it didn't get any engagement. The situation doesn't seem to have changed much in the meantime. Fastmail (the main developer) seems to be the only one deploying it on a large scale. [1]: https://news.ycombinator.com/item?id=30088483 https://news.ycombinator.com/item?id=30088483

bullen 3y ago

SMTP is the main protocol, how you store things is not supposed to be a standard?

ceving 3y ago

No end-to-end encryption. Just another useless example how to do the same in another way.

thomasfl 3y ago

Looking forward to see all kinds of chat, messaging and discussions systems being implemented on top of JMAP. Next generation messaging will hopefully be more open.

rcarmo 3y ago

I see this popping up every year (sometimes multiple times a year) and, as someone who used to be a postmaster and did _a lot_ of IMAP tweaking, I have to say that even though IMAP is old and gnarly and increasingly out of favor (I just can't use it on Google anymore, only on outlook.com and my own archive), JMAP seems to pretty much have failed adoption-wise -- I don't see any relevant e-mail clients supporting it, no webmail front-ends claim to use it, no other providers besides FastMail seem to support it, and even old-style ISPs (of which there are still a few) don't seem to have any interest in it (source: I used to work in telco-run ISPs until some years ago, and still have friends doing that). Are there any third-party implementations of servers or clients with significant adoption?

aidenn0 3y ago
Only servers I'm aware of that support JMAP are Cyrus and James. Clients do seem to be thin on the ground right now, with https://github.com/jmapio/jmap-demo-webmail https://github.com/jmapio/jmap-demo-webmail still being the one to use (as far as I know).
marcus_holmes 3y ago
I'm a Fastmail customer, and speaking to them for my last startup, they were adamant that we couldn't use them for sending transactional emails to our customers. So I'm a bit confused why they're letting 1Password do this. Are we allowed to use JMAP for sending transactional emails via Fastmail now? If not, then JMAP seems to be just a method for creating new Fastmail email clients, because no-one else supports it and Fastmail doesn't allow anything but conventional end-user activity. Or is this wrong? If I wanted to use JMAP for my next startup, is there a transactional email provider who supports it?
- Spivak 3y ago
  There's software like https://github.com/jmapio/jmap-perl https://github.com/jmapio/jmap-perl you can use in front of your chosen provider but that seems like more headache than it's worth unless your imap implementation/lib is incredibly complicated. Doing something like this is essentially a bet that your provider will eventually implement jmap.

deknos 3y ago

Does thunderbird and evolution support this nowadays?

mildred593 3y ago

JMAP is great but is not nearly enough. it would still need some improvements... on the protocols and on the available servers. I experienced slowdowns with Cyrus (which is the most advanced implementation I believe) on complex requests. Also, the great idea behind JMAP is to chain requests so a series of tasks is performed on the server and the client then get the end result. This works great for the majority of cases but the replacement syntax (to replace results from a previous request in a following request) is not powerful enough in some cases (when trying to reuse complex responses in follow-up requests) The other issue is that there is no way to easily groups e-mails from a given contact. There are JMAP extensions for CardDAV and CalDAV that can be of some use but it's difficult to bridge them with the Email spec and get all e-mails from a given contact for instance. Last, it lacks editing features like appending a custom header to existing e-mails. The fact that JMAP abstract away all of the MIME format is great, but makes it difficult to just add a single header to an email.