28 ms·

Making email more modern with JMAP

327 points by mfsch 7y ago

aquabeagle 7y ago
Fastmail should throw some engineering resources towards implementing JMAP support in some mail clients like Mutt.
- javert 7y ago
  That seems like a pretty entitled attitude.
  - techntoke 7y ago
    If they want it to be taken seriously by the open source community, might as well start with mail clients used by the top contributors to things like the Linux kernel and core open source libraries used in most Linux distros. This is how to help ensure adoption and making it a replacement for IMAP.
    - sverhagen 7y ago
      "Ensure adoption" sounds so strong. I won't say I don't know mutt, but it just doesn't strike me as their best bang for buck, in terms of user base, does it? Would it not make more sense for them to invest in, dunno, say, Thunderbird?
      - craftyguy 7y ago
        I think the point OP was trying to make was that they should invest in MUA support for this feature.. mutt was merely an example.
- stonogo 7y ago
  The fastest route to MUA support is compatibility with existing MDAs like procmail, maildrop, and the like. At the very least a JMAP client that populates a local Maildir would get you Thunderbird, Evolution, mutt, and more.
  - namibj 7y ago
    Could a fuse filesystem do the trick for maildir emulation? Assuming that'd be beneficial to a full maildir, but I guess it'd only be so at large sizes.
    - marmaduke 7y ago
      Couldn't à hypothetical fuse fs emulate any file based store for email?
  - inputmice 7y ago
    The strong suit of JMAP (among other things) is getting access to specific properties of a server side parsed email. It will do the MIME parsing for you; separate attachments and text bodies from each other and for example allows a client to fetch only Subject and senders instead of the entire email. By brainlessly dumping all that into a Maildir you'd be giving up a lot of the benefits of JMAP. I'm a big fan of JMAP but I don’t see it replacing IMAP for (desktop) clients that will keep a fully synced Maildir locally.
    - stonogo 7y ago
      Nobody said anything about doing brainless things. There's a lot of sound and fury from Fastmail about JMAP, but the only client is ... Fastmail's web interface. I don't care what JMAP wants to do for me as long as it's not actually available to me. IMAP supports downloading headers too -- that doesn't mean everyone uses IMAP 24/7. The IMAP + local synced maildir use case is extremely common and there's no reason JMAP can't do it better. I merely suggest that starting by making small shims for interoperability would encourage protocol adoption so that when clients arise they'll be in a position to be successful out of the gate.
- qzygr 7y ago
  That's what the ircv3 guys did. I think it's a cancerous attitude: you get your protocol implemented regardless of whether it's good or not, since most managers of open source software can't say no to free code
  - kuschku 7y ago
    I don’t think anyone in the IRCv3 team has provided free code to any other project just for implementing IRCv3 functionality. In fact, the opposite is true, most projects have to spend significant effort to support it — but believe that it is worth it.
- danpalmer 7y ago
  Maybe, but mutt seems like a poor choice? They are providing an open source server implementation, which is a great way to increase adoption. Adding support to widely used open source mail clients is also a great option, but I doubt usage of mutt is anywhere near high enough to provide a good return on investment. Implementing in the default mail client on Ubuntu might be worth it, or maybe the most popular open source web mail client that exists could be worth it. Ideally though they need adoption in something like Outlook, Gmail, Apple Mail/iCloud, etc, but these are all closed source so a) impossible to do without support from the vendor, and b) of limited use to increasing the community of JMAP implementors.
dbg31415 7y ago
So a few years back, everything was super fast on IMAP on my phone. But now it just crawls. On the Gmail App it's still pretty fast. So I just assumed it was slow because Google was being a dick and trying to get people to use their app instead of Apple Mail or Outlook. Is the protocol slow, or is Google throttling it?
- javert 7y ago
  Well, IMAP works fine, so it's not the protocol.
- Felz 7y ago
  I don't know about throttling, but later Android versions are known to mess with email notifications. See: https://github.com/k9mail/k-9/issues/3950 https://github.com/k9mail/k-9/issues/3950
- jivings 7y ago
  Having just this month implemented an IMAP client I can tell you it's blazingly fast for Gmail & Outlook. It's actually much faster and more efficient than using the Gmail API.
  - dbg31415 7y ago
    I just clocked it at 16 seconds to check for new mail in Apple Mail checking against a Gmail Account on an iPhone SE connected to Google Fiber. It feels pretty slow... Meanwhile, same update vs same account in Outlook took about 6 seconds. And in the Gmail App, about 4 seconds. Not super scientific, just me looking at my watch and opening each app and seeing how long it took to update. So maybe something with Apple Mail?
    - jivings 7y ago
      The slow part is opening the IMAP connection and authenticating, after that it's fast. I guess it might depend on how these clients are maintaining their connections.
- 7y ago
mfsch 7y ago
From what I gather it should be possible to use 2FA with JMAP. I‘m looking forward to no longer having to decide between using third-party email clients and properly securing the account that’s probably most worth securing.
- u801e 7y ago
  What would be nice is if the servers supported using client side certificates as an authentication factor. It's possible to configure postfix and dovecot to support this [1], but I haven't looked into whether it's possible to require both the certificate and the username and password to authenticate. [1] https://blog.mortis.eu/blog/2017/06/dovecot-and-postfix-with-client-cert-auth.html https://blog.mortis.eu/blog/2017/06/dovecot-and-postfix-with...
- Freak_NL 7y ago
  Would WebAuthn support be possible with JMAP? That would really move things forward.
  - tialaramex 7y ago
    One high level problem that jumps out at me is: When do we authenticate? For web sites WebAuthn drops nicely into the existing login flow. The user knows what they're trying to do is log in, and so when they're prompted to touch the button or whatever it makes sense in that context. But today most email apps just silently update all the time. So are we authenticating when the app starts and then re-using some credentials obtained to re-connect as necessary? Is that safe? Even if that's all we do (which might be safe) it means the user gets prompted for WebAuthn (to press the button) when the mail app starts, even if they aren't currently interested in email. If they dismiss it, when do we prompt them again? Because meanwhile they of course don't get any new email.
    - Freak_NL 7y ago
      Probably the same as now. Authentication with user supplied credentials grants you a long-living session for the current device and current mail application via a locally stored secret unlocked by the user logging on to his computer or unlocking his device (which too could be done via WebAuthn or a similar 2FA approach; e.g., a fingerprint on a smartphone). It can be invalidated by exceeding an age limit or by the user logging out or otherwise retracting the access grant.
simplecomplex 7y ago
Super happy customer of Fastmail of 7 years now. It’s a joy to use. Fast, responsive, and reliable. No pointless redesigns or 10 second loading screens like gmail.
- craftyguy 7y ago
  .. what does this comment have to do with the actual article other than keying off of 'fastmail' in the title?
  - deleted 7y ago
    [deleted]
  - Gaelan 7y ago
    I mean, JMAP is Fastmail's project, and this is a post on Fastmail's site. Still a little off topic though
  - jhasse 7y ago
    Fastmail's web interfaces uses JMAP under the hood.
    - Guest49521 7y ago
      Hmm. I'm a happy FastMail customer of >2 years, but sometimes the web app loses the connection with the host (especially when on flaky wifi) and fails to re-establish it. This is really aggrevating when you hit send on an email, which disables the input text box but then just hangs indefinitely in a "Sending..." state. I've already reported this to customer support. Symptom of JMAP, or just a run-of-the-mill bug somewhere?
gumby 7y ago
I believe the filtering is for user-initiated search, right? Would be nice to have some standardized support for sieve-like server side prefiltering. That's how I sort my mail into mailboxes, send spam (labeled elsewhere) to the spam trap, and the like.
- eridius 7y ago
  What would this support look like, in the context of JMAP? By the time the email is in your inbox it's already gone through server-side prefiltering. So the only integration I can imagine is a way to fetch and store Sieve scripts so you can edit them from your mail client, but that seems like a very provider-specific thing. For example, Fastmail themselves support Sieve, except they don't give you a single editable script, instead they convert your structured rules and spam settings into 8 labelled sections in a Sieve script and give you 4 free-form edit boxes to insert your own code at specific points (so you can't edit the code that they synthesize for you). I don't know how a protocol like JMAP would expose this in a generic fashion.
  - gumby 7y ago
    Two mechanisms: 1, the ability to write arbitrarily nested if all/any/none of various parameters (any/specific header w/restricted reflex) for server-side searches. The Mac mail app has these for the client side, as does my rss reader. A simpler grammar than needing inMailboxOtherThan yet more powerful. Could be represented as a json tree. 2 - the ability to run that filter n incoming mail basically where sieve is run today, with a similar small grammar for specifying a small set of commands to run like file in mailbox, tag, etc.
Sir_Cmpwn 7y ago
Though I have no complaints about the way they went about implementing it (open standards hooray!), as a mail client author and someone with lots of experience with mail in general - I'm NACK to JMAP. I think that it comes with some questionable design decisions (HTTP? JSON?) which on their face I imagine look fine to the typical HNer, but in many ways is a regression from IMAP. I would prefer to see the limitations of IMAP addressed with a few carefully written extensions to the spec - which is already extensible and has a thriving set of useful extensions in the wild. By no means is IMAP a particularly good protocol, but it is extremely broadly supported and designed more with email in mind than JMAP appears to be, which seems to focus more on bringing shiny web tech to email regardless of its utility in such a context. It's also one of many closely interlinked mail-related specifications, and JMAP fails to integrate well with many related specs imo.
- pjc50 7y ago
  > questionable design decisions (HTTP? JSON?) Those are the premises for JMAP, from which the rest follows.
  - ori_b 7y ago
    Yes, which is why the above post finds it not particularly desirable.
- tjoff 7y ago
  JMAP is literally the only positive thing that has happened to email in decades, please don't dismiss it (or do you know of anything else in the works that has any chance of success?). HTTP and JSON are such an insignificant price to pay for the sliver of hope that email can be saved and not being in complete control of google et al.
  - asjo 7y ago
    > HTTP and JSON are such an insignificant price to pay for the sliver of hope that email can be saved and not being in complete control of google et al. How would JMAP accomplish that, in a way that other protocols don't?
jivings 7y ago
It will be interesting to see how this develops. I've spent the last month implementing and testing IMAP support for my app [1]. It has... not been very fun. Before we were limited to providers with a REST API, so implementing IMAP should in theory allow us to support a much wider range of email servers with a much lower support burden. Although the protocol spec is tight, it seems like some providers randomly bug out or hang [2], respond with custom error messages, offer bespoke functionality [3], or generally don't comply with the spec in breaking ways [4]. As a result I'm more worried about this going forward than I had hoped. I'll now also happily support anything that can help modernize email, so I'll apply to the developer program and will be watching JMAP closely. [1]: https://leavemealone.app/ https://leavemealone.app/ [2]: Yahoo Mail always down (https://downdetector.com/status/yahoo-mail https://downdetector.com/status/yahoo-mail) [3]: https://developers.google.com/gmail/imap/imap-extensions https://developers.google.com/gmail/imap/imap-extensions [4]: https://github.com/mscdex/node-imap/issues/775 https://github.com/mscdex/node-imap/issues/775
- mattmanser 7y ago
  Having worked with IMAP before myself, the spec is very vague a lot. It's a bad spec. We had our own IMAP server implementation and had all sorts of problems even between different versions of outlook. Each project had an inbox which you could cc in, resulting in huge inboxes (for the 2000s at least). You could also drag and drop an email you'd received into the project folder to share it with colleagues. And we had mysterious disappearing emails, but only at some clients. One (of the many) problems I remember is that messages could have an ID, but the spec declared that it could change between sessions, so one version of outlook stored that in a short uint (max 65535) because why would you need more for a temporary id. So have fun if you sent anything more than that. And of course the client had different versions of outlook on different sites so it took ages to even figure out what was happening. Another was that you had to make several back and forths on every request, messages about the messages, to say the payload and number of messages, but the spec was gleefully vague about the order this should happen in. So back then what worked for one email client resulted in an empty inbox for another. I don't think we ever got it working properly with Firebird.
  - 7y ago
verisimilitudes 7y ago
So, how long until everything is JSON-shit over port 80 and how long until port 80 is old news and only port 443 is used for anything and everything? I can't claim to be familiar with IMAP, but it can't be worse than this garbage that requires an HTTP client and a JSON parser. To process JSON, you must inspect every character, which is reason enough it's an awful ''format'' that only an ignorant could hold dear. Douglas Crockford brags about how smart he is for disallowing comments, but there's whitespace allowed everywhere. Let's not forget this asinine integer nonsense, because JavaScript lacks real integers. I'm sure most people think HTTP clients and JSON parsers are standard-library affairs, nowadays, though; let's just entirely ignore matters of licensing and whatnot. It's disgusting to see this as an IETF standard, but people have been complaining about how that moves along for decades; take a look at Google's robots.txt ''standard'' that doesn't do anything useful and effectively makes every existing file standard, rather than do much of anything worthwhile. The WWW and JavaScript are nothing more than a continuation of this worse-is-better filth that UNIX and C previously championed; decades ago, the IETF revised the RFC for the Finger protocol and warned about security, solely because some idiot wrote a shitty Finger server in C; now, we have useless and unnecessary standards such as this that only serve to further this next generation of idiocy. I wonder when all of this will finally collapse under its own weight.
- jimbo1qaz 7y ago
  Can you provide some actual arguments as to what's wrong with JSON? It's a text format that can be inspected by humans, unlike binary formats. (Maybe IMAP is a text format too, so this isn't an advantage inherently) And JMAP may have better semantics than IMAP, and there's nothing inherently wrong with JSON (other than edge cases).
  - verisimilitudes 7y ago
    >Can you provide some actual arguments as to what's wrong with JSON? It's a wasteful, textual format derived from a programming language vomited forth in under a week. It has asinine integer characteristics for no good reason; it's not a random-access format; validating it is even more work; hash table keys that appear multiple times, instead of being in error, instead overwrite each other, defeating most any kind of optimization; it is thus wildly inefficient to manipulate, as manipulating it correctly in all cases requires full and comprehensive parsing with checking. It is an asinine ''format''. >It's a text format that can be inspected by humans, unlike binary formats. Even then, there are much better textual formats that could be used, but I'm of the opinion that numerical (binary) formats are strictly superior, especially as they're used increasingly often. It's better to have good tools that will provide a human-readable representation, rather than pretend a single, inefficient representation for both man and machine is somehow the best option. >(Maybe IMAP is a text format too, so this isn't an advantage inherently) And JMAP may have better semantics than IMAP, and there's nothing inherently wrong with JSON (other than edge cases). Computer programs written by anyone competent are built around edge cases; that's much of what any reliable program is, accounting for edge cases. To disregarded edge cases, of which there are many in JSON, speaks to me that you have little concern for writing software correctly, but I imagine you'd deny this, as who doesn't want to write correct software? Simply understand that using a format with so many edge cases and other defects as JSON is foolhardy for much of anything.
thelittleone 7y ago
"Modern", the darling adjective of startup lingo. We thank you for your service and efficacy. It's time to abdicate the throne. All hail "more modern" the rightful heir, protector of the realm. Pre-modern -> modern -> more modern -> post modern? Jokes aside I enjoyed the experience of Fastmail with the exception of the migration tools. Very responsive. Kudos for submitting JMAP as an open standard.
NetOpWibby 7y ago
FINALLY
wruza 7y ago
I read imap description long time ago, and I know what it does in principle (along with issues itt), but can someone explain what value does it serve today? I have two confusing thoughts. First, imap seems to be an overkill for a line wider than 64kbps. Texts do not consume much traffic and an entire mailbox can be served easily to everyone. There are some huge mailboxes out there, but messengers have this problem too and they solve it somehow (I suspect trivial &last_date=... from programming tg bots). Second, and this may be due to bad clients (haven’t seen good however), imap is just sloooow. It doesn’t update instantly, but instead I have to enter every folder and pull down with my finger to update it. It can show outdated messages which I deleted on a pc for ten seconds before it figures out that they were deleted. My guess that it has something to do with an infinite number of roundtrips which this protocol is based on. Occasionally it shows N unread when there is none. Even push notifications fail to work properly most of the time (same server-side issues?) And with all these complications, I still have to download every attachment by finger. How come that heavy sites/forums/boards/messengers always instantly show new data and mail cannot? I know that I can simply use webmail, but wtf? I’m a developer, but when I see software, I always try to evaluate it from user-only perspective. And from that, mail tech seems simply incompetent. I wish there was a port smtp->tg and tg->smtp, and special per-subj chats, so I could leave all that legacy for someone else to deal with and just talk to them without missing messages or waiting for updates. (Not even considering mail message formatting, spam mismatches, size restrictions and non-delivery laws)
- chrismorgan 7y ago
  The slow mail client issue you’re describing is very probably an artefact of how IMAP IDLE is designed; JMAP fixes this design, as explained at https://jmap.io/#push-mechanism https://jmap.io/#push-mechanism.
inputmice 7y ago
Pretty cool protocol. I've been following this for a couple of years. I had already written a library for an earlier version of JMAP (back then references worked differently / didn’t exist); and it was interesting to see how that improved in the IETF process. I wrote a library [1] and a POC email client for Android [2] earlier this year. It takes a moment to fully understand a now fairly complex protocol but when you get the hang of it it becomes very powerful. Sadly the server support isn’t really there yet. The support for Cyrus hasn’t been released yet (you need git) and some vital functionality like push [3] is still missing. Also no word from Dovecot yet. [1]: https://github.com/iNPUTmice/jmap https://github.com/iNPUTmice/jmap [2]: https://github.com/iNPUTmice/lttrs-android https://github.com/iNPUTmice/lttrs-android [3]: https://github.com/cyrusimap/cyrus-imapd/issues/2714 https://github.com/cyrusimap/cyrus-imapd/issues/2714
- Leace 7y ago
  Do you use Lttrs daily? I'm wondering if it works in practice (even when using non-standard setup) or if it's just a proof-of-concept (that'd mean it may take some time to mature).
  - inputmice 7y ago
    No I don’t use it daily. It can’t even send emails yet and you have to set your login credentials at compile time. I got frustrated having to deal with HTML emails and now I’m waiting for the servers to mature a bit. And honestly also for dovecot to announce support. I mean I love what the fastmail+cyrus team is doing; but if Dovecot is not going to support it, it is probably not worth it to build a JMAP-only MUA.
- brongondwana 7y ago
  I've started defining the bits necessary for storing push subscriptions: see https://github.com/cyrusimap/cyrus-imapd/commit/7144a747267cb968c90128d0d6b4c24199c0ca2e https://github.com/cyrusimap/cyrus-imapd/commit/7144a747267c... But it's rudimentary - it will need sql_db and dblookup components to make it fast enough to integrate with an mboxevent reading component to do the actual pushes. At least - that's how we'll probably implement it at Fastmail, and there's a plan to extract the key parts of our non-blocking push daemon and lift them up into the Cyrus project.
chiefalchemist 7y ago
I didn't see anything about security. I understand sending is outside the scope. But if storage and downloading is to be portable, doesn't encryption have a place?
- bad_user 7y ago
  The encryption of the connection is orthogonal. That's what HTTPS is for. And client-side encryption is managed by solutions like PGP. Doing it on the server-side is incompatible with the email protocol IMO.
  - chiefalchemist 7y ago
    So server side is secured how?
    - bad_user 7y ago
      Not sure I understand the question. What's your threat model?
      - chiefalchemist 7y ago
        On the server, the emails are all encrypted like say passwords? Or are they stored as text?
        TazeTSchnitzel 7y ago
        Passwords aren't encrypted, they are hashed. Every email service out there stores email in what by password standards is plaintext, but hopefully encrypted at rest.
    - LeonM 7y ago
      JMAP (or IMAP for that matter) is the transport format. JMAP runs over HTTPS, so the connection is encrypted. The transport protocol says nothing about how data is stored on the MDA (the server). That's up to the implementation of the MDA, of which no standard exist. The MDA is something that you should trust, if you cannot trust the MDA, you shouldn't use it in the first place. What you need to worry about it transport, and that has been fixed by mandating HTTPS.
z3t4 7y ago
Lets ignore the biggest issue (spam) and focus on the bike shed.
- dane-pgp 7y ago
  No one's stopping you from coming up with your own anti-spam technology. In fact, it's possible for individuals and organisations to work towards multiple goals at once. For example, Fastmail and the IETF have concurrently been working on RFC 8617, which allows for stricter spam filtering: https://dmarc.org/2019/07/arc-protocol-published-as-rfc-8617/ https://dmarc.org/2019/07/arc-protocol-published-as-rfc-8617...
  - z3t4 7y ago
    The reason people choose Gmail or Office instead of smaller providers, is 1) They get less spam 2) Less likely get their mail market as spam. This centralization is killing e-mail. But that's just e-mail. What protocols do people use to send messages ? Facebook messenger, Whatsapp, etc. More proprietary centralized solutions. We need a simple decentralized protocol, as simple as SMTP, bit it need to solve the spam issue, and it need not to be over-engineered (like XMPP). And no, it should not be built ontop of SMTP, SMTP has so many issues, like you can send mail pretending to be anyone. There are solutions like SPF, DKIM etc, but 1) they are built ontop of SMTP 2) They are not stopping spam. We need a new protocol that replaces SMTP, and have spam prevention, identify, etc built in.
    - dane-pgp 7y ago
      Email could be free from spam if you configured your email client to only accept emails from people you manually added (and if SPF/DKIM was enforced). It's not clear how replacing SMTP with a different protocol would make the situation any better. The reason why centralised chat systems can limit spam is that one company gets to decide who can and can't send messages to whom. Recreating that in a decentralised architecture is precisely the problem that needs to be solved, and the solution could then be used by email providers. Now there are no technical reasons not to use SPF/DKIM, and recipients can enforce it more strictly, the problem seems to be determining which mail providers (i.e. which domains) can be trusted to stop their users from sending spam. Spammers have realised this too, of course, which is why a lot of spam comes from domains that are registered cheaply and quickly abandoned. One solution that might be worth exploring is requiring that domain registrants provide a bond to their registrar if they want to send mail from that domain (grandfathering in all existing domains). The bond could be refunded after, say, 12 months of sending enough emails that recipients don't mark as spam. The difficulty with this is creating a reporting mechanism that can't be gamed, either to unfairly cost a domain its bond, or to flood the system with fake positive reports to drown out the true negative ones. Fortunately, with DKIM, there is at least cryptographic proof that a given email was sent by a given domain.
jchook 7y ago
I dislike the details of JMAP’s design. While I appreciate the value it brings, I can see the 5 years of struggle in the sheer size of the multiple specs needed to define some pretty rudimentary mail client/server functionality (retrieve, send, push, pagination, etc). At some points they use JSON (a data format) to describe procedures... like a programming language written in JSON. It feels a bit like using a screwdriver handle as a hammer. How did we end up with JMAP instead of something simpler or modular, especially with GraphQL and other similar innovations on the radar?
- zeveb 7y ago
  > At some points they use JSON (a data format) to describe procedures... like a programming language written in JSON. It feels a bit like using a screwdriver handle as a hammer. And folks laugh at me for suggesting S-expressions until I'm blue in the face! Seriously, this is one of the areas in which they excel (ref.: literally every line of Lisp ever written). They're a pretty nifty data format too (cf. https://sites.google.com/site/steveyegge2/the-emacs-problem https://sites.google.com/site/steveyegge2/the-emacs-problem).
rounce 7y ago
One thing I find weird (probably because I’m new to JMAP and still ignorant) is having position AND anchor+anchorOffset. Why have both? Surely the latter is sufficient and safer (as any position is only guaranteed to be valid when the last response was written)? From a glance of the spec I didn’t see any explanation of why. If anyone could shed more light it’d be much appreciated.
- chrismorgan 7y ago
  Both are necessary because you may not have the necessary IDs. anchor + anchorOffset is useful when you use pagination and new entries might appear at the head of the list but you don’t want them to mess with your pagination once you’re inside. position is useful when you have a total count and wish to jump in at the middle; as a concrete example, when you use lazy loading instead of pagination. For example, if I have a mailbox of 100,000 emails, and jump to the bottom: I don’t want to need to fetch all the IDs (probably well over 1MB to fetch), just so that I can specify an anchor. I want instead to ask “give me the thirty messages from position 99,970”. (Note that when I speak of lazy loading I’m not talking of infinite scrolling, but rather of when you know that you have N messages, and that each message is allocated M pixels of vertical space, so you can allocate an exact space, and provide a proper scrollbar. This is how Fastmail does it and how desktop email clients have historically done it.)
psim1 7y ago
I'm a believer in open protocols, but in recent years, I'm also loving Exchange and ActiveSync. I know that's not going to be a popular opinion here. To be honest, I don't know much about how the magic works, but the configuration is simple, the push is reliable, and everything just works. It doesn't drain my mobile and it doesn't lose its indexing and download for hours like IMAP always seems to. And besides, it's mature -- Exchange and EAS have been around for many years. So here's to JMAP becoming the open standard that replaces that.
- znpy 7y ago
  Yes, but I can’t tun it on my own low hardware at home for free.
  - amaccuish 7y ago
    Sure you can, I use z-push [0] integrated with Kopano which works great. You can also use z-push with IMAP, CalDAV and CardDAV. The only issue I know of using IMAP is that the z-push server polls your mailbox, so it's not true push, but vastly better than the battery drain of IMAP IDLE. [0] http://z-push.org http://z-push.org
0x0 7y ago
They list the wrong RFC number "8261", but the linked RFC is actually "8621".
- brongondwana 7y ago
  Thanks! Have fixed the copy. The link was going to the right place, but somebody (possibly me) put the digits in the wrong order. For your efforts, please enjoy a joke: https://comic.browserling.com/tag/multithreading https://comic.browserling.com/tag/multithreading (that's my excuse - brain was running multithreaded output to the fingers)
Nasreddin_Hodja 7y ago
https://jmap.io/#push-mechanism https://jmap.io/#push-mechanism JMAP going to use rfc8030 push for mobile (3rd party service for notifications, really?) and EventSource (kind of http long polling) for desktop clients, as I understood. I don't like that idea, it's even worse than IMAP4 IDLE. Why not just WebSockets for mobile and desktop clients? Or just building it on top of XMPP protocol (or any other chat protocol).
- nailer 7y ago
  HTML5 Server Sent Events are better than long polling, and they're about as old as websockets, but simpler, as they maintain the concept of HTTP resources whereas websockets replace the concept. HTML5 SSE is also easier to implement. Not hating on websockets, but saying HTML5 SSE is outdated tech is just wrong. If you were making a real time app that works over HTTP either would be fine.
  - Nasreddin_Hodja 7y ago
    > HTML5 Server Sent Events are better than long polling But it works the same way: infinite http connection.
- wchest 7y ago
  > The JMAP protocol is transport agnostic and can be easily transported over a WebSocket, for example, as well as HTTP From section 10. of the jmap.io link you referenced. I haven't used JMAP or looked into the spec in any detail, but wanted to provide that context for anyone coming here and seeing the parent comment first. Sounds like you can use WebSocket or any other transport mechanism.
cotelletta 7y ago
Give me the speed of fastmail with the encryption of protonmail, make the two talk to each other out of the box, and actually make something new instead of just the same old thing. Actually, loan some of your devs to protonmail because both their mobile app and their imap bridge are embarrassing.
akulbe 7y ago
Is the protocol itself important to users? or is it mostly important just to developers? If it makes a practical difference to a user, how so?

327 points by mfsch 7y ago

aquabeagle 7y ago

Fastmail should throw some engineering resources towards implementing JMAP support in some mail clients like Mutt.

javert 7y ago
That seems like a pretty entitled attitude.
- techntoke 7y ago
  If they want it to be taken seriously by the open source community, might as well start with mail clients used by the top contributors to things like the Linux kernel and core open source libraries used in most Linux distros. This is how to help ensure adoption and making it a replacement for IMAP.
  - sverhagen 7y ago
    "Ensure adoption" sounds so strong. I won't say I don't know mutt, but it just doesn't strike me as their best bang for buck, in terms of user base, does it? Would it not make more sense for them to invest in, dunno, say, Thunderbird?
    - craftyguy 7y ago
      I think the point OP was trying to make was that they should invest in MUA support for this feature.. mutt was merely an example.
stonogo 7y ago
The fastest route to MUA support is compatibility with existing MDAs like procmail, maildrop, and the like. At the very least a JMAP client that populates a local Maildir would get you Thunderbird, Evolution, mutt, and more.
- namibj 7y ago
  Could a fuse filesystem do the trick for maildir emulation? Assuming that'd be beneficial to a full maildir, but I guess it'd only be so at large sizes.
  - marmaduke 7y ago
    Couldn't à hypothetical fuse fs emulate any file based store for email?
- inputmice 7y ago
  The strong suit of JMAP (among other things) is getting access to specific properties of a server side parsed email. It will do the MIME parsing for you; separate attachments and text bodies from each other and for example allows a client to fetch only Subject and senders instead of the entire email. By brainlessly dumping all that into a Maildir you'd be giving up a lot of the benefits of JMAP. I'm a big fan of JMAP but I don’t see it replacing IMAP for (desktop) clients that will keep a fully synced Maildir locally.
  - stonogo 7y ago
    Nobody said anything about doing brainless things. There's a lot of sound and fury from Fastmail about JMAP, but the only client is ... Fastmail's web interface. I don't care what JMAP wants to do for me as long as it's not actually available to me. IMAP supports downloading headers too -- that doesn't mean everyone uses IMAP 24/7. The IMAP + local synced maildir use case is extremely common and there's no reason JMAP can't do it better. I merely suggest that starting by making small shims for interoperability would encourage protocol adoption so that when clients arise they'll be in a position to be successful out of the gate.
qzygr 7y ago
That's what the ircv3 guys did. I think it's a cancerous attitude: you get your protocol implemented regardless of whether it's good or not, since most managers of open source software can't say no to free code
- kuschku 7y ago
  I don’t think anyone in the IRCv3 team has provided free code to any other project just for implementing IRCv3 functionality. In fact, the opposite is true, most projects have to spend significant effort to support it — but believe that it is worth it.
danpalmer 7y ago
Maybe, but mutt seems like a poor choice? They are providing an open source server implementation, which is a great way to increase adoption. Adding support to widely used open source mail clients is also a great option, but I doubt usage of mutt is anywhere near high enough to provide a good return on investment. Implementing in the default mail client on Ubuntu might be worth it, or maybe the most popular open source web mail client that exists could be worth it. Ideally though they need adoption in something like Outlook, Gmail, Apple Mail/iCloud, etc, but these are all closed source so a) impossible to do without support from the vendor, and b) of limited use to increasing the community of JMAP implementors.

dbg31415 7y ago

So a few years back, everything was super fast on IMAP on my phone. But now it just crawls. On the Gmail App it's still pretty fast. So I just assumed it was slow because Google was being a dick and trying to get people to use their app instead of Apple Mail or Outlook. Is the protocol slow, or is Google throttling it?

javert 7y ago
Well, IMAP works fine, so it's not the protocol.
Felz 7y ago
I don't know about throttling, but later Android versions are known to mess with email notifications. See: https://github.com/k9mail/k-9/issues/3950 https://github.com/k9mail/k-9/issues/3950
jivings 7y ago
Having just this month implemented an IMAP client I can tell you it's blazingly fast for Gmail & Outlook. It's actually much faster and more efficient than using the Gmail API.
- dbg31415 7y ago
  I just clocked it at 16 seconds to check for new mail in Apple Mail checking against a Gmail Account on an iPhone SE connected to Google Fiber. It feels pretty slow... Meanwhile, same update vs same account in Outlook took about 6 seconds. And in the Gmail App, about 4 seconds. Not super scientific, just me looking at my watch and opening each app and seeing how long it took to update. So maybe something with Apple Mail?
  - jivings 7y ago
    The slow part is opening the IMAP connection and authenticating, after that it's fast. I guess it might depend on how these clients are maintaining their connections.
7y ago

mfsch 7y ago

From what I gather it should be possible to use 2FA with JMAP. I‘m looking forward to no longer having to decide between using third-party email clients and properly securing the account that’s probably most worth securing.

u801e 7y ago
What would be nice is if the servers supported using client side certificates as an authentication factor. It's possible to configure postfix and dovecot to support this [1], but I haven't looked into whether it's possible to require both the certificate and the username and password to authenticate. [1] https://blog.mortis.eu/blog/2017/06/dovecot-and-postfix-with-client-cert-auth.html https://blog.mortis.eu/blog/2017/06/dovecot-and-postfix-with...
Freak_NL 7y ago
Would WebAuthn support be possible with JMAP? That would really move things forward.
- tialaramex 7y ago
  One high level problem that jumps out at me is: When do we authenticate? For web sites WebAuthn drops nicely into the existing login flow. The user knows what they're trying to do is log in, and so when they're prompted to touch the button or whatever it makes sense in that context. But today most email apps just silently update all the time. So are we authenticating when the app starts and then re-using some credentials obtained to re-connect as necessary? Is that safe? Even if that's all we do (which might be safe) it means the user gets prompted for WebAuthn (to press the button) when the mail app starts, even if they aren't currently interested in email. If they dismiss it, when do we prompt them again? Because meanwhile they of course don't get any new email.
  - Freak_NL 7y ago
    Probably the same as now. Authentication with user supplied credentials grants you a long-living session for the current device and current mail application via a locally stored secret unlocked by the user logging on to his computer or unlocking his device (which too could be done via WebAuthn or a similar 2FA approach; e.g., a fingerprint on a smartphone). It can be invalidated by exceeding an age limit or by the user logging out or otherwise retracting the access grant.

simplecomplex 7y ago

Super happy customer of Fastmail of 7 years now. It’s a joy to use. Fast, responsive, and reliable. No pointless redesigns or 10 second loading screens like gmail.

craftyguy 7y ago
.. what does this comment have to do with the actual article other than keying off of 'fastmail' in the title?
- deleted 7y ago
  [deleted]
- Gaelan 7y ago
  I mean, JMAP is Fastmail's project, and this is a post on Fastmail's site. Still a little off topic though
- jhasse 7y ago
  Fastmail's web interfaces uses JMAP under the hood.
  - Guest49521 7y ago
    Hmm. I'm a happy FastMail customer of >2 years, but sometimes the web app loses the connection with the host (especially when on flaky wifi) and fails to re-establish it. This is really aggrevating when you hit send on an email, which disables the input text box but then just hangs indefinitely in a "Sending..." state. I've already reported this to customer support. Symptom of JMAP, or just a run-of-the-mill bug somewhere?

gumby 7y ago

I believe the filtering is for user-initiated search, right? Would be nice to have some standardized support for sieve-like server side prefiltering. That's how I sort my mail into mailboxes, send spam (labeled elsewhere) to the spam trap, and the like.

eridius 7y ago
What would this support look like, in the context of JMAP? By the time the email is in your inbox it's already gone through server-side prefiltering. So the only integration I can imagine is a way to fetch and store Sieve scripts so you can edit them from your mail client, but that seems like a very provider-specific thing. For example, Fastmail themselves support Sieve, except they don't give you a single editable script, instead they convert your structured rules and spam settings into 8 labelled sections in a Sieve script and give you 4 free-form edit boxes to insert your own code at specific points (so you can't edit the code that they synthesize for you). I don't know how a protocol like JMAP would expose this in a generic fashion.
- gumby 7y ago
  Two mechanisms: 1, the ability to write arbitrarily nested if all/any/none of various parameters (any/specific header w/restricted reflex) for server-side searches. The Mac mail app has these for the client side, as does my rss reader. A simpler grammar than needing inMailboxOtherThan yet more powerful. Could be represented as a json tree. 2 - the ability to run that filter n incoming mail basically where sieve is run today, with a similar small grammar for specifying a small set of commands to run like file in mailbox, tag, etc.

Sir_Cmpwn 7y ago

Though I have no complaints about the way they went about implementing it (open standards hooray!), as a mail client author and someone with lots of experience with mail in general - I'm NACK to JMAP. I think that it comes with some questionable design decisions (HTTP? JSON?) which on their face I imagine look fine to the typical HNer, but in many ways is a regression from IMAP. I would prefer to see the limitations of IMAP addressed with a few carefully written extensions to the spec - which is already extensible and has a thriving set of useful extensions in the wild. By no means is IMAP a particularly good protocol, but it is extremely broadly supported and designed more with email in mind than JMAP appears to be, which seems to focus more on bringing shiny web tech to email regardless of its utility in such a context. It's also one of many closely interlinked mail-related specifications, and JMAP fails to integrate well with many related specs imo.

pjc50 7y ago
> questionable design decisions (HTTP? JSON?) Those are the premises for JMAP, from which the rest follows.
- ori_b 7y ago
  Yes, which is why the above post finds it not particularly desirable.
tjoff 7y ago
JMAP is literally the only positive thing that has happened to email in decades, please don't dismiss it (or do you know of anything else in the works that has any chance of success?). HTTP and JSON are such an insignificant price to pay for the sliver of hope that email can be saved and not being in complete control of google et al.
- asjo 7y ago
  > HTTP and JSON are such an insignificant price to pay for the sliver of hope that email can be saved and not being in complete control of google et al. How would JMAP accomplish that, in a way that other protocols don't?

jivings 7y ago

It will be interesting to see how this develops. I've spent the last month implementing and testing IMAP support for my app [1]. It has... not been very fun. Before we were limited to providers with a REST API, so implementing IMAP should in theory allow us to support a much wider range of email servers with a much lower support burden. Although the protocol spec is tight, it seems like some providers randomly bug out or hang [2], respond with custom error messages, offer bespoke functionality [3], or generally don't comply with the spec in breaking ways [4]. As a result I'm more worried about this going forward than I had hoped. I'll now also happily support anything that can help modernize email, so I'll apply to the developer program and will be watching JMAP closely. [1]: https://leavemealone.app/ https://leavemealone.app/ [2]: Yahoo Mail always down (https://downdetector.com/status/yahoo-mail https://downdetector.com/status/yahoo-mail) [3]: https://developers.google.com/gmail/imap/imap-extensions https://developers.google.com/gmail/imap/imap-extensions [4]: https://github.com/mscdex/node-imap/issues/775 https://github.com/mscdex/node-imap/issues/775

mattmanser 7y ago
Having worked with IMAP before myself, the spec is very vague a lot. It's a bad spec. We had our own IMAP server implementation and had all sorts of problems even between different versions of outlook. Each project had an inbox which you could cc in, resulting in huge inboxes (for the 2000s at least). You could also drag and drop an email you'd received into the project folder to share it with colleagues. And we had mysterious disappearing emails, but only at some clients. One (of the many) problems I remember is that messages could have an ID, but the spec declared that it could change between sessions, so one version of outlook stored that in a short uint (max 65535) because why would you need more for a temporary id. So have fun if you sent anything more than that. And of course the client had different versions of outlook on different sites so it took ages to even figure out what was happening. Another was that you had to make several back and forths on every request, messages about the messages, to say the payload and number of messages, but the spec was gleefully vague about the order this should happen in. So back then what worked for one email client resulted in an empty inbox for another. I don't think we ever got it working properly with Firebird.
- 7y ago

verisimilitudes 7y ago

So, how long until everything is JSON-shit over port 80 and how long until port 80 is old news and only port 443 is used for anything and everything? I can't claim to be familiar with IMAP, but it can't be worse than this garbage that requires an HTTP client and a JSON parser. To process JSON, you must inspect every character, which is reason enough it's an awful ''format'' that only an ignorant could hold dear. Douglas Crockford brags about how smart he is for disallowing comments, but there's whitespace allowed everywhere. Let's not forget this asinine integer nonsense, because JavaScript lacks real integers. I'm sure most people think HTTP clients and JSON parsers are standard-library affairs, nowadays, though; let's just entirely ignore matters of licensing and whatnot. It's disgusting to see this as an IETF standard, but people have been complaining about how that moves along for decades; take a look at Google's robots.txt ''standard'' that doesn't do anything useful and effectively makes every existing file standard, rather than do much of anything worthwhile. The WWW and JavaScript are nothing more than a continuation of this worse-is-better filth that UNIX and C previously championed; decades ago, the IETF revised the RFC for the Finger protocol and warned about security, solely because some idiot wrote a shitty Finger server in C; now, we have useless and unnecessary standards such as this that only serve to further this next generation of idiocy. I wonder when all of this will finally collapse under its own weight.

jimbo1qaz 7y ago
Can you provide some actual arguments as to what's wrong with JSON? It's a text format that can be inspected by humans, unlike binary formats. (Maybe IMAP is a text format too, so this isn't an advantage inherently) And JMAP may have better semantics than IMAP, and there's nothing inherently wrong with JSON (other than edge cases).
- verisimilitudes 7y ago
  >Can you provide some actual arguments as to what's wrong with JSON? It's a wasteful, textual format derived from a programming language vomited forth in under a week. It has asinine integer characteristics for no good reason; it's not a random-access format; validating it is even more work; hash table keys that appear multiple times, instead of being in error, instead overwrite each other, defeating most any kind of optimization; it is thus wildly inefficient to manipulate, as manipulating it correctly in all cases requires full and comprehensive parsing with checking. It is an asinine ''format''. >It's a text format that can be inspected by humans, unlike binary formats. Even then, there are much better textual formats that could be used, but I'm of the opinion that numerical (binary) formats are strictly superior, especially as they're used increasingly often. It's better to have good tools that will provide a human-readable representation, rather than pretend a single, inefficient representation for both man and machine is somehow the best option. >(Maybe IMAP is a text format too, so this isn't an advantage inherently) And JMAP may have better semantics than IMAP, and there's nothing inherently wrong with JSON (other than edge cases). Computer programs written by anyone competent are built around edge cases; that's much of what any reliable program is, accounting for edge cases. To disregarded edge cases, of which there are many in JSON, speaks to me that you have little concern for writing software correctly, but I imagine you'd deny this, as who doesn't want to write correct software? Simply understand that using a format with so many edge cases and other defects as JSON is foolhardy for much of anything.

thelittleone 7y ago

"Modern", the darling adjective of startup lingo. We thank you for your service and efficacy. It's time to abdicate the throne. All hail "more modern" the rightful heir, protector of the realm. Pre-modern -> modern -> more modern -> post modern? Jokes aside I enjoyed the experience of Fastmail with the exception of the migration tools. Very responsive. Kudos for submitting JMAP as an open standard.

NetOpWibby 7y ago

FINALLY

wruza 7y ago

I read imap description long time ago, and I know what it does in principle (along with issues itt), but can someone explain what value does it serve today? I have two confusing thoughts. First, imap seems to be an overkill for a line wider than 64kbps. Texts do not consume much traffic and an entire mailbox can be served easily to everyone. There are some huge mailboxes out there, but messengers have this problem too and they solve it somehow (I suspect trivial &last_date=... from programming tg bots). Second, and this may be due to bad clients (haven’t seen good however), imap is just sloooow. It doesn’t update instantly, but instead I have to enter every folder and pull down with my finger to update it. It can show outdated messages which I deleted on a pc for ten seconds before it figures out that they were deleted. My guess that it has something to do with an infinite number of roundtrips which this protocol is based on. Occasionally it shows N unread when there is none. Even push notifications fail to work properly most of the time (same server-side issues?) And with all these complications, I still have to download every attachment by finger. How come that heavy sites/forums/boards/messengers always instantly show new data and mail cannot? I know that I can simply use webmail, but wtf? I’m a developer, but when I see software, I always try to evaluate it from user-only perspective. And from that, mail tech seems simply incompetent. I wish there was a port smtp->tg and tg->smtp, and special per-subj chats, so I could leave all that legacy for someone else to deal with and just talk to them without missing messages or waiting for updates. (Not even considering mail message formatting, spam mismatches, size restrictions and non-delivery laws)

chrismorgan 7y ago
The slow mail client issue you’re describing is very probably an artefact of how IMAP IDLE is designed; JMAP fixes this design, as explained at https://jmap.io/#push-mechanism https://jmap.io/#push-mechanism.

inputmice 7y ago

Pretty cool protocol. I've been following this for a couple of years. I had already written a library for an earlier version of JMAP (back then references worked differently / didn’t exist); and it was interesting to see how that improved in the IETF process. I wrote a library [1] and a POC email client for Android [2] earlier this year. It takes a moment to fully understand a now fairly complex protocol but when you get the hang of it it becomes very powerful. Sadly the server support isn’t really there yet. The support for Cyrus hasn’t been released yet (you need git) and some vital functionality like push [3] is still missing. Also no word from Dovecot yet. [1]: https://github.com/iNPUTmice/jmap https://github.com/iNPUTmice/jmap [2]: https://github.com/iNPUTmice/lttrs-android https://github.com/iNPUTmice/lttrs-android [3]: https://github.com/cyrusimap/cyrus-imapd/issues/2714 https://github.com/cyrusimap/cyrus-imapd/issues/2714

Leace 7y ago
Do you use Lttrs daily? I'm wondering if it works in practice (even when using non-standard setup) or if it's just a proof-of-concept (that'd mean it may take some time to mature).
- inputmice 7y ago
  No I don’t use it daily. It can’t even send emails yet and you have to set your login credentials at compile time. I got frustrated having to deal with HTML emails and now I’m waiting for the servers to mature a bit. And honestly also for dovecot to announce support. I mean I love what the fastmail+cyrus team is doing; but if Dovecot is not going to support it, it is probably not worth it to build a JMAP-only MUA.
brongondwana 7y ago
I've started defining the bits necessary for storing push subscriptions: see https://github.com/cyrusimap/cyrus-imapd/commit/7144a747267cb968c90128d0d6b4c24199c0ca2e https://github.com/cyrusimap/cyrus-imapd/commit/7144a747267c... But it's rudimentary - it will need sql_db and dblookup components to make it fast enough to integrate with an mboxevent reading component to do the actual pushes. At least - that's how we'll probably implement it at Fastmail, and there's a plan to extract the key parts of our non-blocking push daemon and lift them up into the Cyrus project.

chiefalchemist 7y ago

I didn't see anything about security. I understand sending is outside the scope. But if storage and downloading is to be portable, doesn't encryption have a place?

bad_user 7y ago
The encryption of the connection is orthogonal. That's what HTTPS is for. And client-side encryption is managed by solutions like PGP. Doing it on the server-side is incompatible with the email protocol IMO.
- chiefalchemist 7y ago
  So server side is secured how?
  - bad_user 7y ago
    Not sure I understand the question. What's your threat model?
    - chiefalchemist 7y ago
      On the server, the emails are all encrypted like say passwords? Or are they stored as text?
      - TazeTSchnitzel 7y ago
        Passwords aren't encrypted, they are hashed. Every email service out there stores email in what by password standards is plaintext, but hopefully encrypted at rest.
  - LeonM 7y ago
    JMAP (or IMAP for that matter) is the transport format. JMAP runs over HTTPS, so the connection is encrypted. The transport protocol says nothing about how data is stored on the MDA (the server). That's up to the implementation of the MDA, of which no standard exist. The MDA is something that you should trust, if you cannot trust the MDA, you shouldn't use it in the first place. What you need to worry about it transport, and that has been fixed by mandating HTTPS.

z3t4 7y ago

Lets ignore the biggest issue (spam) and focus on the bike shed.

dane-pgp 7y ago
No one's stopping you from coming up with your own anti-spam technology. In fact, it's possible for individuals and organisations to work towards multiple goals at once. For example, Fastmail and the IETF have concurrently been working on RFC 8617, which allows for stricter spam filtering: https://dmarc.org/2019/07/arc-protocol-published-as-rfc-8617/ https://dmarc.org/2019/07/arc-protocol-published-as-rfc-8617...
- z3t4 7y ago
  The reason people choose Gmail or Office instead of smaller providers, is 1) They get less spam 2) Less likely get their mail market as spam. This centralization is killing e-mail. But that's just e-mail. What protocols do people use to send messages ? Facebook messenger, Whatsapp, etc. More proprietary centralized solutions. We need a simple decentralized protocol, as simple as SMTP, bit it need to solve the spam issue, and it need not to be over-engineered (like XMPP). And no, it should not be built ontop of SMTP, SMTP has so many issues, like you can send mail pretending to be anyone. There are solutions like SPF, DKIM etc, but 1) they are built ontop of SMTP 2) They are not stopping spam. We need a new protocol that replaces SMTP, and have spam prevention, identify, etc built in.
  - dane-pgp 7y ago
    Email could be free from spam if you configured your email client to only accept emails from people you manually added (and if SPF/DKIM was enforced). It's not clear how replacing SMTP with a different protocol would make the situation any better. The reason why centralised chat systems can limit spam is that one company gets to decide who can and can't send messages to whom. Recreating that in a decentralised architecture is precisely the problem that needs to be solved, and the solution could then be used by email providers. Now there are no technical reasons not to use SPF/DKIM, and recipients can enforce it more strictly, the problem seems to be determining which mail providers (i.e. which domains) can be trusted to stop their users from sending spam. Spammers have realised this too, of course, which is why a lot of spam comes from domains that are registered cheaply and quickly abandoned. One solution that might be worth exploring is requiring that domain registrants provide a bond to their registrar if they want to send mail from that domain (grandfathering in all existing domains). The bond could be refunded after, say, 12 months of sending enough emails that recipients don't mark as spam. The difficulty with this is creating a reporting mechanism that can't be gamed, either to unfairly cost a domain its bond, or to flood the system with fake positive reports to drown out the true negative ones. Fortunately, with DKIM, there is at least cryptographic proof that a given email was sent by a given domain.

jchook 7y ago

I dislike the details of JMAP’s design. While I appreciate the value it brings, I can see the 5 years of struggle in the sheer size of the multiple specs needed to define some pretty rudimentary mail client/server functionality (retrieve, send, push, pagination, etc). At some points they use JSON (a data format) to describe procedures... like a programming language written in JSON. It feels a bit like using a screwdriver handle as a hammer. How did we end up with JMAP instead of something simpler or modular, especially with GraphQL and other similar innovations on the radar?

zeveb 7y ago
> At some points they use JSON (a data format) to describe procedures... like a programming language written in JSON. It feels a bit like using a screwdriver handle as a hammer. And folks laugh at me for suggesting S-expressions until I'm blue in the face! Seriously, this is one of the areas in which they excel (ref.: literally every line of Lisp ever written). They're a pretty nifty data format too (cf. https://sites.google.com/site/steveyegge2/the-emacs-problem https://sites.google.com/site/steveyegge2/the-emacs-problem).

rounce 7y ago

One thing I find weird (probably because I’m new to JMAP and still ignorant) is having position AND anchor+anchorOffset. Why have both? Surely the latter is sufficient and safer (as any position is only guaranteed to be valid when the last response was written)? From a glance of the spec I didn’t see any explanation of why. If anyone could shed more light it’d be much appreciated.

chrismorgan 7y ago
Both are necessary because you may not have the necessary IDs. anchor + anchorOffset is useful when you use pagination and new entries might appear at the head of the list but you don’t want them to mess with your pagination once you’re inside. position is useful when you have a total count and wish to jump in at the middle; as a concrete example, when you use lazy loading instead of pagination. For example, if I have a mailbox of 100,000 emails, and jump to the bottom: I don’t want to need to fetch all the IDs (probably well over 1MB to fetch), just so that I can specify an anchor. I want instead to ask “give me the thirty messages from position 99,970”. (Note that when I speak of lazy loading I’m not talking of infinite scrolling, but rather of when you know that you have N messages, and that each message is allocated M pixels of vertical space, so you can allocate an exact space, and provide a proper scrollbar. This is how Fastmail does it and how desktop email clients have historically done it.)

psim1 7y ago

I'm a believer in open protocols, but in recent years, I'm also loving Exchange and ActiveSync. I know that's not going to be a popular opinion here. To be honest, I don't know much about how the magic works, but the configuration is simple, the push is reliable, and everything just works. It doesn't drain my mobile and it doesn't lose its indexing and download for hours like IMAP always seems to. And besides, it's mature -- Exchange and EAS have been around for many years. So here's to JMAP becoming the open standard that replaces that.

znpy 7y ago
Yes, but I can’t tun it on my own low hardware at home for free.
- amaccuish 7y ago
  Sure you can, I use z-push [0] integrated with Kopano which works great. You can also use z-push with IMAP, CalDAV and CardDAV. The only issue I know of using IMAP is that the z-push server polls your mailbox, so it's not true push, but vastly better than the battery drain of IMAP IDLE. [0] http://z-push.org http://z-push.org

0x0 7y ago

They list the wrong RFC number "8261", but the linked RFC is actually "8621".

brongondwana 7y ago
Thanks! Have fixed the copy. The link was going to the right place, but somebody (possibly me) put the digits in the wrong order. For your efforts, please enjoy a joke: https://comic.browserling.com/tag/multithreading https://comic.browserling.com/tag/multithreading (that's my excuse - brain was running multithreaded output to the fingers)

Nasreddin_Hodja 7y ago

https://jmap.io/#push-mechanism https://jmap.io/#push-mechanism JMAP going to use rfc8030 push for mobile (3rd party service for notifications, really?) and EventSource (kind of http long polling) for desktop clients, as I understood. I don't like that idea, it's even worse than IMAP4 IDLE. Why not just WebSockets for mobile and desktop clients? Or just building it on top of XMPP protocol (or any other chat protocol).

nailer 7y ago
HTML5 Server Sent Events are better than long polling, and they're about as old as websockets, but simpler, as they maintain the concept of HTTP resources whereas websockets replace the concept. HTML5 SSE is also easier to implement. Not hating on websockets, but saying HTML5 SSE is outdated tech is just wrong. If you were making a real time app that works over HTTP either would be fine.
- Nasreddin_Hodja 7y ago
  > HTML5 Server Sent Events are better than long polling But it works the same way: infinite http connection.
wchest 7y ago
> The JMAP protocol is transport agnostic and can be easily transported over a WebSocket, for example, as well as HTTP From section 10. of the jmap.io link you referenced. I haven't used JMAP or looked into the spec in any detail, but wanted to provide that context for anyone coming here and seeing the parent comment first. Sounds like you can use WebSocket or any other transport mechanism.

cotelletta 7y ago

Give me the speed of fastmail with the encryption of protonmail, make the two talk to each other out of the box, and actually make something new instead of just the same old thing. Actually, loan some of your devs to protonmail because both their mobile app and their imap bridge are embarrassing.

akulbe 7y ago

Is the protocol itself important to users? or is it mostly important just to developers? If it makes a practical difference to a user, how so?