5 ms·

IMAP's killer design flaw was that it imposed a strict UID consistency requirement between replica servers if client sync was to work correctly, i.e. you needed

IMAP's killer design flaw was that it imposed a strict UID consistency requirement between replica servers if client sync was to work correctly, i.e. you needed a master-slave setup or Paxos, you couldn't use a true multi-master setup or CRDTs with eventual consistency because of the possibility that clients might miss UIDs. There have been attempts to solve this with even-odd UIDs etc. but those broke down on further analysis. There have also been papers claiming to implement multi-master IMAP servers over CRDTs but these never addressed the strict UID consistency requirement and never tested with actual client implementations for sync correctness. How does JMAP address the distributed multi-master server sync problem, assuming eventual consistency?

throw0101a 7y ago
> IMAP's killer design flaw was that it imposed a strict UID consistency requirement between replica servers if client sync was to work correctly That may be a problem for folks running things at the proverbial web scale, but if it's just a Dovecot server handling a couple of dozen or hundred accounts, is it that big of an issue?
- jorangreef 7y ago
  > is it that big of an issue? No, unless you're designing a protocol to replace IMAP and you want all the proverbial web scale systems to adopt your new protocol. And then again, interoperability and universal adoption is probably something the smaller Dovecot installations would want. Master-master conflict-free eventual consistency systems are easier and safer to operate, there's no nightmare damage from split-brain and things just work, even in the worst scenario. Master-slave failover is notoriously prone to error, even for small deployments.
  - dvdhnt 7y ago
    This isn't on topic of GP, but you seem to have a handle on what this is, so I thought I'd ask... The conversation seems to focus on JMAP as a replacement for IMAP and primarily as an email protocol. The RFC identifies itself as - > a protocol for clients to efficiently query, > fetch, and modify JSON-based data objects, with support for push > notification of changes and fast resynchronisation and for out-of- > band binary data upload/download. It reads like it applies to APIs in general, or is this only for mail/messaging services? edit: I see now, this is the core spec, not specifically the mail spec as referenced in another comment. Thanks. source: https://news.ycombinator.com/item?id=20478227 https://news.ycombinator.com/item?id=20478227
    - e12e 7y ago
      Note that you can retrieve arbitrary files over IMAP too - there's not that much "email" about it. And as one quickly realize: it makes sense to solve the general problem first (implement a document database, essentially) and then add "special sauce" for email (eg: a notion of sender/receivers, threads etc). I've wondered for a while how hard it would be to wrap dbmails[db] (postgres) schema/db in graphql and call it a day, for a jmap work-a-like, for example. [db] http://dbmail.org/ http://dbmail.org/
    - theOnliest 7y ago
      (I also work at Fastmail, on the server side.) It does describe APIs in general, and that is one of the reasons I like it so much. All of our internal APIs are JMAP: when you need to do something with a user, you call User/set; with a rule, call Rule/set, and so on. JMAP makes it super easy to build new APIs because you can reuse most of the code; the only things you need to define are the data types (and any custom implementation for access controls, etc.). Although Fastmail itself has much more legacy code (we’ve been around a while!), new things can start with a framework called Ix, which is available at https://github.com/fastmail/Ix https://github.com/fastmail/Ix. Ix is the basis for Topicbox (www.topicbox.com), which is built ground-up on top of JMAP.
- rrix2 7y ago
  It's a constant issue for people who run software like offlineimap across multiple machines: https://www.offlineimap.org/doc/FAQ.html#what-is-the-uid-validity-problem-for-folder https://www.offlineimap.org/doc/FAQ.html#what-is-the-uid-val...
scottlamb 7y ago
> IMAP's killer design flaw was that it imposed a strict UID consistency requirement between replica servers if client sync was to work correctly, i.e. you needed a master-slave setup or Paxos, you couldn't use a true multi-master setup or CRDTs with eventual consistency because of the possibility that clients might miss UIDs. +1, and it's been a while since I've dealt with this, so my memory's fuzzy, but I think it's even worse than that in a couple ways: * The failure mode here is not just that the client misses UIDs but that a given mailbox/uidvalidity/uid means something different to the client than to this replica. Then when the client says to delete one message, they delete a different one. This would be a lot less likely if the server could use a uuid for a message (so collisions are unlikely) instead of following a sequence and/or if it the identifier didn't need to change whenever it moves to another folder. (My memory is even fuzzier here, but iirc in gmail's case, its emulation of the folder model also means a message can have multiple uids at a time, one per label. That mismatch between the folder setup clients expect and the label model is another problem with IMAP. Strictly speaking, I think you can do it all with one folder and an IMAP flag for each label; it's just that the clients don't expect it to work this way.) * Even with a synchronous replication design, there's the possibility that due to a bug or bad machine the server replicas won't match entirely, or just the client state doesn't match. And if you discover and fix such a problem on the server side, your only option for addressing the bad client state is to bump uidvalidity, which is super annoying to the client. It'd be nicer if you could do something that kicks off some softer sync that doesn't use as much bandwidth, lets the client be more usable while it's happening, and/or lets the server know what mismatches were actually found for diagnostic purposes. I haven't looked into JMAP before, but I hope it materially improves this situation...
- nmjenkins 7y ago
  Author of JMAP here. * JMAP uses opaque ids rather than sequential ids, which makes a lot of this easier (the servers can ensure they have their own id namespace to avoid collisions). * The resync mechanism (see the "/changes" method) just indicates something may have changed and then you fetch those records. It's definitely possible to make your master-master replication work so that /changes works regardless of which master you were previously talking to, at the expense of some loss of efficiency (you may end up refetching some data you actually already had).

throw0101a 7y ago

> IMAP's killer design flaw was that it imposed a strict UID consistency requirement between replica servers if client sync was to work correctly That may be a problem for folks running things at the proverbial web scale, but if it's just a Dovecot server handling a couple of dozen or hundred accounts, is it that big of an issue?

jorangreef 7y ago
> is it that big of an issue? No, unless you're designing a protocol to replace IMAP and you want all the proverbial web scale systems to adopt your new protocol. And then again, interoperability and universal adoption is probably something the smaller Dovecot installations would want. Master-master conflict-free eventual consistency systems are easier and safer to operate, there's no nightmare damage from split-brain and things just work, even in the worst scenario. Master-slave failover is notoriously prone to error, even for small deployments.
- dvdhnt 7y ago
  This isn't on topic of GP, but you seem to have a handle on what this is, so I thought I'd ask... The conversation seems to focus on JMAP as a replacement for IMAP and primarily as an email protocol. The RFC identifies itself as - > a protocol for clients to efficiently query, > fetch, and modify JSON-based data objects, with support for push > notification of changes and fast resynchronisation and for out-of- > band binary data upload/download. It reads like it applies to APIs in general, or is this only for mail/messaging services? edit: I see now, this is the core spec, not specifically the mail spec as referenced in another comment. Thanks. source: https://news.ycombinator.com/item?id=20478227 https://news.ycombinator.com/item?id=20478227
  - e12e 7y ago
    Note that you can retrieve arbitrary files over IMAP too - there's not that much "email" about it. And as one quickly realize: it makes sense to solve the general problem first (implement a document database, essentially) and then add "special sauce" for email (eg: a notion of sender/receivers, threads etc). I've wondered for a while how hard it would be to wrap dbmails[db] (postgres) schema/db in graphql and call it a day, for a jmap work-a-like, for example. [db] http://dbmail.org/ http://dbmail.org/
  - theOnliest 7y ago
    (I also work at Fastmail, on the server side.) It does describe APIs in general, and that is one of the reasons I like it so much. All of our internal APIs are JMAP: when you need to do something with a user, you call User/set; with a rule, call Rule/set, and so on. JMAP makes it super easy to build new APIs because you can reuse most of the code; the only things you need to define are the data types (and any custom implementation for access controls, etc.). Although Fastmail itself has much more legacy code (we’ve been around a while!), new things can start with a framework called Ix, which is available at https://github.com/fastmail/Ix https://github.com/fastmail/Ix. Ix is the basis for Topicbox (www.topicbox.com), which is built ground-up on top of JMAP.
rrix2 7y ago
It's a constant issue for people who run software like offlineimap across multiple machines: https://www.offlineimap.org/doc/FAQ.html#what-is-the-uid-validity-problem-for-folder https://www.offlineimap.org/doc/FAQ.html#what-is-the-uid-val...

scottlamb 7y ago

> IMAP's killer design flaw was that it imposed a strict UID consistency requirement between replica servers if client sync was to work correctly, i.e. you needed a master-slave setup or Paxos, you couldn't use a true multi-master setup or CRDTs with eventual consistency because of the possibility that clients might miss UIDs. +1, and it's been a while since I've dealt with this, so my memory's fuzzy, but I think it's even worse than that in a couple ways: * The failure mode here is not just that the client misses UIDs but that a given mailbox/uidvalidity/uid means something different to the client than to this replica. Then when the client says to delete one message, they delete a different one. This would be a lot less likely if the server could use a uuid for a message (so collisions are unlikely) instead of following a sequence and/or if it the identifier didn't need to change whenever it moves to another folder. (My memory is even fuzzier here, but iirc in gmail's case, its emulation of the folder model also means a message can have multiple uids at a time, one per label. That mismatch between the folder setup clients expect and the label model is another problem with IMAP. Strictly speaking, I think you can do it all with one folder and an IMAP flag for each label; it's just that the clients don't expect it to work this way.) * Even with a synchronous replication design, there's the possibility that due to a bug or bad machine the server replicas won't match entirely, or just the client state doesn't match. And if you discover and fix such a problem on the server side, your only option for addressing the bad client state is to bump uidvalidity, which is super annoying to the client. It'd be nicer if you could do something that kicks off some softer sync that doesn't use as much bandwidth, lets the client be more usable while it's happening, and/or lets the server know what mismatches were actually found for diagnostic purposes. I haven't looked into JMAP before, but I hope it materially improves this situation...

nmjenkins 7y ago
Author of JMAP here. * JMAP uses opaque ids rather than sequential ids, which makes a lot of this easier (the servers can ensure they have their own id namespace to avoid collisions). * The resync mechanism (see the "/changes" method) just indicates something may have changed and then you fetch those records. It's definitely possible to make your master-master replication work so that /changes works regardless of which master you were previously talking to, at the expense of some loss of efficiency (you may end up refetching some data you actually already had).