4 ms·
Pretty much no client or server uses it that way though. The UID extension is more or less required these days. But IMAP is pretty twisted in other ways too. I
by ketralnis 4y ago
Pretty much no client or server uses it that way though. The UID extension is more or less required these days.
But IMAP is pretty twisted in other ways too. IMAP is _not_ a request-response protocol like you might think from other protocols. It looks like it at first because you can say things like (from memory, not the real protocol):
> cmd1 LIST (here cmd1 is a request/response identifier chosen by the client; the server sends responses prefixed by this identifier)
< cmd1 LIST RESPONSE...
But you can also multiplex many simultaneous commands like
> cmd1 LIST folder1
> cmd2 LIST folder2
> cmd3 LIST folder3
< cmd3 LIST folder3 RESPONSE...
< cmd1 LIST folder1 RESPONSE...
< cmd2 LIST folder2 RESPONSE...
(note that they can come out of order)
But it gets weirder because you can receive _unsolicited_ responses. Sometimes you want that like
> cmd1 WATCH folder1 (starts a watch which may have responses but not immediately)
> (do other stuff)
< * NEW MESSAGE folder1 DATA... (some time later)
< * FLAGS CHANGE folder1 DATA... (much more time later)
(the * means that you didn't ask for this so it's not prefixed by an ID that you gave) Here it's sort of solicited or at least something you did on purpose, but there are other more complicated cases where you'll receive data that you never asked for. Again from loose memory but an example might be
> cmd1 OPEN folder1
> (do stuff with folder1)
(much later)
< * CLOSING folder1 WAS DELETED
That means that your client needs to always be listening for data on the socket and reading it out even if the client is sitting in the background, and that if you send a command you might receive responses to other previous or unsolicited commands before the response to the command you just sent.
It's a cool way to do what it wants to do for GUI clients in particular because IMAP and GUIs have the same event-oriented model. But it makes implementing IMAP as a library a little more difficult because you need bidirectional hooks into the containing application. And you can't easily just have a `if POP3 then... else if IMAP4 then...` abstraction layer without some work to make it happen. Heck even just having a function that returns the list of messages requires a background thread and a little bit of lying about what's happening underneath. Leaving data on the socket unconsumed can get you disconnected by keepalive checks so the background task is required if you want to avoid that (which by observing mail.app it seems to just accept the disconnection and reconnect when it happens). Python has an imap library in the stdlib which "avoids" this by being so low-level that it sidesteps some of the issues by making you implement request/response on top of it, and its model also makes perfect IDLE implementation difficult (maybe impossible? but I didn't put enough time into it to prove this).
Because of the complication many GUI clients don't actually support the liveness built in to the protocol. They just ignore any reponses they didn't ask for, don't use WATCH/IDLE, and do the 15 minute polling cycle as if they were POP3. The protocol has all of the tools to have the liveness of Google Docs but I'm not aware of any GUIs that use it right.
- alerighi 4y agoWhat is difficult about that? Every protocol works this way, nothing new. The fact that the server may send you messages even they are not responses... of course, otherwise how do you think notifications can work? The only other options is the client to poll the server periodically (how it was done by older clients, and iPhone client...), that is inefficient, consumes resources and you don't get immediate notifications. With an open socket the software stays in the background and gets woken up by the OS when a byte arrives, how is it complex? > The protocol has all of the tools to have the liveness of Google Docs but I'm not aware of any GUIs that use it right Never had any problem with Thunderbird. If you have Thunderbird open in the background and the GMail web interface both open at the same time most of the time you see first the notification from Thunderbird than the email in the GMail web interface. It's really fast, to the point that if I send myself a mail from my phone it seems to arrive immediately, and it's quite impressive.
- ketralnis 4y ago> What is difficult about that? It's not independently difficult per se, it's just mismatched to the way that other protocols often work in a way that, in practise, few clients implement well. Most clients and libraries implement it as if it were request-response and as a result either produce wrong answers or are notably less efficient as you note. > Every protocol works this way, nothing new. No? POP3 and HTTP as easy examples are simple blocking request-response. Some do either pipelining or multiplexing with the command-tagging, but don't have unsolicited messages. I can think of only a few other comparable protocols with this paradigm but "every" is just not true. > of course, otherwise how do you think notifications can work? Yes? This is indeed _why_ it works this way. I don't know what you're correcting me about. It does this to implement notifications and liveness. We're agreeing. > Never had any problem with Thunderbird Thunderbird is one of the better implementers, yes. There are messages that it ignores like "folder deleted" but according to Dovecot https://www.dovecot.org/clients/ https://www.dovecot.org/clients/ it implements IDLE correctly which is uncommon among clients. See https://wiki2.dovecot.org/Clients https://wiki2.dovecot.org/Clients for more client quirks. The dovecot config file https://dovecot.org/doc/dovecot-example.conf https://dovecot.org/doc/dovecot-example.conf has a imap_client_workarounds setting for many of these common bugs including "delay-newmail" which disables several of the unsolicited message types until the server is confident that the client is actually expecting to receive something. In practise, in real life, clients get this wrong. That's all I'm trying to say. Hacker News always has the well-actuallies but I'm pretty familiar with this and have implemented both an IMAP server and client. I'm not sure what you're trying to "correct" me about.