3 ms·

If you open a support ticket I can look at your search issues, ask for Bron. It shouldn't be slow unless you're deliberately bypassing the Xapian indexes by usi

If you open a support ticket I can look at your search issues, ask for Bron. It shouldn't be slow unless you're deliberately bypassing the Xapian indexes by using substr searches.

Semaphor 9y ago
How long is search supposed to take? Entering something common (like "gmail") takes around 5 seconds for me with 32k results. My old google mail account takes 1.5s to display the first page.
- mike-cardwell 9y ago
  Looking at the Cyrus code, the server doesn't even start sending search results to the client until it has completed running the search. And due to the response being JSON, and there being no pagination in the protocol, presumably the Fastmail client wont start displaying the results until it has retrieved them all either. So I'm not surprised it's slow.
  - brongondwana 9y ago
    Are you looking at JMAP getMessageList here? It paginates the response (offset/anchor and count), but yes - it has to build the complete list. Not that we're using that in FastMail production yet, we're using XCONVMULTISORT still. It's roughly the same algorithm though. The biggest expense is all the index.c stuff to support IMAP protocol requirements. There's a lot of cost in there which will go away with JMAP, and which will go away even more when we replace the separate cyrus.index files with a structured database per user. That's still got a ton of work to do though... one of the goals for long term Cyrus improvements.
    - mike-cardwell 9y ago
      Does this allow for the client to say, "Give me all results matching x", and for the server to say, here's the first N results, but there are potentially more, and then the client can then ask for more, in multiple requests until it has them all, allowing them to progressively show results until they're all retrieved? That's how I would expect a JSON based protocol to work for pretty much all types of requests that provide potentially large lists in a response. I would expect getMailboxes to work this way too to be honest.

If you open a support ticket I can look at your search issues, ask for Bron. It shouldn't be slow unless you're deliberately bypassing the Xapian indexes by using substr searches.

Semaphor 9y ago

How long is search supposed to take? Entering something common (like "gmail") takes around 5 seconds for me with 32k results. My old google mail account takes 1.5s to display the first page.

mike-cardwell 9y ago
Looking at the Cyrus code, the server doesn't even start sending search results to the client until it has completed running the search. And due to the response being JSON, and there being no pagination in the protocol, presumably the Fastmail client wont start displaying the results until it has retrieved them all either. So I'm not surprised it's slow.
- brongondwana 9y ago
  Are you looking at JMAP getMessageList here? It paginates the response (offset/anchor and count), but yes - it has to build the complete list. Not that we're using that in FastMail production yet, we're using XCONVMULTISORT still. It's roughly the same algorithm though. The biggest expense is all the index.c stuff to support IMAP protocol requirements. There's a lot of cost in there which will go away with JMAP, and which will go away even more when we replace the separate cyrus.index files with a structured database per user. That's still got a ton of work to do though... one of the goals for long term Cyrus improvements.
  - mike-cardwell 9y ago
    Does this allow for the client to say, "Give me all results matching x", and for the server to say, here's the first N results, but there are potentially more, and then the client can then ask for more, in multiple requests until it has them all, allowing them to progressively show results until they're all retrieved? That's how I would expect a JSON based protocol to work for pretty much all types of requests that provide potentially large lists in a response. I would expect getMailboxes to work this way too to be honest.