8 ms·

Guide to email in Emacs using mu and mu4e

167 points by gjstein 8y ago

avar 8y ago
Maybe I'm missing something obvious here, but calling "mu index" from the offlineimap postsynchook is going to suck, because mu4e invokes "mu server"[1], that aquires a Xapian lock on the index, which means you can't run the "mu index" while reading your E-Mail. This is the worst thing about mu/mu4e. You need to do the indexing with a timer in Emacs, and because Emacs is single-threaded this blocks things / slows down the UI. After trying to get it to work in some automated way I gave up. I manually issue a key-combo to re-index and then read my E-Mail. Another comment: I recommend not using Emacs to send E-Mail, but to outsource this to a local smtpd such as Exim (which I use with mu4e on Debian). Then if you don't have WiFi or whatever the smtpd just queues those up for later sending, and has queue inspection tools etc. All of the solutions for doing this in Emacs itself are dirty hacks full of edge cases and bugs smtpds have solved decades ago. Despite some rough edges, mu+mu4 (plus Exim, or similar). is an amazing tool, and the most useful E-Mail client I've ever used. 1. https://www.djcbsoftware.nl/code/mu/mu4e/mu-server.html https://www.djcbsoftware.nl/code/mu/mu4e/mu-server.html
- fusiongyro 8y ago
  The locking thing has not turned out to be a really serious problem for me, and I have several gigabytes of email. mu index is really fast. Also, using the Emacs sendmail hasn't been a real problem. It turns out I don't need to be productive during the ~1s it takes to send the email and I have not run into any edge cases or bugs in the ~5 years I've been rocking this setup.
  - avar 8y ago
    It doesn't matter how fast the index is, only one process can hold the Xapian lock, if I'm reading my E-Mail in Emacs with mu4e (which invokes "mu server") and try to index on the command-line I get: $ mu index [...] mu: mu_store_new_writable: xapian error 'Unable to get write lock on <Maildir>/xapian: already locked' (11) How do you send E-Mail with no network connection? Last I checked this was some ad-hoc thing where you neede dto queue them up in some queue managed by Emacs, and remember to turn that mode off when you were back on WiFi (or hack up your own hooks to automate this), v.s. sending it to a local smtpd which already has retry logic built-in.
    - fusiongyro 8y ago
      Why on earth would I try and index it twice concurrently? I never have a scenario where I need to send email without network.
      - avar 8y ago
        No, you wouldn't index from two processes concurrently, but for Emacs to read your mail in mu4e and for you to concurrently issue "mu index" on the command-line two processes ("mu server" and "mu index") would need to poen the same Xapian database read-write at the same time, and just like SQLite Xapian doesn't support that.
        fusiongyro 8y ago
        ... I still don't see why I would do that. Emacs runs offlineimap and then Emacs runs mu index. Why would I do it again myself from the command line?
        ams6110 8y ago
        Yeah I use notmuch with Emacs and it's the same. If you try to search, refresh search results, or read email while the index process is running you get an error. It's not a big deal for me, because I download and index my mail from the command line, so I'll never be reading or sending email at the same time.
        donio 8y ago
        This used to be true but notmuch has added support for Xapian's DB_RETRY_LOCK feature a couple of years ago so if you have non-ancient notmuch (0.23+) and Xapian (1.3.2+) then the second process will wait for the lock instead of erroring out. The feature can be disabled at compile time if you don't want it for some reason.
- pulisse 8y ago
  > You need to do the indexing with a timer in Emacs, and because Emacs is single-threaded this blocks things / slows down the UI. It's worth noting that multithreading recently landed in Emacs master, so it's now possible to avoid this sort of problem.
  - avar 8y ago
    Now possible as in mu4e would need to use some API that's not in a release version of Emacs, or is it more transparent than that?
    - pulisse 8y ago
      The former. I wasn't suggesting that mu4e should be converted right away to use this feature, but more concerned that an unqualified statement that long-running IO must block in Emacs would create the wrong impression.
- kkylin 8y ago
  You can configure mu4e to queue mail to send later: https://www.djcbsoftware.nl/code/mu/mu4e/Queuing-mail.html https://www.djcbsoftware.nl/code/mu/mu4e/Queuing-mail.html
  - avar 8y ago
    Yeah, that's the dirty hack I was referring to. I don't need to tell an smtpd "now's when I expect not to have an Internet connection", which this method relies on, and could happen as I walk into another room and lose WiFi. I just give it mail to deliver, and if it doesn't work it keeps retrying until it works (with configurable backoff and a setting saying when to give up for good).
    - kkylin 8y ago
      Yup, I see where you're coming from. I've actually gotten used to mu4e's queuing behavior, and just leave it on all the time. For one thing, I like having a chance to edit/delete mail after hitting send (sort of like gmail's "undo"), and for another, I like the satisfaction of having a dozen or so messages sent all at once at the end of an email session.
- dima55 8y ago
  The locking is only a problem because he's doing it wrong, I think. I'm doing this: http://notes.secretsauce.net/notes/2014/12/12_mu4e-and-desktop-notifications.html I.e. there's an external script that invokes IMAP IDLE. And when IMAP tells it there's new mail, the script tells EMACS there's new mail. Then the reindexing happens within the emacs, and it uses the lock it already has. Makes this particular issue go away completely.
  - f6e9ed793b 8y ago
    That must be it. I also trigger mu4e indexing using emacsclient, after IMAP sync. BTW you do not need to have mu4e open to index mail (and single-window mu4e mode will not have a * mu4e-main * buffer), it is enough to (progn (require 'mu4e) (mu4e-update-mail-and-index)).
- jfb 8y ago
  I must also be missing something because I never explicitly index my mail. I fetch it with mbsync every two minutes and then use mu4e to read it and new mail shows up as one would expect. ¯\_(ツ)_/¯
  - avar 8y ago
    How many tens of GB of mail do you have? I think this problem only tends to show up on big mailboxes.
    - jfb 8y ago
      10-25GB. But a fast computer, so?
      - tom4000 8y ago
        Good for you. To save power and use smaller machines should always be an option. Indexing saves much resources here.
        jfb 8y ago
        Agreed 100%. I'm just stating my case, not denying others'.
- f6e9ed793b 8y ago
  > Maybe I'm missing something obvious here, but calling "mu index" from the offlineimap postsynchook is going to suck, because mu4e invokes "mu server"[1], that aquires a Xapian lock on the index, which means you can't run the "mu index" while reading your E-Mail. > This is the worst thing about mu/mu4e. You need to do the indexing with a timer in Emacs, and because Emacs is single-threaded this blocks things / slows down the UI. After trying to get it to work in some automated way I gave up. I manually issue a key-combo to re-index and then read my E-Mail. I tried both approaches (actually, I had mu4e reindex on `g` query refresh, no need for separate keybindings), and prefer having indexing run after IMAP sync. I don't notice any problems (and I mostly use very slow Atom hardware). > Another comment: I recommend not using Emacs to send E-Mail, but to outsource this to a local smtpd such as Exim (which I use with mu4e on Debian). Then if you don't have WiFi or whatever the smtpd just queues those up for later sending, and has queue inspection tools etc. All of the solutions for doing this in Emacs itself are dirty hacks full of edge cases and bugs smtpds have solved decades ago. FUD. smtpmail.el can do this. smtpmail.el can queue mail, and you can run it in a separate Emacs process periodically or on demand, to attempt to deliver those emails. The emails are queued as files in a directory, so you don't need separate queue inspection tools. I run smtpmail.el this way with mu4e, and think it is a far superior solution to having to install and configure yet another external program.
- michaelmrose 8y ago
  I use imapnotify to connect to my imap server and respond only when a mail is received by running offlineimap to sync the mail and telling mu4e via emacsclient to sync. Since mu is never ever called except by emacs and only exactly when an email has come in there is never contention for the lock.
- nickysielicki 8y ago
  I recommend notmuch in place of mu/mu4e and msmtp in place of a full-blown smtpd implementation. I also recommend isync's mbsync over offlineimap
  - tom4000 8y ago
    Agree with using Isync. Notmuch lacks the backward synchronization to the IMAP server and to another mail client then notmuch. I primarily use mu4e but want to see changes on my mobile as well as on any remote computer using a web client in a browser. As far as I know notmuch can just be synchronized to another notmuch instance.
    - nickysielicki 8y ago
      If you use https://github.com/gauteh/gmailieer https://github.com/gauteh/gmailieer you can have bidirectional sync between gmail tags and notmuch tags. Personally, I don't bother with it. When I have to do mail on my phone, it's usually so limited in scope that I don't need to retain state when I sit down at my main computer. I can imagine that's not for everyone, though.
- arnoooooo 8y ago
  My mail fetching script kills mu before calling "mu index", like so : $ pgrep mu && kill `pgrep mu` $ mu index It works perfectly. Mu4e then restarts mu when it needs it. I think I saw that as the recommended setup somewhere in the mu docs.
  - u801e 8y ago
    > $ pgrep mu && kill `pgrep mu` This could be rewritten as: $ pkill mu
  - tom4000 8y ago
    With `pkill $(pgrep mu)` or even with `pkill mu` you can kill mutt and any mu$ process as well. `pkill -x mu` matches the whole name without substrings.
autopoiesis 8y ago
How does this compare to using notmuch-emacs? And can someone say whether notmuch or mu is preferable these days?
- dkhenry 8y ago
  I don't have a lot of experience with mu, but I went with notmuch and have really liked it. It reminds me a lot of the standard gmail interface.
- nextos 8y ago
  I have extensively used both, and spent some time using Gnus too. Plus a lot of time on Mutt. In my opinion, Notmuch is better than Mu because it has a really neat architecture that allows customizing it to embrace any possible email workflow with a few trivial Bash commands. Notmuch is purely tag based. It will never ever modify your mailbox. It's your task to map tags to imperative mailbox actions (delete, move...) using some shell hooks. Most workflows can be trivially implemented by calling the notmuch backend and piping the output to mv or rm using xargs or parallel. Since searching in Notmuch is really quick, I recommend never manually assigning tags and tagging everything with rules written in the backend for those repetitive searches that you want a shortcut for. This is not specific to Notmuch, but also applies to Mu. Mu has a hybrid tag/folder model, and the interface resembles Mutt in some ways. The other key difference with Notmuch is that Mu stores tags using a special header in each message. Notmuch uses a separate database, which is not synced across computers (unless you rsync it explicitly). I prefer the latter approach, as modifying your mailbox to store tags is a bit ugly and slow. I also favor not tagging things manually, but using rules. Then syncing tags becomes unnecessary.
  - BeetleB 8y ago
    Thanks for the comparison. I'm wondering what the benefits of mu4e are over notmuch (your comment doesn't list any - which makes sense because you preferred notmuch). The reason I ask: When I look around, I tend to see a lot more references to mu4e than to notmuch. Is it just easier to start with, or does it actually have some nicer features that notmuch doesn't? Not sure if it's relevant, but: I download all emails and do not want to sync with the server.
federicotdn 8y ago
Not completely related, but I have a question regarding Emacs and email: Is there a way to read/write emails on Emacs, without having any emails or credentials stored locally as plaintext? I'm heavily prioritizing ease of configuration and security over speed or other additional features.
- avar 8y ago
  Sure, you sync them locally with e.g. offlineimap, and specify a "remotepasseval" key in your config which can invoke a command that invokes whatever password manager you use, same for sending. I don't do this and just keep my password in plain-text and use full-disk-encryption, because I don't want to have to be constantly unlocking a keyring to have some background E-Mail syncing job work.
- pfortuny 8y ago
  If you save the passwords in a gpg-encrypted file, emacs primpts you for the password to the file when needed. Otherwise, password-read or something like that.
- aerique 8y ago
  I don't know if the others use it but Gnus can read credentials from an .authinfo.gpg file. https://www.gnu.org/software/emacs/manual/html_node/auth/GnuPG-and-EasyPG-Assistant-Configuration.html https://www.gnu.org/software/emacs/manual/html_node/auth/Gnu...
  - globuous 8y ago
    That's what I do actually: https://www.emacswiki.org/emacs/GnusAuthinfo https://www.emacswiki.org/emacs/GnusAuthinfo When I set it up with gmail tough, I think you either need a 2fa token (which requires 2fa on gmail) or tick "allow unsafe apps" in the admin panel, otherwise you get rejected by gmail. I'm sorry I don't have much info, it's already been a few months I've set it up. However, if anyone's struggling, I'll be glad to help ;)
- PuercoPop 8y ago
  Sure. Although the documentation doesnt make it explicit your .authinfo file can have a gpg extension and Emacs will encrypt and decrypt it for you using gpg automatically
Myrmornis 8y ago
I found trying to do this sort of thing (e.g. gnus + dovecot) a very helpful introduction to software engineering; once I got a job as a software engineer I found I didn't have time for a local setup that needed so much attention.
- pfortuny 8y ago
  It needs no attention once you set it up. And the first steup is a bunch of copy-pastes and minor customization.
  - Myrmornis 8y ago
    No, I'm speaking from experience unfortunately! Things break, emacs changes, your OS changes, you decide you need a more recent version of something in order to get a certain feature or bug fix. You get a new laptop. Etc.
    - microcolonel 8y ago
      Not to take away from what you have said, I and several others I know have used Gnus for roughly the last five years without any memorable change in configuration, across three major releases of Emacs.
    - michaericalribo 8y ago
      I agree. I’ve learned immensely from using tools like mu and offlineimap, writing utilities for a maildir, and it can be great fun...but I need direct and unmediated access to email, I need it to ‘just work’ and let me copy-paste html text and images and drag and drop attachments. I have to use email to get other stuff done, I can’t spend time focusing on email itself.
    - pfortuny 8y ago
      Yeah, I see what you mean. I tend to agree.
NesteaIsBad 8y ago
I use Fastmail and the interface is so good that I can say this solution is really crappy.
- unhammer 8y ago
  I use Fastmail and think their web interface is great, but I still also love having offline access to my mail[1], and being able to create an org-mode TODO from an email with a link back to the message, and being able to hit a key to open a new message while coding. Also, every day I get an email with some log files from a service at work; because `mbsync` puts these in my Maildir, I can run a script (using `ripmime`) to extract the logs (going all the way back in time) and show me a report etc., which is so much faster than if I were to get those attachments from a GUI. [1] I'm still spooked from the time my Gmail account was suspended for no apparent reason, having to contact support and being without email for nearly a week.
kkylin 8y ago
I've been using mu4e for a while, and despite some rough edges (e.g., I have the same dissatisfaction with timers and xapian lock that others have), I've been pretty happy with the set-up. I did find that for working with gmail, mbsync works more consistently than offlineimap.
arohner 8y ago
Is there any Emacs-based solution to access a GMail account with 2FA, without enabling "insecure access" on Gmail? I've looked at a lot of solutions, and AFAICT: - SMTP + app-based passwords require "insecure access" being enabled -Using the GMail HTTP API does not require insecure access, but none of the standard CLI email tools support it.
- kuwze 8y ago
  I'm going to break the HN code of ethics (oh gosh) and explicitly bump this. All of the solutions I have found just point at the insecure access option.
- Jim_Heckler 8y ago
  As far as I can tell, when 2FA is enabled app passwords are the only non-2FA way to use your account. I have an app password in my .authinfo.gpg that I use for authorizing offlineimap and SMTP through Emacs. Obviously this isn't as secure as 2FA, but since the password is PGP encrypted I don't worry about it ever being comprised.
- Allaman 8y ago
  Just looked at my account. There is no option for insecure access when you activate 2FA. From https://myaccount.google.com/lesssecureapps https://myaccount.google.com/lesssecureapps "This setting is not available for accounts with 2-step verification enabled. Such accounts require an app-specific password to access less secure apps"
- geofft 8y ago
  Can you use XOAUTH2? https://developers.google.com/gmail/imap/xoauth2-protocol https://developers.google.com/gmail/imap/xoauth2-protocol offlineimap supports it, and the config file has setup instructions: https://github.com/OfflineIMAP/offlineimap/blob/master/offlineimap.conf#L900 https://github.com/OfflineIMAP/offlineimap/blob/master/offli...
- peterbecich 8y ago
  It may be possible with Thunderbird and Maildir. There is an issue: https://emacs.stackexchange.com/questions/38412/read-and-edit-thunderbirds-maildir-with-gnus https://emacs.stackexchange.com/questions/38412/read-and-edi...
djcb 8y ago
mu/mu4e author here... please let me know if you have any specific questions.
- philjackson 8y ago
  No questions. Just a "thanks" - mu4e is great.
- pfortuny 8y ago
  As you are here: there is a constant issue with address autocompletion on osx... both emacs25 and 26... But on the other hand. great job and THANKS.
- tom4000 8y ago
  For rendering HTML emails I use either shr or w3m. To get it more readable I make the text area smaller with writeroom-mode/olivetti-mode in the mu4e-view-mode-hook. But both modes apply just after opening an email a second time. Do you have any hint?
clircle 8y ago
For anyone thinking about a setup like this: skip offlineimap and use mbsync instead. It's much faster, and syncing (rather than indexing) mail is the main speed bottleneck to reading email in emacs.
- gregf 8y ago
  Could you share your mbsync config? Every time I have set it up it only pulls emails down. It doesn't sync back with the imap server for emails I have read or deleted.
- defanor 8y ago
  `doveadm sync` is also a nice option if the server supports it: combined with a custom `sendmail` command, one can both send and receive/synchronize messages over SSH, avoiding additional authentication.
aidenn0 8y ago
I'm really happy with my e-mail setup of offlineimap, afew, notmuch, bower. http://www.offlineimap.org/ http://www.offlineimap.org/ https://github.com/afewmail/afew https://github.com/afewmail/afew https://github.com/wangp/bower https://github.com/wangp/bower
dannyobrien 8y ago
I guess exploring around with setups like this is part of the command-line-and-lots-of-email-subculture's current zeitgest. I hadn't heard of mu as a CLI tool before, though I knew that mu4e was one of the emacs mail-reading options. I'm currently playing around with mblaze, which is another command line mail-reading toolkit (https://github.com/chneukirchen/mblaze https://github.com/chneukirchen/mblaze ), and building some notmuch-friendly tools on top of that ( https://github.com/dannyob/mblaze-much https://github.com/dannyob/mblaze-much ). I'm not sure it's the right way forward, but it's proving to be a fun experiment.
kashyapc 8y ago
FWIW, here's an alternative setup to consider if you don't want everything to do in Emacs (I use it daily for its wonderful Org-Mode): * OfflineIMAP -- to download and synchronize local Maildir with remote IMAP. * Mutt -- E-mail client (understatement). * Notmuch -- for fast local indexing and querying. * Postfix -- the wonderful mail transfer agent. Here[1] are some config files (from my colleague, Paul Frields) that you can adapt from. Been using the above set up for about five years. They heavily help with the high-traffic open source projects that use mailing lists for patches. (Even for general mail setup, this is still useful). These tools are a part of my day-to-day happiness at work. [1] https://pfrields.fedorapeople.org/configs/ https://pfrields.fedorapeople.org/configs/
another-cuppa 8y ago
I used to use gnus and liked it. I don't think I can use this at work because I need the exchange invitation crap which thunderbird thankfully supports. I just can't be bothered to put significant effort into making that work. But I think I should definitely look into this for home.
jpeeler 8y ago
If you're using notmuch and Gmail, I'd recommend taking a look at gmailieer - https://github.com/gauteh/gmailieer https://github.com/gauteh/gmailieer. It uses the Gmail API to fetch mail into a local maildir and keeps the tags synced bidirectionally. The downside is that for whatever reason the Gmail API doesn't support the mute label, so having that tag sync is not possible. (If a Googler reads this, please fix that! For others, there's already an issue filed as referenced in the gmailieer readme.)

167 points by gjstein 8y ago

avar 8y ago

Maybe I'm missing something obvious here, but calling "mu index" from the offlineimap postsynchook is going to suck, because mu4e invokes "mu server"[1], that aquires a Xapian lock on the index, which means you can't run the "mu index" while reading your E-Mail. This is the worst thing about mu/mu4e. You need to do the indexing with a timer in Emacs, and because Emacs is single-threaded this blocks things / slows down the UI. After trying to get it to work in some automated way I gave up. I manually issue a key-combo to re-index and then read my E-Mail. Another comment: I recommend not using Emacs to send E-Mail, but to outsource this to a local smtpd such as Exim (which I use with mu4e on Debian). Then if you don't have WiFi or whatever the smtpd just queues those up for later sending, and has queue inspection tools etc. All of the solutions for doing this in Emacs itself are dirty hacks full of edge cases and bugs smtpds have solved decades ago. Despite some rough edges, mu+mu4 (plus Exim, or similar). is an amazing tool, and the most useful E-Mail client I've ever used. 1. https://www.djcbsoftware.nl/code/mu/mu4e/mu-server.html https://www.djcbsoftware.nl/code/mu/mu4e/mu-server.html

fusiongyro 8y ago
The locking thing has not turned out to be a really serious problem for me, and I have several gigabytes of email. mu index is really fast. Also, using the Emacs sendmail hasn't been a real problem. It turns out I don't need to be productive during the ~1s it takes to send the email and I have not run into any edge cases or bugs in the ~5 years I've been rocking this setup.
- avar 8y ago
  It doesn't matter how fast the index is, only one process can hold the Xapian lock, if I'm reading my E-Mail in Emacs with mu4e (which invokes "mu server") and try to index on the command-line I get: $ mu index [...] mu: mu_store_new_writable: xapian error 'Unable to get write lock on <Maildir>/xapian: already locked' (11) How do you send E-Mail with no network connection? Last I checked this was some ad-hoc thing where you neede dto queue them up in some queue managed by Emacs, and remember to turn that mode off when you were back on WiFi (or hack up your own hooks to automate this), v.s. sending it to a local smtpd which already has retry logic built-in.
  - fusiongyro 8y ago
    Why on earth would I try and index it twice concurrently? I never have a scenario where I need to send email without network.
    - avar 8y ago
      No, you wouldn't index from two processes concurrently, but for Emacs to read your mail in mu4e and for you to concurrently issue "mu index" on the command-line two processes ("mu server" and "mu index") would need to poen the same Xapian database read-write at the same time, and just like SQLite Xapian doesn't support that.
      - fusiongyro 8y ago
        ... I still don't see why I would do that. Emacs runs offlineimap and then Emacs runs mu index. Why would I do it again myself from the command line?
      - ams6110 8y ago
        Yeah I use notmuch with Emacs and it's the same. If you try to search, refresh search results, or read email while the index process is running you get an error. It's not a big deal for me, because I download and index my mail from the command line, so I'll never be reading or sending email at the same time.
        donio 8y ago
        This used to be true but notmuch has added support for Xapian's DB_RETRY_LOCK feature a couple of years ago so if you have non-ancient notmuch (0.23+) and Xapian (1.3.2+) then the second process will wait for the lock instead of erroring out. The feature can be disabled at compile time if you don't want it for some reason.
pulisse 8y ago
> You need to do the indexing with a timer in Emacs, and because Emacs is single-threaded this blocks things / slows down the UI. It's worth noting that multithreading recently landed in Emacs master, so it's now possible to avoid this sort of problem.
- avar 8y ago
  Now possible as in mu4e would need to use some API that's not in a release version of Emacs, or is it more transparent than that?
  - pulisse 8y ago
    The former. I wasn't suggesting that mu4e should be converted right away to use this feature, but more concerned that an unqualified statement that long-running IO must block in Emacs would create the wrong impression.
kkylin 8y ago
You can configure mu4e to queue mail to send later: https://www.djcbsoftware.nl/code/mu/mu4e/Queuing-mail.html https://www.djcbsoftware.nl/code/mu/mu4e/Queuing-mail.html
- avar 8y ago
  Yeah, that's the dirty hack I was referring to. I don't need to tell an smtpd "now's when I expect not to have an Internet connection", which this method relies on, and could happen as I walk into another room and lose WiFi. I just give it mail to deliver, and if it doesn't work it keeps retrying until it works (with configurable backoff and a setting saying when to give up for good).
  - kkylin 8y ago
    Yup, I see where you're coming from. I've actually gotten used to mu4e's queuing behavior, and just leave it on all the time. For one thing, I like having a chance to edit/delete mail after hitting send (sort of like gmail's "undo"), and for another, I like the satisfaction of having a dozen or so messages sent all at once at the end of an email session.
dima55 8y ago
The locking is only a problem because he's doing it wrong, I think. I'm doing this: http://notes.secretsauce.net/notes/2014/12/12_mu4e-and-desktop-notifications.html I.e. there's an external script that invokes IMAP IDLE. And when IMAP tells it there's new mail, the script tells EMACS there's new mail. Then the reindexing happens within the emacs, and it uses the lock it already has. Makes this particular issue go away completely.
- f6e9ed793b 8y ago
  That must be it. I also trigger mu4e indexing using emacsclient, after IMAP sync. BTW you do not need to have mu4e open to index mail (and single-window mu4e mode will not have a * mu4e-main * buffer), it is enough to (progn (require 'mu4e) (mu4e-update-mail-and-index)).
jfb 8y ago
I must also be missing something because I never explicitly index my mail. I fetch it with mbsync every two minutes and then use mu4e to read it and new mail shows up as one would expect. ¯\_(ツ)_/¯
- avar 8y ago
  How many tens of GB of mail do you have? I think this problem only tends to show up on big mailboxes.
  - jfb 8y ago
    10-25GB. But a fast computer, so?
    - tom4000 8y ago
      Good for you. To save power and use smaller machines should always be an option. Indexing saves much resources here.
      - jfb 8y ago
        Agreed 100%. I'm just stating my case, not denying others'.
f6e9ed793b 8y ago
> Maybe I'm missing something obvious here, but calling "mu index" from the offlineimap postsynchook is going to suck, because mu4e invokes "mu server"[1], that aquires a Xapian lock on the index, which means you can't run the "mu index" while reading your E-Mail. > This is the worst thing about mu/mu4e. You need to do the indexing with a timer in Emacs, and because Emacs is single-threaded this blocks things / slows down the UI. After trying to get it to work in some automated way I gave up. I manually issue a key-combo to re-index and then read my E-Mail. I tried both approaches (actually, I had mu4e reindex on `g` query refresh, no need for separate keybindings), and prefer having indexing run after IMAP sync. I don't notice any problems (and I mostly use very slow Atom hardware). > Another comment: I recommend not using Emacs to send E-Mail, but to outsource this to a local smtpd such as Exim (which I use with mu4e on Debian). Then if you don't have WiFi or whatever the smtpd just queues those up for later sending, and has queue inspection tools etc. All of the solutions for doing this in Emacs itself are dirty hacks full of edge cases and bugs smtpds have solved decades ago. FUD. smtpmail.el can do this. smtpmail.el can queue mail, and you can run it in a separate Emacs process periodically or on demand, to attempt to deliver those emails. The emails are queued as files in a directory, so you don't need separate queue inspection tools. I run smtpmail.el this way with mu4e, and think it is a far superior solution to having to install and configure yet another external program.
michaelmrose 8y ago
I use imapnotify to connect to my imap server and respond only when a mail is received by running offlineimap to sync the mail and telling mu4e via emacsclient to sync. Since mu is never ever called except by emacs and only exactly when an email has come in there is never contention for the lock.
nickysielicki 8y ago
I recommend notmuch in place of mu/mu4e and msmtp in place of a full-blown smtpd implementation. I also recommend isync's mbsync over offlineimap
- tom4000 8y ago
  Agree with using Isync. Notmuch lacks the backward synchronization to the IMAP server and to another mail client then notmuch. I primarily use mu4e but want to see changes on my mobile as well as on any remote computer using a web client in a browser. As far as I know notmuch can just be synchronized to another notmuch instance.
  - nickysielicki 8y ago
    If you use https://github.com/gauteh/gmailieer https://github.com/gauteh/gmailieer you can have bidirectional sync between gmail tags and notmuch tags. Personally, I don't bother with it. When I have to do mail on my phone, it's usually so limited in scope that I don't need to retain state when I sit down at my main computer. I can imagine that's not for everyone, though.
arnoooooo 8y ago
My mail fetching script kills mu before calling "mu index", like so : $ pgrep mu && kill `pgrep mu` $ mu index It works perfectly. Mu4e then restarts mu when it needs it. I think I saw that as the recommended setup somewhere in the mu docs.
- u801e 8y ago
  > $ pgrep mu && kill `pgrep mu` This could be rewritten as: $ pkill mu
- tom4000 8y ago
  With `pkill $(pgrep mu)` or even with `pkill mu` you can kill mutt and any mu$ process as well. `pkill -x mu` matches the whole name without substrings.

autopoiesis 8y ago

How does this compare to using notmuch-emacs? And can someone say whether notmuch or mu is preferable these days?

dkhenry 8y ago
I don't have a lot of experience with mu, but I went with notmuch and have really liked it. It reminds me a lot of the standard gmail interface.
nextos 8y ago
I have extensively used both, and spent some time using Gnus too. Plus a lot of time on Mutt. In my opinion, Notmuch is better than Mu because it has a really neat architecture that allows customizing it to embrace any possible email workflow with a few trivial Bash commands. Notmuch is purely tag based. It will never ever modify your mailbox. It's your task to map tags to imperative mailbox actions (delete, move...) using some shell hooks. Most workflows can be trivially implemented by calling the notmuch backend and piping the output to mv or rm using xargs or parallel. Since searching in Notmuch is really quick, I recommend never manually assigning tags and tagging everything with rules written in the backend for those repetitive searches that you want a shortcut for. This is not specific to Notmuch, but also applies to Mu. Mu has a hybrid tag/folder model, and the interface resembles Mutt in some ways. The other key difference with Notmuch is that Mu stores tags using a special header in each message. Notmuch uses a separate database, which is not synced across computers (unless you rsync it explicitly). I prefer the latter approach, as modifying your mailbox to store tags is a bit ugly and slow. I also favor not tagging things manually, but using rules. Then syncing tags becomes unnecessary.
- BeetleB 8y ago
  Thanks for the comparison. I'm wondering what the benefits of mu4e are over notmuch (your comment doesn't list any - which makes sense because you preferred notmuch). The reason I ask: When I look around, I tend to see a lot more references to mu4e than to notmuch. Is it just easier to start with, or does it actually have some nicer features that notmuch doesn't? Not sure if it's relevant, but: I download all emails and do not want to sync with the server.

federicotdn 8y ago

Not completely related, but I have a question regarding Emacs and email: Is there a way to read/write emails on Emacs, without having any emails or credentials stored locally as plaintext? I'm heavily prioritizing ease of configuration and security over speed or other additional features.

avar 8y ago
Sure, you sync them locally with e.g. offlineimap, and specify a "remotepasseval" key in your config which can invoke a command that invokes whatever password manager you use, same for sending. I don't do this and just keep my password in plain-text and use full-disk-encryption, because I don't want to have to be constantly unlocking a keyring to have some background E-Mail syncing job work.
pfortuny 8y ago
If you save the passwords in a gpg-encrypted file, emacs primpts you for the password to the file when needed. Otherwise, password-read or something like that.
aerique 8y ago
I don't know if the others use it but Gnus can read credentials from an .authinfo.gpg file. https://www.gnu.org/software/emacs/manual/html_node/auth/GnuPG-and-EasyPG-Assistant-Configuration.html https://www.gnu.org/software/emacs/manual/html_node/auth/Gnu...
- globuous 8y ago
  That's what I do actually: https://www.emacswiki.org/emacs/GnusAuthinfo https://www.emacswiki.org/emacs/GnusAuthinfo When I set it up with gmail tough, I think you either need a 2fa token (which requires 2fa on gmail) or tick "allow unsafe apps" in the admin panel, otherwise you get rejected by gmail. I'm sorry I don't have much info, it's already been a few months I've set it up. However, if anyone's struggling, I'll be glad to help ;)
PuercoPop 8y ago
Sure. Although the documentation doesnt make it explicit your .authinfo file can have a gpg extension and Emacs will encrypt and decrypt it for you using gpg automatically

Myrmornis 8y ago

I found trying to do this sort of thing (e.g. gnus + dovecot) a very helpful introduction to software engineering; once I got a job as a software engineer I found I didn't have time for a local setup that needed so much attention.

pfortuny 8y ago
It needs no attention once you set it up. And the first steup is a bunch of copy-pastes and minor customization.
- Myrmornis 8y ago
  No, I'm speaking from experience unfortunately! Things break, emacs changes, your OS changes, you decide you need a more recent version of something in order to get a certain feature or bug fix. You get a new laptop. Etc.
  - microcolonel 8y ago
    Not to take away from what you have said, I and several others I know have used Gnus for roughly the last five years without any memorable change in configuration, across three major releases of Emacs.
  - michaericalribo 8y ago
    I agree. I’ve learned immensely from using tools like mu and offlineimap, writing utilities for a maildir, and it can be great fun...but I need direct and unmediated access to email, I need it to ‘just work’ and let me copy-paste html text and images and drag and drop attachments. I have to use email to get other stuff done, I can’t spend time focusing on email itself.
  - pfortuny 8y ago
    Yeah, I see what you mean. I tend to agree.

NesteaIsBad 8y ago

I use Fastmail and the interface is so good that I can say this solution is really crappy.

unhammer 8y ago
I use Fastmail and think their web interface is great, but I still also love having offline access to my mail[1], and being able to create an org-mode TODO from an email with a link back to the message, and being able to hit a key to open a new message while coding. Also, every day I get an email with some log files from a service at work; because `mbsync` puts these in my Maildir, I can run a script (using `ripmime`) to extract the logs (going all the way back in time) and show me a report etc., which is so much faster than if I were to get those attachments from a GUI. [1] I'm still spooked from the time my Gmail account was suspended for no apparent reason, having to contact support and being without email for nearly a week.

kkylin 8y ago

I've been using mu4e for a while, and despite some rough edges (e.g., I have the same dissatisfaction with timers and xapian lock that others have), I've been pretty happy with the set-up. I did find that for working with gmail, mbsync works more consistently than offlineimap.

arohner 8y ago

Is there any Emacs-based solution to access a GMail account with 2FA, without enabling "insecure access" on Gmail? I've looked at a lot of solutions, and AFAICT: - SMTP + app-based passwords require "insecure access" being enabled -Using the GMail HTTP API does not require insecure access, but none of the standard CLI email tools support it.

kuwze 8y ago
I'm going to break the HN code of ethics (oh gosh) and explicitly bump this. All of the solutions I have found just point at the insecure access option.
Jim_Heckler 8y ago
As far as I can tell, when 2FA is enabled app passwords are the only non-2FA way to use your account. I have an app password in my .authinfo.gpg that I use for authorizing offlineimap and SMTP through Emacs. Obviously this isn't as secure as 2FA, but since the password is PGP encrypted I don't worry about it ever being comprised.
Allaman 8y ago
Just looked at my account. There is no option for insecure access when you activate 2FA. From https://myaccount.google.com/lesssecureapps https://myaccount.google.com/lesssecureapps "This setting is not available for accounts with 2-step verification enabled. Such accounts require an app-specific password to access less secure apps"
geofft 8y ago
Can you use XOAUTH2? https://developers.google.com/gmail/imap/xoauth2-protocol https://developers.google.com/gmail/imap/xoauth2-protocol offlineimap supports it, and the config file has setup instructions: https://github.com/OfflineIMAP/offlineimap/blob/master/offlineimap.conf#L900 https://github.com/OfflineIMAP/offlineimap/blob/master/offli...
peterbecich 8y ago
It may be possible with Thunderbird and Maildir. There is an issue: https://emacs.stackexchange.com/questions/38412/read-and-edit-thunderbirds-maildir-with-gnus https://emacs.stackexchange.com/questions/38412/read-and-edi...

djcb 8y ago

mu/mu4e author here... please let me know if you have any specific questions.

philjackson 8y ago
No questions. Just a "thanks" - mu4e is great.
pfortuny 8y ago
As you are here: there is a constant issue with address autocompletion on osx... both emacs25 and 26... But on the other hand. great job and THANKS.
tom4000 8y ago
For rendering HTML emails I use either shr or w3m. To get it more readable I make the text area smaller with writeroom-mode/olivetti-mode in the mu4e-view-mode-hook. But both modes apply just after opening an email a second time. Do you have any hint?

clircle 8y ago

For anyone thinking about a setup like this: skip offlineimap and use mbsync instead. It's much faster, and syncing (rather than indexing) mail is the main speed bottleneck to reading email in emacs.

gregf 8y ago
Could you share your mbsync config? Every time I have set it up it only pulls emails down. It doesn't sync back with the imap server for emails I have read or deleted.
defanor 8y ago
`doveadm sync` is also a nice option if the server supports it: combined with a custom `sendmail` command, one can both send and receive/synchronize messages over SSH, avoiding additional authentication.

aidenn0 8y ago

I'm really happy with my e-mail setup of offlineimap, afew, notmuch, bower. http://www.offlineimap.org/ http://www.offlineimap.org/ https://github.com/afewmail/afew https://github.com/afewmail/afew https://github.com/wangp/bower https://github.com/wangp/bower

dannyobrien 8y ago

I guess exploring around with setups like this is part of the command-line-and-lots-of-email-subculture's current zeitgest. I hadn't heard of mu as a CLI tool before, though I knew that mu4e was one of the emacs mail-reading options. I'm currently playing around with mblaze, which is another command line mail-reading toolkit (https://github.com/chneukirchen/mblaze https://github.com/chneukirchen/mblaze ), and building some notmuch-friendly tools on top of that ( https://github.com/dannyob/mblaze-much https://github.com/dannyob/mblaze-much ). I'm not sure it's the right way forward, but it's proving to be a fun experiment.

kashyapc 8y ago

FWIW, here's an alternative setup to consider if you don't want everything to do in Emacs (I use it daily for its wonderful Org-Mode): * OfflineIMAP -- to download and synchronize local Maildir with remote IMAP. * Mutt -- E-mail client (understatement). * Notmuch -- for fast local indexing and querying. * Postfix -- the wonderful mail transfer agent. Here[1] are some config files (from my colleague, Paul Frields) that you can adapt from. Been using the above set up for about five years. They heavily help with the high-traffic open source projects that use mailing lists for patches. (Even for general mail setup, this is still useful). These tools are a part of my day-to-day happiness at work. [1] https://pfrields.fedorapeople.org/configs/ https://pfrields.fedorapeople.org/configs/

another-cuppa 8y ago

I used to use gnus and liked it. I don't think I can use this at work because I need the exchange invitation crap which thunderbird thankfully supports. I just can't be bothered to put significant effort into making that work. But I think I should definitely look into this for home.

jpeeler 8y ago

If you're using notmuch and Gmail, I'd recommend taking a look at gmailieer - https://github.com/gauteh/gmailieer https://github.com/gauteh/gmailieer. It uses the Gmail API to fetch mail into a local maildir and keeps the tags synced bidirectionally. The downside is that for whatever reason the Gmail API doesn't support the mute label, so having that tag sync is not possible. (If a Googler reads this, please fix that! For others, there's already an issue filed as referenced in the gmailieer readme.)