4 ms·

I use sieve as part of my home-brew spam filter setup, but I hate it. The syntax is atrocious and the documentation is abysmal. The only way I was able to get

by lisper 5y ago

I use sieve as part of my home-brew spam filter setup, but I hate it. The syntax is atrocious and the documentation is abysmal. The only way I was able to get my setup to work was by copying examples I found on the web and incrementally tweaking them by trial and error until they did the right thing. It took days and the end result was only about a dozen lines of code. How hard could it be to design a sane mail filtering language? I guess I'll just have to take that on one of these days.

nijave 5y ago
Not a huge fan of sieve either. For spam filtering, I've spent some time over the last few years tweaking spamassassin weights and have had pretty decent results with that. I think this was the one I started with for SA which was pretty decent then did some additional tweaks based on what I was seeing come in https://gist.github.com/ishmaelen/6c7a209dd00f9d2b968fd2bb0e8d3729 https://gist.github.com/ishmaelen/6c7a209dd00f9d2b968fd2bb0e...
upofadown 5y ago
FDM seems to have a nice way to define filters: https://en.wikipedia.org/wiki/Fdm_(software) https://en.wikipedia.org/wiki/Fdm_(software) https://acampbell.uk/wp/2019/04/11/replacing-procmail-with-fdm/ https://acampbell.uk/wp/2019/04/11/replacing-procmail-with-f... https://hackriculture.fr/tag/fdm.html https://hackriculture.fr/tag/fdm.html https://manpages.debian.org/testing/fdm/fdm.conf.5.en.html https://manpages.debian.org/testing/fdm/fdm.conf.5.en.html
- ninjin 5y ago
  Seconding fdm. I have about 1,000 lines of rules to process when I fetch via POP3 and it has decent debugging facilities as well. Not to mention, it is fast and comes with a friendly – but powerful – syntax.
wahern 5y ago
If I had the time I'd love to simply reimplement procmail. In principal its exceedingly simple as procmail doesn't parse structured headers or entity bodies; the expressions are applied to the raw lines. The fact procmail abstains from more complex parsing is probably why it's seen as few bugs as it has given the approach to parsing--classic Unix string slicing and dicing rather than a less bug-prone stream of scalar tokens which I personally prefer when doing string processing in C. (Can't overflow buffers when you're not taking pointers into and copying fragments every which way.) Probably a good exercise as a Rust project, but I run OpenBSD so C (or C+Perl or C+Lua--Lua making it trivial to build a single binary) would probably be a better fit--no runtime or even build dependencies. An interim measure for OpenBSD might be to apply pledge and unveil sandboxing. I keep meaning to inspect how procmail loads and parses procmailrc to see if they could be applied early enough to provide realistic mitigations.
- pabs3 5y ago
  procmail is now on GitHub, so perhaps you could contribute there rather than reimplementing? https://github.com/BuGlessRB/procmail https://github.com/BuGlessRB/procmail
  - wahern 5y ago
    Aye, will check it out. EDIT: The original author has resumed maintenance?
    - pabs3 5y ago
      Correct, although I don't know how much maintenance it will get or for how long.
bogantech 5y ago
Roundcube has a sieve plugin and makes it very easy to manage Sieve rules. I don't need webmail at all but it's useful just for this.
JNRowe 5y ago
Iff you like Python, then mailprocessing¹ is a nice solution for this. You just write a small Python script to implement your rules: for mail in processor: if mail['X-Bugzilla-URL'].contains('zephyr-ds'): mail.move('Zephyr') elif ... Given the layer it operates on you don't need to worry quite so much about processing errors dropping mails either. ¹ https://mailprocessing.github.io/mailprocessing/ https://mailprocessing.github.io/mailprocessing/
- polski-g 5y ago
  How well does it handle non-ascii character sets or non-compliance emails? I haven't been able to find a mail processing library that gracefully supports something like Arabic characters in the headers/body and also does not include a content-encoding header.
  - JNRowe 5y ago
    Behind the scenes mailprocessing basically works by executing your Python configuration file with some injected globals to make processing easier, so you can apply any hacks you feel like to workaround breakages in the incoming mail. In the specific case of Arabic characters in headers, you can see from the source¹ that when decoding fails an error is logged but you're still able to work on that header. How much work you're willing to put in to recover from that error by guessing charsets(perhaps via chardet²) is up to you. The level of customisation that is possible is my favourite feature of this type of software. You can start with a config that is just a couple of lines, but it grows with you with no artificial ceiling imposed by the software author's imagination. ¹ https://github.com/mailprocessing/mailprocessing/blob/master/mailprocessing/mail/maildir.py#L119 https://github.com/mailprocessing/mailprocessing/blob/master... ² https://github.com/chardet/chardet https://github.com/chardet/chardet

by lisper 5y ago

nijave 5y ago

Not a huge fan of sieve either. For spam filtering, I've spent some time over the last few years tweaking spamassassin weights and have had pretty decent results with that. I think this was the one I started with for SA which was pretty decent then did some additional tweaks based on what I was seeing come in https://gist.github.com/ishmaelen/6c7a209dd00f9d2b968fd2bb0e8d3729 https://gist.github.com/ishmaelen/6c7a209dd00f9d2b968fd2bb0e...

upofadown 5y ago

FDM seems to have a nice way to define filters: https://en.wikipedia.org/wiki/Fdm_(software) https://en.wikipedia.org/wiki/Fdm_(software) https://acampbell.uk/wp/2019/04/11/replacing-procmail-with-fdm/ https://acampbell.uk/wp/2019/04/11/replacing-procmail-with-f... https://hackriculture.fr/tag/fdm.html https://hackriculture.fr/tag/fdm.html https://manpages.debian.org/testing/fdm/fdm.conf.5.en.html https://manpages.debian.org/testing/fdm/fdm.conf.5.en.html

ninjin 5y ago
Seconding fdm. I have about 1,000 lines of rules to process when I fetch via POP3 and it has decent debugging facilities as well. Not to mention, it is fast and comes with a friendly – but powerful – syntax.

wahern 5y ago

If I had the time I'd love to simply reimplement procmail. In principal its exceedingly simple as procmail doesn't parse structured headers or entity bodies; the expressions are applied to the raw lines. The fact procmail abstains from more complex parsing is probably why it's seen as few bugs as it has given the approach to parsing--classic Unix string slicing and dicing rather than a less bug-prone stream of scalar tokens which I personally prefer when doing string processing in C. (Can't overflow buffers when you're not taking pointers into and copying fragments every which way.) Probably a good exercise as a Rust project, but I run OpenBSD so C (or C+Perl or C+Lua--Lua making it trivial to build a single binary) would probably be a better fit--no runtime or even build dependencies. An interim measure for OpenBSD might be to apply pledge and unveil sandboxing. I keep meaning to inspect how procmail loads and parses procmailrc to see if they could be applied early enough to provide realistic mitigations.

pabs3 5y ago
procmail is now on GitHub, so perhaps you could contribute there rather than reimplementing? https://github.com/BuGlessRB/procmail https://github.com/BuGlessRB/procmail
- wahern 5y ago
  Aye, will check it out. EDIT: The original author has resumed maintenance?
  - pabs3 5y ago
    Correct, although I don't know how much maintenance it will get or for how long.

bogantech 5y ago

Roundcube has a sieve plugin and makes it very easy to manage Sieve rules. I don't need webmail at all but it's useful just for this.

JNRowe 5y ago

Iff you like Python, then mailprocessing¹ is a nice solution for this. You just write a small Python script to implement your rules: for mail in processor: if mail['X-Bugzilla-URL'].contains('zephyr-ds'): mail.move('Zephyr') elif ... Given the layer it operates on you don't need to worry quite so much about processing errors dropping mails either. ¹ https://mailprocessing.github.io/mailprocessing/ https://mailprocessing.github.io/mailprocessing/

polski-g 5y ago
How well does it handle non-ascii character sets or non-compliance emails? I haven't been able to find a mail processing library that gracefully supports something like Arabic characters in the headers/body and also does not include a content-encoding header.
- JNRowe 5y ago
  Behind the scenes mailprocessing basically works by executing your Python configuration file with some injected globals to make processing easier, so you can apply any hacks you feel like to workaround breakages in the incoming mail. In the specific case of Arabic characters in headers, you can see from the source¹ that when decoding fails an error is logged but you're still able to work on that header. How much work you're willing to put in to recover from that error by guessing charsets(perhaps via chardet²) is up to you. The level of customisation that is possible is my favourite feature of this type of software. You can start with a config that is just a couple of lines, but it grows with you with no artificial ceiling imposed by the software author's imagination. ¹ https://github.com/mailprocessing/mailprocessing/blob/master/mailprocessing/mail/maildir.py#L119 https://github.com/mailprocessing/mailprocessing/blob/master... ² https://github.com/chardet/chardet https://github.com/chardet/chardet