6 ms·

Bichon: A lightweight, high-performance Rust email archiver with WebUI

62 points by rendx 9mo ago

via https://news.ycombinator.com/item?id=46429250 https://news.ycombinator.com/item?id=46429250

deleted 9mo ago
[deleted]
Onavo 9mo ago
Note the rustmailer project mentioned is heavily proprietary licensed. Don't expect to be able to use any of this.
- DetroitThrow 9mo ago
  This one appears to be AGPL and safe to use, but I just read the main project license and frowned. I'm not sure I understand who that project is for.
  - sea-gold 9mo ago
    Bichon is in fact AGPL: https://github.com/rustmailer/bichon/blob/main/LICENSE https://github.com/rustmailer/bichon/blob/main/LICENSE It must not be using any RustMailer code.
    - copirate 9mo ago
      If it's the same author they can reuse the code. They can copy the code but they can't link it.
- jszymborski 9mo ago
  actually the rustmailer licensing is a bit confusing. The license file in the repo has the text of the BSD Zero and Apache license but the README describes it as "source-available"?
  - DetroitThrow 9mo ago
    I read the license MD and there are some clauses that make it incompatible with Apache. I think you might just have seen the disclosure text for Apache/BSD zero.
- sea-gold 9mo ago
  RustMailer (a separate project) has a commercial license: https://github.com/rustmailer/rustmailer/blob/main/LICENSE https://github.com/rustmailer/rustmailer/blob/main/LICENSE > RustMailer is source-available. The code is open on GitHub, but requires a valid commercial license key for production use. License Purchase: https://rustmailer.com/pricing https://rustmailer.com/pricing The mention of BSD/Apache is from here[1] and looks to be a listing of open-source licenses that this project uses. [1] https://github.com/rustmailer/rustmailer/blob/main/license.html https://github.com/rustmailer/rustmailer/blob/main/license.h...
- rustmailer 9mo ago
  I’m the author of Bichon, so let me clarify this. First, rustmailer is my GitHub username. Under this GitHub account there are two main projects: RustMailer and Bichon. RustMailer is a middleware product with the same name. It is commercially licensed. I hope it can help developers save time on email-related development and also generate a bit of income for me, so this middleware requires a license key to run long-term. Bichon, on the other hand, is an independent project built on the experience of developing RustMailer. It does reuse a lot of similar code, but it is licensed under the AGPL. These are two completely different projects. RustMailer provides many API interfaces similar to EmailEngine and is positioned as middleware, while Bichon is an email archiving (or unified search) project. Bichon is positioned as a permanently free project. Users never need to worry that if you use Bichon today you will be asked to pay later — you can use it with confidence, and all of its code is fully open and transparent.
znnajdla 9mo ago
I’m been meaning to build something like this myself. Seems like this was designed to be run on a server, however. What if I want to run it locally on a laptop that’s not always on? Does it recover gracefully from restarts and such? Is it a SQLite database I can query myself? Does it have an immutable archive/backup mode with trash recovery?
- sea-gold 9mo ago
  Sorry, I don't know all the answers. However, I do know this (from the README): > Core Engine (Storage & Search): Tantivy[1] > Acts as both the primary storage for email content and the full-text search index. This unified approach ensures high performance and eliminates data redundancy. [1] https://github.com/quickwit-oss/tantivy https://github.com/quickwit-oss/tantivy
  - rustmailer 9mo ago
    You are right.
- wwfn 9mo ago
  If looking for local email, why not a traditional client (thunderbird, claws, even outlook) or the more flexible/cli friendly maildir and notmuch? There are a bunch of front ends, including WebUIs https://notmuchmail.org/frontends/ https://notmuchmail.org/frontends/ Is what you're looking for a pretty good fit for how email was originally used? Or am missing something obvious
  - pepa65 9mo ago
    Actually, that is what I would like, but if I use Thunderbird on my Maildirs (served by Dovecot), the search (particularly in the mail body) is not really feasible. Perhaps there is a search integration for Dovecot that I am not aware of??
    - pepa65 9mo ago
      I just learned that I can enable full text search on the server on Dovecot through Xapian, which is now indexing..!
tosapple 9mo ago
I've got bichons, and they're great dogs.
V__ 9mo ago
A small warning for anyone wanting to use this in a setting with legal requirements: > Please note that the term “sync” in Bichon may be misleading—“download” might be more accurate. [1] If an email is deleted before bichon can download it, then it may not get archived. Most of the time the legal requirements are, that all emails hitting an inbox have to be archived, regardless of user action. For those cases, a solution like mailpiler is better. Just BCC any incoming and outgoing mail to mailpiler and that is it. [1] https://github.com/rustmailer/bichon/wiki/FAQ-(Frequently-Asked-Questions) https://github.com/rustmailer/bichon/wiki/FAQ-(Frequently-As...
- rustmailer 9mo ago
  Bichon connects to the IMAP server via IMAP and uses a sync interval, for example 10 minutes. Every 10 minutes, Bichon downloads emails from the IMAP server to the local machine, then stores and indexes them. If an email is permanently deleted within that interval, Bichon has no way to detect it. That is indeed how Bichon currently works.
  - pabs3 9mo ago
    IMAP has mechanisms to notify clients about incoming emails (IIRC IMAP IDLE), have you considered implementing those?
    - rustmailer 9mo ago
      IMAP IDLE tends to be unstable and unreliable in real-world deployments, especially for long-running background services. It also introduces significant operational and implementation complexity, which makes it harder to maintain and reason about. For a system like Bichon, a simpler diff-based approach is usually easier to operate and more predictable.
      - pabs3 9mo ago
        Could you use IDLE to trigger the diffs?
jszymborski 9mo ago
Thank god! I hacked together some awful pipeline where Sieve copies mail to a folder and a python script read, stored, and deleted the email. Super fragile, using an email folder as a queue kinda sucked.
nubg 9mo ago
Ah this is so close. If it could have a s3 backend for storage of the email blobs (should be trivial since the email files are just key values from email id to email bytes) one could use it as a backup tool, too. Author, is anything like that possible/planned? (the index itself can stay filesystem only)
- rustmailer 9mo ago
  I am the author of Bichon. Long-term backup and permanent archival features are not currently on Bichon’s roadmap, but you’re very welcome to open a GitHub issue to share your idea. Bichon is entirely driven by user needs, and any good suggestions or plans can be adopted and implemented.
mbStavola 9mo ago
Bichons and Rust? I'm already sold!
sea-gold 9mo ago
Here is a very similar project: https://emailengine.app/ https://emailengine.app/ Similar to Bichon, it also has a commercial license[1]: > EmailEngine is not free open-source software. It is "source available" software, meaning you can view and copy the source code, but you need a paid subscription[2] to run it beyond the free 14-day trial. Each EmailEngine instance comes with this trial, so you can test EmailEngine without any commitment. EmailEngine is based on ImapFlow[3], a modern and easy-to-use IMAP client library for Node.js that is MIT-licensed. [1] https://github.com/postalsys/emailengine/blob/master/LICENSE_EMAILENGINE.txt https://github.com/postalsys/emailengine/blob/master/LICENSE... [2] https://postalsys.com/plans https://postalsys.com/plans [3] https://github.com/postalsys/imapflow https://github.com/postalsys/imapflow
- isaachinman 9mo ago
  For anyone reading, I would highly recommend steering clear of EmailEngine.
  - cipehr 9mo ago
    Why?
    - andris9 9mo ago
      EmailEngine author here. The commenter tried the EmailEngine trial back in 2024 and appears to have had a negative experience. Since then, he’s repeatedly criticized EmailEngine and related components like the ImapFlow IMAP library, often while promoting his own product.
      - isaachinman 9mo ago
        Yes indeed. The criticism is well-founded and comes from months of experience with the product.
      - rustmailer 9mo ago
        It’s a great honor to be able to talk with Andris. A few months ago I learned that the author of EmailEngine not only created EmailEngine, but is also behind many foundational Node.js email libraries that are widely used. My own projects, RustMailer and Bichon, are built on the shoulders of many great Rust email libraries. EmailEngine is undoubtedly a success, and Andris has spent years quietly and diligently contributing to these core libraries. I have deep respect for you.
KellyCriterion 9mo ago
Raise your hand if you read the headline first as: "Bitcoin: A leightweight, high-performance... Rust email" - then I re-read because I was irritated because of the first Bitcoin token and then adding email :-D LOL

62 points by rendx 9mo ago

via https://news.ycombinator.com/item?id=46429250 https://news.ycombinator.com/item?id=46429250

deleted 9mo ago

[deleted]

Onavo 9mo ago

Note the rustmailer project mentioned is heavily proprietary licensed. Don't expect to be able to use any of this.

DetroitThrow 9mo ago
This one appears to be AGPL and safe to use, but I just read the main project license and frowned. I'm not sure I understand who that project is for.
- sea-gold 9mo ago
  Bichon is in fact AGPL: https://github.com/rustmailer/bichon/blob/main/LICENSE https://github.com/rustmailer/bichon/blob/main/LICENSE It must not be using any RustMailer code.
  - copirate 9mo ago
    If it's the same author they can reuse the code. They can copy the code but they can't link it.
jszymborski 9mo ago
actually the rustmailer licensing is a bit confusing. The license file in the repo has the text of the BSD Zero and Apache license but the README describes it as "source-available"?
- DetroitThrow 9mo ago
  I read the license MD and there are some clauses that make it incompatible with Apache. I think you might just have seen the disclosure text for Apache/BSD zero.
sea-gold 9mo ago
RustMailer (a separate project) has a commercial license: https://github.com/rustmailer/rustmailer/blob/main/LICENSE https://github.com/rustmailer/rustmailer/blob/main/LICENSE > RustMailer is source-available. The code is open on GitHub, but requires a valid commercial license key for production use. License Purchase: https://rustmailer.com/pricing https://rustmailer.com/pricing The mention of BSD/Apache is from here[1] and looks to be a listing of open-source licenses that this project uses. [1] https://github.com/rustmailer/rustmailer/blob/main/license.html https://github.com/rustmailer/rustmailer/blob/main/license.h...
rustmailer 9mo ago
I’m the author of Bichon, so let me clarify this. First, rustmailer is my GitHub username. Under this GitHub account there are two main projects: RustMailer and Bichon. RustMailer is a middleware product with the same name. It is commercially licensed. I hope it can help developers save time on email-related development and also generate a bit of income for me, so this middleware requires a license key to run long-term. Bichon, on the other hand, is an independent project built on the experience of developing RustMailer. It does reuse a lot of similar code, but it is licensed under the AGPL. These are two completely different projects. RustMailer provides many API interfaces similar to EmailEngine and is positioned as middleware, while Bichon is an email archiving (or unified search) project. Bichon is positioned as a permanently free project. Users never need to worry that if you use Bichon today you will be asked to pay later — you can use it with confidence, and all of its code is fully open and transparent.

znnajdla 9mo ago

I’m been meaning to build something like this myself. Seems like this was designed to be run on a server, however. What if I want to run it locally on a laptop that’s not always on? Does it recover gracefully from restarts and such? Is it a SQLite database I can query myself? Does it have an immutable archive/backup mode with trash recovery?

sea-gold 9mo ago
Sorry, I don't know all the answers. However, I do know this (from the README): > Core Engine (Storage & Search): Tantivy[1] > Acts as both the primary storage for email content and the full-text search index. This unified approach ensures high performance and eliminates data redundancy. [1] https://github.com/quickwit-oss/tantivy https://github.com/quickwit-oss/tantivy
- rustmailer 9mo ago
  You are right.
wwfn 9mo ago
If looking for local email, why not a traditional client (thunderbird, claws, even outlook) or the more flexible/cli friendly maildir and notmuch? There are a bunch of front ends, including WebUIs https://notmuchmail.org/frontends/ https://notmuchmail.org/frontends/ Is what you're looking for a pretty good fit for how email was originally used? Or am missing something obvious
- pepa65 9mo ago
  Actually, that is what I would like, but if I use Thunderbird on my Maildirs (served by Dovecot), the search (particularly in the mail body) is not really feasible. Perhaps there is a search integration for Dovecot that I am not aware of??
  - pepa65 9mo ago
    I just learned that I can enable full text search on the server on Dovecot through Xapian, which is now indexing..!

tosapple 9mo ago

I've got bichons, and they're great dogs.

V__ 9mo ago

A small warning for anyone wanting to use this in a setting with legal requirements: > Please note that the term “sync” in Bichon may be misleading—“download” might be more accurate. [1] If an email is deleted before bichon can download it, then it may not get archived. Most of the time the legal requirements are, that all emails hitting an inbox have to be archived, regardless of user action. For those cases, a solution like mailpiler is better. Just BCC any incoming and outgoing mail to mailpiler and that is it. [1] https://github.com/rustmailer/bichon/wiki/FAQ-(Frequently-Asked-Questions) https://github.com/rustmailer/bichon/wiki/FAQ-(Frequently-As...

rustmailer 9mo ago
Bichon connects to the IMAP server via IMAP and uses a sync interval, for example 10 minutes. Every 10 minutes, Bichon downloads emails from the IMAP server to the local machine, then stores and indexes them. If an email is permanently deleted within that interval, Bichon has no way to detect it. That is indeed how Bichon currently works.
- pabs3 9mo ago
  IMAP has mechanisms to notify clients about incoming emails (IIRC IMAP IDLE), have you considered implementing those?
  - rustmailer 9mo ago
    IMAP IDLE tends to be unstable and unreliable in real-world deployments, especially for long-running background services. It also introduces significant operational and implementation complexity, which makes it harder to maintain and reason about. For a system like Bichon, a simpler diff-based approach is usually easier to operate and more predictable.
    - pabs3 9mo ago
      Could you use IDLE to trigger the diffs?

jszymborski 9mo ago

Thank god! I hacked together some awful pipeline where Sieve copies mail to a folder and a python script read, stored, and deleted the email. Super fragile, using an email folder as a queue kinda sucked.

nubg 9mo ago

Ah this is so close. If it could have a s3 backend for storage of the email blobs (should be trivial since the email files are just key values from email id to email bytes) one could use it as a backup tool, too. Author, is anything like that possible/planned? (the index itself can stay filesystem only)

rustmailer 9mo ago
I am the author of Bichon. Long-term backup and permanent archival features are not currently on Bichon’s roadmap, but you’re very welcome to open a GitHub issue to share your idea. Bichon is entirely driven by user needs, and any good suggestions or plans can be adopted and implemented.

mbStavola 9mo ago

Bichons and Rust? I'm already sold!

sea-gold 9mo ago

Here is a very similar project: https://emailengine.app/ https://emailengine.app/ Similar to Bichon, it also has a commercial license[1]: > EmailEngine is not free open-source software. It is "source available" software, meaning you can view and copy the source code, but you need a paid subscription[2] to run it beyond the free 14-day trial. Each EmailEngine instance comes with this trial, so you can test EmailEngine without any commitment. EmailEngine is based on ImapFlow[3], a modern and easy-to-use IMAP client library for Node.js that is MIT-licensed. [1] https://github.com/postalsys/emailengine/blob/master/LICENSE_EMAILENGINE.txt https://github.com/postalsys/emailengine/blob/master/LICENSE... [2] https://postalsys.com/plans https://postalsys.com/plans [3] https://github.com/postalsys/imapflow https://github.com/postalsys/imapflow

isaachinman 9mo ago
For anyone reading, I would highly recommend steering clear of EmailEngine.
- cipehr 9mo ago
  Why?
  - andris9 9mo ago
    EmailEngine author here. The commenter tried the EmailEngine trial back in 2024 and appears to have had a negative experience. Since then, he’s repeatedly criticized EmailEngine and related components like the ImapFlow IMAP library, often while promoting his own product.
    - isaachinman 9mo ago
      Yes indeed. The criticism is well-founded and comes from months of experience with the product.
    - rustmailer 9mo ago
      It’s a great honor to be able to talk with Andris. A few months ago I learned that the author of EmailEngine not only created EmailEngine, but is also behind many foundational Node.js email libraries that are widely used. My own projects, RustMailer and Bichon, are built on the shoulders of many great Rust email libraries. EmailEngine is undoubtedly a success, and Andris has spent years quietly and diligently contributing to these core libraries. I have deep respect for you.

KellyCriterion 9mo ago

Raise your hand if you read the headline first as: "Bitcoin: A leightweight, high-performance... Rust email" - then I re-read because I was irritated because of the first Bitcoin token and then adding email :-D LOL