3 ms·

Always interesting to see new MTAs! I'm not convinced that storing mails (including attachments) in MongoDB is a good solution. What's wrong with the filesystem

by Tepix 8y ago

Always interesting to see new MTAs! I'm not convinced that storing mails (including attachments) in MongoDB is a good solution. What's wrong with the filesystem? It's designed for that purpose. You can (and should) of course store metadata in a DB to make searching easier, but when it comes to the actual emails I think the filesystem is the best place. One less thing to break. Easier to backup, mangle etc.

mboehm 8y ago
Well, scaling a filesystem does not really sound easy to me. Scaling a dbms is rather common, I guess. I rather question, why do they store their binaries in mongodb? imho, s3 would be the perfect fit for that requirement.
nijave 8y ago
It's significantly easier to make an external data store resilient than to try to do that with a local filesystem. Backups, clustering, HA are all easier when there's a strong separation. In addition things like querying and lookups will be faster if you're utilizing a database
- Tepix 8y ago
  A filesystem doesn't have to be local. There are plenty of filesystems to choose from with different properties.
  - andris9 8y ago
    Using maildir (the current standard for mail storage) with any kind of remote file system is not the best environment to have. Maildir stores every email into a separate file and then uses rename() for every flag change. Marking 1000 messages as "Seen/Unseen" or adding a flag or moving to Trash means 1000 rename() calls in the file system. Multiply that with thousands of users also logged in to the system and doing their own things - you get many millions of inodes and rename()'s flying around all over. Good luck keeping a consistent copy or not crashing the mail server software as it waits for some queued rename() call over network FS to finish (and expects it to take nanoseconds as this is how long it takes in a local FS) while blocking everything else.
    - Tepix 8y ago
      I don't see how storing the mails in a database will improve the situation. You will still have to update 1000s of records when you mark 1000 messages as seen/unseen or move them into the trash. And the DB will then write the changes to a filesystem. One of Maildir's design goals was to make it safer to use on networked file systems such as NFS (compared to mbox).

by Tepix 8y ago

mboehm 8y ago

Well, scaling a filesystem does not really sound easy to me. Scaling a dbms is rather common, I guess. I rather question, why do they store their binaries in mongodb? imho, s3 would be the perfect fit for that requirement.

nijave 8y ago

It's significantly easier to make an external data store resilient than to try to do that with a local filesystem. Backups, clustering, HA are all easier when there's a strong separation. In addition things like querying and lookups will be faster if you're utilizing a database

Tepix 8y ago
A filesystem doesn't have to be local. There are plenty of filesystems to choose from with different properties.
- andris9 8y ago
  Using maildir (the current standard for mail storage) with any kind of remote file system is not the best environment to have. Maildir stores every email into a separate file and then uses rename() for every flag change. Marking 1000 messages as "Seen/Unseen" or adding a flag or moving to Trash means 1000 rename() calls in the file system. Multiply that with thousands of users also logged in to the system and doing their own things - you get many millions of inodes and rename()'s flying around all over. Good luck keeping a consistent copy or not crashing the mail server software as it waits for some queued rename() call over network FS to finish (and expects it to take nanoseconds as this is how long it takes in a local FS) while blocking everything else.
  - Tepix 8y ago
    I don't see how storing the mails in a database will improve the situation. You will still have to update 1000s of records when you mark 1000 messages as seen/unseen or move them into the trash. And the DB will then write the changes to a filesystem. One of Maildir's design goals was to make it safer to use on networked file systems such as NFS (compared to mbox).