5 ms·

The sane old-school way to store mail is using directories and files. Mozilla Thunderbird does this and I've never had a corruption issue. If need be, you can i

by loopz 7y ago

The sane old-school way to store mail is using directories and files. Mozilla Thunderbird does this and I've never had a corruption issue. If need be, you can index or open individual files in text editor, as they're all plain text. Would hope for revitalization of the application, but works for my personal use cases. Yeah, files, are hard.

Moru 7y ago
Yes, just use thunderbird/netscape and be done with it. I still have my old installation running, upgrading and migrating as needed since netscape time. I have mails from 1999 on there still. At work they were using outlook for mail and wondering why it crashes now and then (past 2 GB it starts writing over the start of the file or some such). We get a lot of big documents on email so switched to thunderbird and all is well since. Had to switch mailserver though since that one also only handled 2 GB. No idea what they were running but most likely something MS-related. Most of their stuff was MS at the time.
q3k 7y ago
The Maildir format, however, barely uses any tricky filesystem features. In order, all the semantics you need to support are: - atomic file create with content - atomic directory create - atomic file move - atomic file delete - single-level file enumeration No file modification in place which is the actual hard path. Not even appending to existing files. Incidentally, reliable databases tend to also not rely on file modification, opting for a log-and-compact strategy instead. It's when you start modifying existing data that 'just use a filesystem' becomes unviable.
- blattimwind 7y ago
  > - atomic file create with content No FS guarantees this. You can emulate this by writing to a tmp location, syncing and performing a move.
  - gizmo686 7y ago
    When you use rename to overwrite a file, it guarentees that at any point in time, newpath will refer to either the original file, or the new file. This is atomic enough to give you effectivly atomic file creation with contents. It is not technically atomic, because there is a period where oldpath and newpath both refer to the same file, but I don't think that is an issue for this usecase.
  - manwe150 7y ago
    Linux has the O_TMPFILE flag to open, for which this is one of the use cases described in the man page. I can’t say about “guaranteed” though. I assume Microsoft Transactional NTFS (TxF) also had (or has) this capability—but deprecated.
    - blattimwind 7y ago
      TxF is a corner stone of Windows Update, I can't really imagine them removing it. I always assumed they marked it deprecated to avoid supporting it towards customers.
garaetjjte 7y ago
I don't think Thunderbird is good example here.. It stores inboxes in mbox-like format, then some metadata in very weird .msf format, and on top on that SQLite database for search indexing. And it has some bewildering bugs like mixing up titles and content from two different messages, or slapping on messages date pulled from thin air..
- cesarb 7y ago
  > or slapping on messages date pulled from thin air.. IME, this happens when the message doesn't have a date (that is, Date header missing or invalid). There's an open request on bugzilla to use the last Received date in these cases.
Avamander 7y ago
I wish there'd be a native e-mail client that utilized a RDBMS properly, I want a really fast full-text search, I want fast filters. The current ones have all been very unsatisfactory, mostly because the file-based approach doesn't allow them to. It really isn't sane in my opinion.
- ses1984 7y ago
  "fast full text search" is something better provided by software that specializes in fast full text search. Rdbms for the email use case is kind of orthogonal to fast full text search.
  - Avamander 7y ago
    A RDBMS can provide a fast full text search alongside other useful features, can you elaborate how it's orthogonal? Oh, and I also dislike even the idea that I have to find some "specialized software" to have basic functionality in my e-mail client. That's why most still suck a*.
    - ses1984 7y ago
      You probably want something that checks for similarly spelled words, maybe natural language processing, whole or partial word matching, indices on all this, etc. If all you want is to grep your email, that's different, I think you can accomplish that with pine or other text mode email clients.
    - gnaritas 7y ago
      > Oh, and I also dislike even the idea that I have to find some "specialized software" to have basic functionality in my e-mail client. No you don't, you're asking for exactly that by wanting an RDBMs for your email client. That's about as specialized as it gets.
- bronson 7y ago
  Have you tried notmuch? https://notmuchmail.org/ https://notmuchmail.org/ It uses a Xapian index, not an RDBMS, but it's impressive. RDBMSes aren't known for producing good natural language search results.

by loopz 7y ago

Moru 7y ago

Yes, just use thunderbird/netscape and be done with it. I still have my old installation running, upgrading and migrating as needed since netscape time. I have mails from 1999 on there still. At work they were using outlook for mail and wondering why it crashes now and then (past 2 GB it starts writing over the start of the file or some such). We get a lot of big documents on email so switched to thunderbird and all is well since. Had to switch mailserver though since that one also only handled 2 GB. No idea what they were running but most likely something MS-related. Most of their stuff was MS at the time.

q3k 7y ago

The Maildir format, however, barely uses any tricky filesystem features. In order, all the semantics you need to support are: - atomic file create with content - atomic directory create - atomic file move - atomic file delete - single-level file enumeration No file modification in place which is the actual hard path. Not even appending to existing files. Incidentally, reliable databases tend to also not rely on file modification, opting for a log-and-compact strategy instead. It's when you start modifying existing data that 'just use a filesystem' becomes unviable.

blattimwind 7y ago
> - atomic file create with content No FS guarantees this. You can emulate this by writing to a tmp location, syncing and performing a move.
- gizmo686 7y ago
  When you use rename to overwrite a file, it guarentees that at any point in time, newpath will refer to either the original file, or the new file. This is atomic enough to give you effectivly atomic file creation with contents. It is not technically atomic, because there is a period where oldpath and newpath both refer to the same file, but I don't think that is an issue for this usecase.
- manwe150 7y ago
  Linux has the O_TMPFILE flag to open, for which this is one of the use cases described in the man page. I can’t say about “guaranteed” though. I assume Microsoft Transactional NTFS (TxF) also had (or has) this capability—but deprecated.
  - blattimwind 7y ago
    TxF is a corner stone of Windows Update, I can't really imagine them removing it. I always assumed they marked it deprecated to avoid supporting it towards customers.

garaetjjte 7y ago

I don't think Thunderbird is good example here.. It stores inboxes in mbox-like format, then some metadata in very weird .msf format, and on top on that SQLite database for search indexing. And it has some bewildering bugs like mixing up titles and content from two different messages, or slapping on messages date pulled from thin air..

cesarb 7y ago
> or slapping on messages date pulled from thin air.. IME, this happens when the message doesn't have a date (that is, Date header missing or invalid). There's an open request on bugzilla to use the last Received date in these cases.

Avamander 7y ago

I wish there'd be a native e-mail client that utilized a RDBMS properly, I want a really fast full-text search, I want fast filters. The current ones have all been very unsatisfactory, mostly because the file-based approach doesn't allow them to. It really isn't sane in my opinion.

ses1984 7y ago
"fast full text search" is something better provided by software that specializes in fast full text search. Rdbms for the email use case is kind of orthogonal to fast full text search.
- Avamander 7y ago
  A RDBMS can provide a fast full text search alongside other useful features, can you elaborate how it's orthogonal? Oh, and I also dislike even the idea that I have to find some "specialized software" to have basic functionality in my e-mail client. That's why most still suck a*.
  - ses1984 7y ago
    You probably want something that checks for similarly spelled words, maybe natural language processing, whole or partial word matching, indices on all this, etc. If all you want is to grep your email, that's different, I think you can accomplish that with pine or other text mode email clients.
  - gnaritas 7y ago
    > Oh, and I also dislike even the idea that I have to find some "specialized software" to have basic functionality in my e-mail client. No you don't, you're asking for exactly that by wanting an RDBMs for your email client. That's about as specialized as it gets.
bronson 7y ago
Have you tried notmuch? https://notmuchmail.org/ https://notmuchmail.org/ It uses a Xapian index, not an RDBMS, but it's impressive. RDBMSes aren't known for producing good natural language search results.