3 ms·

Isn't Maildir too wasteful?

by bzb3 6y ago

effie 6y ago
In what way?
- bzb3 6y ago
  Each email in its own file means a minimum of x KB on disk for each email, where x is usually 4. For people with thousands of emails that's very wasteful.
  - Silhouette 6y ago
    I would take a safe Maildir implementation over subtle (or sometimes catastrophic) data loss bugs any day. The one really serious problem I have with Thunderbird is that after being stung more than once, I feel like not only do I have to keep normal backups, I also have to retain old backups indefinitely, just in case a folder was corrupted silently a long time ago and I only find out months or years later when I need to search older messages.
  - deleted 6y ago
    [deleted]
  - ethelward 6y ago
    That's ~4MB per thousand emails; while I agree it's conceptually wasteful, I doubt it would have any real consequences for anyone.
    - jcranmer 6y ago
      Waste isn't the issue with big folders; filesystem performance (especially NTFS, which is to say Windows users, which is to say ~90% of users) degrades pretty horribly when folders have tens of thousands of files. Even on Linux filesystems, having a folder with a 100,000 files in it was having issues.
      - kevincox 6y ago
        FWIW on ext4 I benchmarked adding millions of files to a folder and compared with `xx/...` and `x/x/x/x/...` and found the performance to be basically identical between the options and didn't degrade with the file count. The one thing that I did find is that deleting folders with many files was very slow. I guess that path wasn't well optimized. However even then you could deleted hundreds of files a second so for regular mail usage it likely isn't a major issue.
  - vetinari 6y ago
    Only if you use filesystem that cannot efficiently store small files, either together with metadata or packing several small files together, and allocates a full allocation unit for each separately.
  - effie 6y ago
    Maildir, as opposed to emails stored in one big file, does cause some disk space to get wasted, but in past 10 years, it is immaterial. Even million empty emails, each taking 4kB of disk space, would take only 4GB in total. The small files aren't a concern for saving disk space: the big ones are. Maildir has the advantage that searching, adding and deleting emails is very fast, much faster than if those operations were made on single big file. Worth it.
flas9sd 6y ago
for personal mail the advantage in Maildir is incremental backups acting on the file system level are efficient. With an mbox file or sqlite db the whole file has to be backuped and transfered again. Of course for backup/snapshot mechanisms working on the block level one big file does not matter.

Isn't Maildir too wasteful?

by bzb3 6y ago

Isn't Maildir too wasteful?

effie 6y ago

In what way?

bzb3 6y ago
Each email in its own file means a minimum of x KB on disk for each email, where x is usually 4. For people with thousands of emails that's very wasteful.
- Silhouette 6y ago
  I would take a safe Maildir implementation over subtle (or sometimes catastrophic) data loss bugs any day. The one really serious problem I have with Thunderbird is that after being stung more than once, I feel like not only do I have to keep normal backups, I also have to retain old backups indefinitely, just in case a folder was corrupted silently a long time ago and I only find out months or years later when I need to search older messages.
- deleted 6y ago
  [deleted]
- ethelward 6y ago
  That's ~4MB per thousand emails; while I agree it's conceptually wasteful, I doubt it would have any real consequences for anyone.
  - jcranmer 6y ago
    Waste isn't the issue with big folders; filesystem performance (especially NTFS, which is to say Windows users, which is to say ~90% of users) degrades pretty horribly when folders have tens of thousands of files. Even on Linux filesystems, having a folder with a 100,000 files in it was having issues.
    - kevincox 6y ago
      FWIW on ext4 I benchmarked adding millions of files to a folder and compared with `xx/...` and `x/x/x/x/...` and found the performance to be basically identical between the options and didn't degrade with the file count. The one thing that I did find is that deleting folders with many files was very slow. I guess that path wasn't well optimized. However even then you could deleted hundreds of files a second so for regular mail usage it likely isn't a major issue.
- vetinari 6y ago
  Only if you use filesystem that cannot efficiently store small files, either together with metadata or packing several small files together, and allocates a full allocation unit for each separately.
- effie 6y ago
  Maildir, as opposed to emails stored in one big file, does cause some disk space to get wasted, but in past 10 years, it is immaterial. Even million empty emails, each taking 4kB of disk space, would take only 4GB in total. The small files aren't a concern for saving disk space: the big ones are. Maildir has the advantage that searching, adding and deleting emails is very fast, much faster than if those operations were made on single big file. Worth it.

flas9sd 6y ago

for personal mail the advantage in Maildir is incremental backups acting on the file system level are efficient. With an mbox file or sqlite db the whole file has to be backuped and transfered again. Of course for backup/snapshot mechanisms working on the block level one big file does not matter.