8 ms·
"mbox" is a family of several mutually incompatible mailbox formats
- JonnieCache 15y agoInteresting URL there. A dot after the tld, and hosted on ntlworld, which was once the UK's most hated ISP (now owned by virgin and hated very slightly less.)
- tententen 15y agoA dot after the tld That's actually how it's supposed to be done: http://www.dns-sd.org/TrailingDotsInDomainNames.html http://www.dns-sd.org/TrailingDotsInDomainNames.html "It's a little-known fact, but fully-qualified (unambiguous) DNS domain names have a dot at the end. People running DNS servers usually know this (if you miss the trailing dots out, your DNS configuration is unlikely to work) but the general public usually doesn't. A domain name that doesn't have a dot at the end is not fully-qualified and is potentially ambiguous. This was documented in the DNS specification, RFC 1034, way back in 1987 ..." but the site works just fine without the trailing dot as well.
- djcapelis 15y ago> but the site works just fine without the trailing dot as well. In your current environment! And probably in all the environments connected to the Internet. But potentially without the dot it would be perfect valid for this to route elsewhere in someone's environment. :)
- julian37 15y agoThe dot after the tld is optional (if you don't need to specify a fully-qualified DNS name). Try for example: http://news.ycombinator.com./item?id=3438428 http://news.ycombinator.com./item?id=3438428 EDIT: I learned something new today, adding the trailing dot can be a good idea: http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/web-fully-qualified-domain-name.html http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/we...
- mike-cardwell 15y agohttps://news.ycombinator.com./ https://news.ycombinator.com./ Chrome doesn't accept the certificate when the trailing dot is in the request. I bet if you configured your site to redirect to a URL with a trailing dot, it would break a lot of badly written robots. I'd be interested to hear about how it affects the major search engines before I'd want to implement it on any important sites.
- slaven 15y agoAnother wrinkle: some mailers stuff full 8-bit chars into mbox which will choke some parsers and end parsing prematurely (most expect 7-bit ASCII). But it beats mdir handily in one way: if you're dealing with mailboxes with 100s of thousands of messages it is way more efficient.
- ZenPsycho 15y agoway more efficient along which dimension? Space, or processing speed? By roughly how much? Enough to be worth the costs in potential data loss?
- sliverstorm 15y agoEfficient use of inodes, perhaps?
- slaven 15y agoEach mdir email is a file, so depending on your OS it consumes disk fragments very quickly and relies on your OS file handlers to allocate and load messages. mbox as a mapped file and a good index blows it away in speed and space usage.
- __david__ 15y agoVery true, but reclaiming space in the middle of a huge mailbox is dramatically worse in the mbox case. Doing time-machine style rsync backups is also dramatically more space efficient in the maildir case.
- kijin 15y agoOne way to get around the different escaping rules between mboxo and mboxrd is to insert a space (0x20) at the beginning of any line that could be misinterpreted. This eliminates the need for any further escaping. Leading whitespace is almost always ignored by mail clients when displaying the message, especially when the text is flowed. I've received received e-mails that were formatted this way, though I'm not sure exactly which software, whether client or server, is responsible for doing it.
- dsr_ 15y agoNow you now have n+1 standards to keep track of. As a side effect, this breaks every reader which is unaware of it and handling binary content in mail (many, many messages) or languages or encodings which are space sensitive.
- kijin 15y agoBinary content and non-ASCII characters are almost always base64-encoded, and base64 never produces a From_ sequence (because of the space). Anyaway, I'm not endorsing the leading-space technique. I'm just observing that it's out there in the wild.
- riobard 15y agoA side question: what's the best way to build an email message store which stores millions of messages (and adding tens of thousands on average each day) on disk so that it can support: 1) Fast looking up messages sent/received by a particular email address, time, tag, priority, and a handful other properties and 2) Appending thousands of new messages per second at peak?
- beza1e1 15y agoYou could look at http://notmuchmail.org/ http://notmuchmail.org/ which builds on top of maildir.
- gwillen 15y agoIf you only ever want to append messages, and never want to mutate or remove them, I would do this scheme in two parts: 1) an append-only message store; 2) a set of indices stored alongside (1). To append thousands of messages per second, I would probably want a system with multiple writers appending to multiple files (mbox) or directories (maildir), and cycling/archiving the mailboxes as they get large. As for (2), I know a lot less about indexing messages than I do about storing them, but I would try out an off the shelf SQL or noSQL system and measure the performance, before trying to build anything fancy.
- JoshTriplett 15y agoMaildir (or a very similar one-message-per-file store) plus an index of some kind. If not for the continuous addition of messages, I'd suggest notmuch (http://notmuchmail.org/ http://notmuchmail.org/), which does a great job of indexing in any way you could want. Unfortunately, as far as I know notmuch does not have an "online" indexing mechanism; someone wrote a notmuch-deliver that adds a message and indexes it, but I doubt it would meet your performance requirements. You could probably make notmuch do what you want, but you might want a new indexing mechanism instead. I still think you want something very similar to Maildir underneath though.
- riobard 15y agoI was using the one-message-per-file approach, but keeping millions of small files in a filesystem is quite a challenge: I have to worry about available inodes, and listing/manipulating these files takes minutes!
- Karellen 15y agoSee also JWZ's article on the "Content-Length" header (mboxcl and mboxcl2 formats from the original article) http://www.jwz.org/doc/content-length.html http://www.jwz.org/doc/content-length.html