8 ms·

Email represents three important abstractions of the real life: First, it's a task queue. And you're the worker that processes the tasks. Anything that helps a

Email represents three important abstractions of the real life: First, it's a task queue. And you're the worker that processes the tasks. Anything that helps an email service to be that abstraction is very good for users -- prioritizing, postponing, auto-processing, branching, delegating, etc. Second, it's a log. As the article rightfully says -- an immutable one. Again, anything that helps email be a good log is very good for the user -- search, filtering, archiving, etc. Third, it's a file storage. This is the most overlooked part, unfortunately. Search, sharing, organizing into collections -- all that would be of great help to users. Yet, Google pushes that silly AMP :\

brongondwana 9y ago
Bah - file storage. Yep, it certainly is. And to a large degree, because it's a log. Once you email something as an attachment (even to yourself) it's like a version control commit. That copy is locked in, and won't change. You can find it and refer back to it later. It has a timestamp on it, and it's probably stored in the cloud somewhere with backups and such. Hence "memory" - because that's how people use it :)
- dgudkov 9y ago
  >And to a large degree, because it's a log. I agree. Although, it's more than just a log. For instance, it would be great to have the ability to see a collection of all files sent by a particular sender, with file versions (with dates and links to according emails) stacked under one file name. Another suggestion -- view a collection of files in all emails with particular label. Finally, in some cases it can be helpful to have a local read-only copy of files in a particular collection. It's like one-way read-only Dropbox. This would make automated processing of email attachments so much simpler. (BTW, a product idea, anyone?!).
  - brongondwana 9y ago
    So one of the smartass login messages in our internal slack is "JMAP will fix it". Interestingly enough, JMAP will make this kind of thing much easier - and we have thought about it in the design! Our Cyrus server already internally keeps a blobId which is a digest of the binary content of each MIME part. Which means you can easily tell that two attachments are identical without reading the contents or guessing based on size and name. And it's easy to search for messages with file attachments and fetch just the file attachement details via JMAP, making this kind of interface relatively easy to write. The usual caveats about "we haven't got a particular timeframe in mind" apply - but once JMAP is standard, anyone can write a client to that standard!
    - pdfernhout 9y ago
      Hi Bron. JMAP is a great design for what it is (a much better IMAP). Thanks for creating it. I can see JMAP becoming a great standard for email-oriented client/server communications. That said, I'd still encourage you to think bigger to make a more general system. Of course, the risk of thinking bigger is you may end up with either nothing or a monstrosity no one wants to use. And also you'd likely have to adapt your business model eventually if the bigger picture changes through your actions. So, it's fair to think smaller too. From a related post I made to the Thunderbird Planning list in April 2017 on what I see as the bigger picture: https://mail.mozilla.org/pipermail/tb-planning/2017-April/005533.html https://mail.mozilla.org/pipermail/tb-planning/2017-April/00... "My perspective is from wanting to create tools to support a "social semantic desktop" of which email is a subset of the items. Other types of items include news messages, RSS feed messages, chat messages, notification messages, IoT messages, blog posts, pingbacks, saved web pages, commands constructing complex shared documents, VR world update messages, code snippets, notes, and more. Email is a special case of a much larger world of messages. So, for me, a good assumption is that people will eventually have billions of message items comprising terabytes of data in their local (shared-with-email) store. Eventually each user would be part of a global federation of users that together have billions of times that number of messages as a federated semantic web. I understand that such a vision of messaging on that scale is not yet the goal of most people in this discussion right now. But I'd still urge people to think a little beyond redoing Thunderbird as-it-is and try to imagine a personal messaging system that processes, archives, indexes, and recalls vast numbers of items every day in interaction with a user and automated systems as part of a larger federation of similar systems using shared semantic standards." I've been beating my head against that wall for years with very limited progress (as have others) -- so I can't say it isn't a high risk endeavor with a low probability of success. But the benefit for humanity could potentially be huge if such an effort succeeded. Maybe it might just take a few tweaks to JMAP to generalize it enough to realize some of that big picture?
      - brongondwana 9y ago
        JMAP is pretty generalizable to other datatypes! It's a large part of why we split it into Core and Mail. We also have Calendar and Contacts modules in a reasonable state already. We also use JMAP-style APIs for Files, for Notes, and for all the extra Preferences and group management stuff in both FastMail and Topicbox.
- lloeki 9y ago
  > it's like a version control commit. As an alternative to mbox/maildir/whatevs I've been toying with a prototype of email storage that works in a content-addressed way with objects very much like how git operates internally. > Our Cyrus server already internally keeps a blobId which is a digest of the binary content of each MIME part. Which means you can easily tell that two attachments are identical without reading the contents or guessing based on size and name. This deduplication property then comes naturally from the above design, and not just for attachments but also for email headers and bodies. Many other things come easily such as rsync "just working" for fetching, or making use of git packs[0] [0]: https://git-scm.com/docs/pack-format https://git-scm.com/docs/pack-format
  - brongondwana 9y ago
    You're not the first person to think of using git-like storage for email, and you won't be the last! Sadly deduplication isn't quite that easy - while the binary formats might be the same, MIME is very un-roundtrip-friendly when you start from the MIME structure. There are many ways to encode the same bytes, so if you need to reconstruct the raw bytes (FETCH RFC822) then you need a way to recover those original bytes. I've started work on that a few times, but it's never been worth the effort to make it stable.
igammarays 9y ago
You nailed it. And if AMP jeopardizes this core idea of "email as a reliable log", which is how many people use it, it will slowly eat away at the usefulness of the service itself. People will stop trusting it and using it as such.
DCF 9y ago
>> Third, it's a file storage. This is the most overlooked part, unfortunately. When gmail first launched I remember it having such relatively high storage space that my friends and I used it extensively as a music sharing/storage system. An mp3 was just small enough to make it a comfortable email attachment and the storage space was great in an age without infinite cloud hosts. Huge amounts of gmail's early utility to me was simply the file storage. Lots of people would give you a free email address. Google made a big deal out of the massive space they offered.
- phireal 9y ago
  Someone even went so far as to build a FUSE filesystem: https://en.wikipedia.org/wiki/GmailFS https://en.wikipedia.org/wiki/GmailFS
- dkurth 9y ago
  A while back, Gmail began blocking .js file attachments, but I didn't realize until today that this means my old JavaScript file attachments would be retroactively blocked. I tracked down an email from two years ago with a JavaScript file attached that I needed, but I couldn't download it! I was eventually able to get it by doing "Show Original" in Gmail and copy/pasting the base64-encoded attachment data and decoding it. But it made me nervous about using Gmail as a way to hang on to files, since things I attach now might not be available later.
  - ecshafer 9y ago
    I had this same issue. Google also blocked .jar and .zip if I remember. I had lost some old school projects that I tried to recover, which we're turned in via email. I couldn't download the attachments for this, despite the fact I sent them in the first place! Much less, why couldn't I allow someone to send me a .js file I wanted it? It's a weird restriction to not even be given a choice.
erikb 9y ago
> Search, sharing, organizing into collections All these features does gmail already have for years. And yes, people also use gmail for file storage. AMP is a stupid idea, but it doesn't mean they do it instead of other features. The other features are there already.
tslafan 9y ago
The great thing is that email is so flexible it can be many things to different people. It may not do any of those things really well but it allows the user the freedom to use it in creative ways.

brongondwana 9y ago

Bah - file storage. Yep, it certainly is. And to a large degree, because it's a log. Once you email something as an attachment (even to yourself) it's like a version control commit. That copy is locked in, and won't change. You can find it and refer back to it later. It has a timestamp on it, and it's probably stored in the cloud somewhere with backups and such. Hence "memory" - because that's how people use it :)

dgudkov 9y ago
>And to a large degree, because it's a log. I agree. Although, it's more than just a log. For instance, it would be great to have the ability to see a collection of all files sent by a particular sender, with file versions (with dates and links to according emails) stacked under one file name. Another suggestion -- view a collection of files in all emails with particular label. Finally, in some cases it can be helpful to have a local read-only copy of files in a particular collection. It's like one-way read-only Dropbox. This would make automated processing of email attachments so much simpler. (BTW, a product idea, anyone?!).
- brongondwana 9y ago
  So one of the smartass login messages in our internal slack is "JMAP will fix it". Interestingly enough, JMAP will make this kind of thing much easier - and we have thought about it in the design! Our Cyrus server already internally keeps a blobId which is a digest of the binary content of each MIME part. Which means you can easily tell that two attachments are identical without reading the contents or guessing based on size and name. And it's easy to search for messages with file attachments and fetch just the file attachement details via JMAP, making this kind of interface relatively easy to write. The usual caveats about "we haven't got a particular timeframe in mind" apply - but once JMAP is standard, anyone can write a client to that standard!
  - pdfernhout 9y ago
    Hi Bron. JMAP is a great design for what it is (a much better IMAP). Thanks for creating it. I can see JMAP becoming a great standard for email-oriented client/server communications. That said, I'd still encourage you to think bigger to make a more general system. Of course, the risk of thinking bigger is you may end up with either nothing or a monstrosity no one wants to use. And also you'd likely have to adapt your business model eventually if the bigger picture changes through your actions. So, it's fair to think smaller too. From a related post I made to the Thunderbird Planning list in April 2017 on what I see as the bigger picture: https://mail.mozilla.org/pipermail/tb-planning/2017-April/005533.html https://mail.mozilla.org/pipermail/tb-planning/2017-April/00... "My perspective is from wanting to create tools to support a "social semantic desktop" of which email is a subset of the items. Other types of items include news messages, RSS feed messages, chat messages, notification messages, IoT messages, blog posts, pingbacks, saved web pages, commands constructing complex shared documents, VR world update messages, code snippets, notes, and more. Email is a special case of a much larger world of messages. So, for me, a good assumption is that people will eventually have billions of message items comprising terabytes of data in their local (shared-with-email) store. Eventually each user would be part of a global federation of users that together have billions of times that number of messages as a federated semantic web. I understand that such a vision of messaging on that scale is not yet the goal of most people in this discussion right now. But I'd still urge people to think a little beyond redoing Thunderbird as-it-is and try to imagine a personal messaging system that processes, archives, indexes, and recalls vast numbers of items every day in interaction with a user and automated systems as part of a larger federation of similar systems using shared semantic standards." I've been beating my head against that wall for years with very limited progress (as have others) -- so I can't say it isn't a high risk endeavor with a low probability of success. But the benefit for humanity could potentially be huge if such an effort succeeded. Maybe it might just take a few tweaks to JMAP to generalize it enough to realize some of that big picture?
    - brongondwana 9y ago
      JMAP is pretty generalizable to other datatypes! It's a large part of why we split it into Core and Mail. We also have Calendar and Contacts modules in a reasonable state already. We also use JMAP-style APIs for Files, for Notes, and for all the extra Preferences and group management stuff in both FastMail and Topicbox.
lloeki 9y ago
> it's like a version control commit. As an alternative to mbox/maildir/whatevs I've been toying with a prototype of email storage that works in a content-addressed way with objects very much like how git operates internally. > Our Cyrus server already internally keeps a blobId which is a digest of the binary content of each MIME part. Which means you can easily tell that two attachments are identical without reading the contents or guessing based on size and name. This deduplication property then comes naturally from the above design, and not just for attachments but also for email headers and bodies. Many other things come easily such as rsync "just working" for fetching, or making use of git packs[0] [0]: https://git-scm.com/docs/pack-format https://git-scm.com/docs/pack-format
- brongondwana 9y ago
  You're not the first person to think of using git-like storage for email, and you won't be the last! Sadly deduplication isn't quite that easy - while the binary formats might be the same, MIME is very un-roundtrip-friendly when you start from the MIME structure. There are many ways to encode the same bytes, so if you need to reconstruct the raw bytes (FETCH RFC822) then you need a way to recover those original bytes. I've started work on that a few times, but it's never been worth the effort to make it stable.

igammarays 9y ago

You nailed it. And if AMP jeopardizes this core idea of "email as a reliable log", which is how many people use it, it will slowly eat away at the usefulness of the service itself. People will stop trusting it and using it as such.

DCF 9y ago

>> Third, it's a file storage. This is the most overlooked part, unfortunately. When gmail first launched I remember it having such relatively high storage space that my friends and I used it extensively as a music sharing/storage system. An mp3 was just small enough to make it a comfortable email attachment and the storage space was great in an age without infinite cloud hosts. Huge amounts of gmail's early utility to me was simply the file storage. Lots of people would give you a free email address. Google made a big deal out of the massive space they offered.

phireal 9y ago
Someone even went so far as to build a FUSE filesystem: https://en.wikipedia.org/wiki/GmailFS https://en.wikipedia.org/wiki/GmailFS
dkurth 9y ago
A while back, Gmail began blocking .js file attachments, but I didn't realize until today that this means my old JavaScript file attachments would be retroactively blocked. I tracked down an email from two years ago with a JavaScript file attached that I needed, but I couldn't download it! I was eventually able to get it by doing "Show Original" in Gmail and copy/pasting the base64-encoded attachment data and decoding it. But it made me nervous about using Gmail as a way to hang on to files, since things I attach now might not be available later.
- ecshafer 9y ago
  I had this same issue. Google also blocked .jar and .zip if I remember. I had lost some old school projects that I tried to recover, which we're turned in via email. I couldn't download the attachments for this, despite the fact I sent them in the first place! Much less, why couldn't I allow someone to send me a .js file I wanted it? It's a weird restriction to not even be given a choice.

erikb 9y ago

> Search, sharing, organizing into collections All these features does gmail already have for years. And yes, people also use gmail for file storage. AMP is a stupid idea, but it doesn't mean they do it instead of other features. The other features are there already.

tslafan 9y ago

The great thing is that email is so flexible it can be many things to different people. It may not do any of those things really well but it allows the user the freedom to use it in creative ways.