4 ms·

So one of the smartass login messages in our internal slack is "JMAP will fix it". Interestingly enough, JMAP will make this kind of thing much easier - and we

So one of the smartass login messages in our internal slack is "JMAP will fix it". Interestingly enough, JMAP will make this kind of thing much easier - and we have thought about it in the design! Our Cyrus server already internally keeps a blobId which is a digest of the binary content of each MIME part. Which means you can easily tell that two attachments are identical without reading the contents or guessing based on size and name. And it's easy to search for messages with file attachments and fetch just the file attachement details via JMAP, making this kind of interface relatively easy to write. The usual caveats about "we haven't got a particular timeframe in mind" apply - but once JMAP is standard, anyone can write a client to that standard!

pdfernhout 9y ago
Hi Bron. JMAP is a great design for what it is (a much better IMAP). Thanks for creating it. I can see JMAP becoming a great standard for email-oriented client/server communications. That said, I'd still encourage you to think bigger to make a more general system. Of course, the risk of thinking bigger is you may end up with either nothing or a monstrosity no one wants to use. And also you'd likely have to adapt your business model eventually if the bigger picture changes through your actions. So, it's fair to think smaller too. From a related post I made to the Thunderbird Planning list in April 2017 on what I see as the bigger picture: https://mail.mozilla.org/pipermail/tb-planning/2017-April/005533.html https://mail.mozilla.org/pipermail/tb-planning/2017-April/00... "My perspective is from wanting to create tools to support a "social semantic desktop" of which email is a subset of the items. Other types of items include news messages, RSS feed messages, chat messages, notification messages, IoT messages, blog posts, pingbacks, saved web pages, commands constructing complex shared documents, VR world update messages, code snippets, notes, and more. Email is a special case of a much larger world of messages. So, for me, a good assumption is that people will eventually have billions of message items comprising terabytes of data in their local (shared-with-email) store. Eventually each user would be part of a global federation of users that together have billions of times that number of messages as a federated semantic web. I understand that such a vision of messaging on that scale is not yet the goal of most people in this discussion right now. But I'd still urge people to think a little beyond redoing Thunderbird as-it-is and try to imagine a personal messaging system that processes, archives, indexes, and recalls vast numbers of items every day in interaction with a user and automated systems as part of a larger federation of similar systems using shared semantic standards." I've been beating my head against that wall for years with very limited progress (as have others) -- so I can't say it isn't a high risk endeavor with a low probability of success. But the benefit for humanity could potentially be huge if such an effort succeeded. Maybe it might just take a few tweaks to JMAP to generalize it enough to realize some of that big picture?
- brongondwana 9y ago
  JMAP is pretty generalizable to other datatypes! It's a large part of why we split it into Core and Mail. We also have Calendar and Contacts modules in a reasonable state already. We also use JMAP-style APIs for Files, for Notes, and for all the extra Preferences and group management stuff in both FastMail and Topicbox.
  - pdfernhout 9y ago
    Neat! Yes, I see the potential there, which is why I an interested in JMAP. Just wondering how it might could get even better at all that. As related market research, Kolab uses IMAP as a general-purpose data store: https://github.com/pdfernhout/Twirlip2/blob/master/design/market-research.md https://github.com/pdfernhout/Twirlip2/blob/master/design/ma...
- carussell 9y ago
  Sounds like jwz's "intertwingle"[1] strawman with a smidge of "unity of interface"[2], or JetBrains's old Omea information manager[3]. 1. https://www-archive.mozilla.org/blue-sky/misc/199805/intertwingle.html https://www-archive.mozilla.org/blue-sky/misc/199805/intertw... 2. https://www-archive.mozilla.org/unity-of-interface.html https://www-archive.mozilla.org/unity-of-interface.html 3. https://www.jetbrains.com/omea/features/ https://www.jetbrains.com/omea/features/
  - pdfernhout 9y ago
    Thanks for those links, especially the first to Jamie Zawinski's 1998 post on "vast volumes of email" and interwingled data. Yes, that is the sort of concepts and implementation I am talking about (and had hoped perhaps to move Thunderbird towards). Though I might prefer a different UI than JetBrain's Omea -- as well as a FOSS one. In one sketch of some ideas, I was thinking of that all more as a set of plugins: https://github.com/pdfernhout/Twirlip2 https://github.com/pdfernhout/Twirlip2 Jamie is obviously a brilliant and humane guy -- and his Lisp roots really show in his design of general purpose systems. Looking at Jamie's website, I was hanging around the CMU Robotics Labs around 1986 when he was too so I wonder if we may have ever talked in passing then? LISP as a programming mindset encourages such generalized designs about information handling, transformation, and linking -- as does inspirations from early AI research (thinking about thinking). For example there is the book Representation and Meaning by Herbert Simon and Laurent Siklossy from 1972 -- as one of many early AI books. I have that one because Professor Simon was nice enough to give me a copy back then during a few conversations we had in his office. Before that, in high school circa 1980 I did an independent study project using Patrick Winston's book on AI. William Kent's 1978 book "Data and Reality" (the first edition -- the third edition was dumbed down by someone else) is another great inspiration. There is a sort of cross over between AI-inspired design and human-centric "augmenting" design. An interesting comment on the AI/IA crossover by Brad Neuberg (who worked on Open HyperScope): http://codinginparadise.org/ebooks/html/blog/ia_vs__ai.html http://codinginparadise.org/ebooks/html/blog/ia_vs__ai.html "I‘ve traditionally been in the IA camp my whole career; I’ve spent the last year going deep into the AI camp. Traditionally these have been opposite worlds but I’m wondering if there might be some powerful, interesting ways to combine them together. One is humanistic and and design focused, the other tends to be more analytical; the space between them is where I’m interested in exploring in the future." Back around 1988 I applied to be in the NeXT developer program with a concept for an application (embarrassingly called "Orgi" back then, but I was young and reckless/foolish) where the idea was that any data item could hook up with any other data item (using triples). That application never got acted on until I met Steve Jobs when he gave a talk at Princeton and asked him personally about the developer program. And such ideas came out of my earlier interest in triplestores -- ideas that contributed to the roots of WordNet by my undergraduate advisor at Princeton, George A. Miller started just as I was graduating. In any case, as I said, "I've been beating my head against that wall for years with very limited progress (as have others)". :-) Jamie was obviously smart enough to connect with such ideas and move on to a happier life selling beer, pizza, and conversational ambiance. Obviously a much smarter guy than I am. :-) And I did not realize his early interest before, so thanks. I knew of another person who became much happier leaving contentious academia after his PhD and being a professor to run a hardware store. See also Matthew B. Crawford's "The Case for Working with Your Hands" written by someone with a PhD who left the non-profit public policy sector to do motorcycle repair: http://www.nytimes.com/2009/05/24/magazine/24labor-t.html http://www.nytimes.com/2009/05/24/magazine/24labor-t.html That said, maybe if Jamie had run Mozilla we would not have seen such a tragic waste of billions of non-profit dollars mostly pissed away -- including by mostly abandoning Thunderbird? Or, maybe his voice would have just been ignored? Hard to day. Innovation is such a complex thing. Some ideas I've collected for supporting innovation at such places: https://github.com/pdfernhout/High-Performance-Organizations-Reading-List https://github.com/pdfernhout/High-Performance-Organizations... Doug Engelbart and many people he inspired are others near the root of this "intertwingled" data ideas (including "Open HyperScope" as a successor to "Augment" http://hyperscope.org/ http://hyperscope.org/ ). Ted Nelson and people he inspired with Xanadu. Alan Kay and Dan Ingalls with Smalltalk and its implications as a thinking tool (Dan ran a family hotel for years afterwards). Ward Cunningham with the Wiki server idea and now the Smallest Federated Wiki idea. Mitch Kapor and the Chandler Project. The recent Dat Project by Max Ogden and others also connects with all this somewhat. There is this great book called "Design and Memory" from 1980 by Peter Huyck and Nellie Kremenak exploring such ideas. And of course "As We may Think" by Vannevar Bush. But the roots go further back though annotated scrolls (and the Talmud) and millennia of libraries -- and even back to the first "Standing Bear" cave paintings (see the book, "The Walking People"). This issue of (re)creating a decentralized alternative based around an email-ish feeling of interlinked locally-searchable information is increasingly important because it potentially is a way to move beyond negative aspects of centralized (anti-)social media. But, hey, my wife and I have more than 5000 books in our house, so all this might just be my hoarding tendencies trying to justify themselves? :-) Well, off to an unrelated job. And it is unrelated precisely because most of the big paying players in this space (e.g. Google and Facebook) want to get between people and their data -- and typical startups make deals with investors which limit their ability to do FOSS. Still, I can hope there will eventually be, say, an "Automattic" of this "intertwingling" equivalent of WordPress. I actually had hoped Automattic might itself become such a company, but they sadly seem to now see themselves just as a Wix/Squarespace competitor (e.g. adopting Slack for internal communications -- and not even Mattermost or Matrix.org if they want to let someone else own that space and cooperate with another FOSS system). At least it is nice to think about these things in my spare time -- although sadly at the cost of other experiences. Maybe it is all just a mistake to use my non-job time like that, like Piotr Solnica suggests: https://twitter.com/_solnic_/status/953578620617920513 https://twitter.com/_solnic_/status/953578620617920513

pdfernhout 9y ago

Hi Bron. JMAP is a great design for what it is (a much better IMAP). Thanks for creating it. I can see JMAP becoming a great standard for email-oriented client/server communications. That said, I'd still encourage you to think bigger to make a more general system. Of course, the risk of thinking bigger is you may end up with either nothing or a monstrosity no one wants to use. And also you'd likely have to adapt your business model eventually if the bigger picture changes through your actions. So, it's fair to think smaller too. From a related post I made to the Thunderbird Planning list in April 2017 on what I see as the bigger picture: https://mail.mozilla.org/pipermail/tb-planning/2017-April/005533.html https://mail.mozilla.org/pipermail/tb-planning/2017-April/00... "My perspective is from wanting to create tools to support a "social semantic desktop" of which email is a subset of the items. Other types of items include news messages, RSS feed messages, chat messages, notification messages, IoT messages, blog posts, pingbacks, saved web pages, commands constructing complex shared documents, VR world update messages, code snippets, notes, and more. Email is a special case of a much larger world of messages. So, for me, a good assumption is that people will eventually have billions of message items comprising terabytes of data in their local (shared-with-email) store. Eventually each user would be part of a global federation of users that together have billions of times that number of messages as a federated semantic web. I understand that such a vision of messaging on that scale is not yet the goal of most people in this discussion right now. But I'd still urge people to think a little beyond redoing Thunderbird as-it-is and try to imagine a personal messaging system that processes, archives, indexes, and recalls vast numbers of items every day in interaction with a user and automated systems as part of a larger federation of similar systems using shared semantic standards." I've been beating my head against that wall for years with very limited progress (as have others) -- so I can't say it isn't a high risk endeavor with a low probability of success. But the benefit for humanity could potentially be huge if such an effort succeeded. Maybe it might just take a few tweaks to JMAP to generalize it enough to realize some of that big picture?

brongondwana 9y ago
JMAP is pretty generalizable to other datatypes! It's a large part of why we split it into Core and Mail. We also have Calendar and Contacts modules in a reasonable state already. We also use JMAP-style APIs for Files, for Notes, and for all the extra Preferences and group management stuff in both FastMail and Topicbox.
- pdfernhout 9y ago
  Neat! Yes, I see the potential there, which is why I an interested in JMAP. Just wondering how it might could get even better at all that. As related market research, Kolab uses IMAP as a general-purpose data store: https://github.com/pdfernhout/Twirlip2/blob/master/design/market-research.md https://github.com/pdfernhout/Twirlip2/blob/master/design/ma...
carussell 9y ago
Sounds like jwz's "intertwingle"[1] strawman with a smidge of "unity of interface"[2], or JetBrains's old Omea information manager[3]. 1. https://www-archive.mozilla.org/blue-sky/misc/199805/intertwingle.html https://www-archive.mozilla.org/blue-sky/misc/199805/intertw... 2. https://www-archive.mozilla.org/unity-of-interface.html https://www-archive.mozilla.org/unity-of-interface.html 3. https://www.jetbrains.com/omea/features/ https://www.jetbrains.com/omea/features/
- pdfernhout 9y ago
  Thanks for those links, especially the first to Jamie Zawinski's 1998 post on "vast volumes of email" and interwingled data. Yes, that is the sort of concepts and implementation I am talking about (and had hoped perhaps to move Thunderbird towards). Though I might prefer a different UI than JetBrain's Omea -- as well as a FOSS one. In one sketch of some ideas, I was thinking of that all more as a set of plugins: https://github.com/pdfernhout/Twirlip2 https://github.com/pdfernhout/Twirlip2 Jamie is obviously a brilliant and humane guy -- and his Lisp roots really show in his design of general purpose systems. Looking at Jamie's website, I was hanging around the CMU Robotics Labs around 1986 when he was too so I wonder if we may have ever talked in passing then? LISP as a programming mindset encourages such generalized designs about information handling, transformation, and linking -- as does inspirations from early AI research (thinking about thinking). For example there is the book Representation and Meaning by Herbert Simon and Laurent Siklossy from 1972 -- as one of many early AI books. I have that one because Professor Simon was nice enough to give me a copy back then during a few conversations we had in his office. Before that, in high school circa 1980 I did an independent study project using Patrick Winston's book on AI. William Kent's 1978 book "Data and Reality" (the first edition -- the third edition was dumbed down by someone else) is another great inspiration. There is a sort of cross over between AI-inspired design and human-centric "augmenting" design. An interesting comment on the AI/IA crossover by Brad Neuberg (who worked on Open HyperScope): http://codinginparadise.org/ebooks/html/blog/ia_vs__ai.html http://codinginparadise.org/ebooks/html/blog/ia_vs__ai.html "I‘ve traditionally been in the IA camp my whole career; I’ve spent the last year going deep into the AI camp. Traditionally these have been opposite worlds but I’m wondering if there might be some powerful, interesting ways to combine them together. One is humanistic and and design focused, the other tends to be more analytical; the space between them is where I’m interested in exploring in the future." Back around 1988 I applied to be in the NeXT developer program with a concept for an application (embarrassingly called "Orgi" back then, but I was young and reckless/foolish) where the idea was that any data item could hook up with any other data item (using triples). That application never got acted on until I met Steve Jobs when he gave a talk at Princeton and asked him personally about the developer program. And such ideas came out of my earlier interest in triplestores -- ideas that contributed to the roots of WordNet by my undergraduate advisor at Princeton, George A. Miller started just as I was graduating. In any case, as I said, "I've been beating my head against that wall for years with very limited progress (as have others)". :-) Jamie was obviously smart enough to connect with such ideas and move on to a happier life selling beer, pizza, and conversational ambiance. Obviously a much smarter guy than I am. :-) And I did not realize his early interest before, so thanks. I knew of another person who became much happier leaving contentious academia after his PhD and being a professor to run a hardware store. See also Matthew B. Crawford's "The Case for Working with Your Hands" written by someone with a PhD who left the non-profit public policy sector to do motorcycle repair: http://www.nytimes.com/2009/05/24/magazine/24labor-t.html http://www.nytimes.com/2009/05/24/magazine/24labor-t.html That said, maybe if Jamie had run Mozilla we would not have seen such a tragic waste of billions of non-profit dollars mostly pissed away -- including by mostly abandoning Thunderbird? Or, maybe his voice would have just been ignored? Hard to day. Innovation is such a complex thing. Some ideas I've collected for supporting innovation at such places: https://github.com/pdfernhout/High-Performance-Organizations-Reading-List https://github.com/pdfernhout/High-Performance-Organizations... Doug Engelbart and many people he inspired are others near the root of this "intertwingled" data ideas (including "Open HyperScope" as a successor to "Augment" http://hyperscope.org/ http://hyperscope.org/ ). Ted Nelson and people he inspired with Xanadu. Alan Kay and Dan Ingalls with Smalltalk and its implications as a thinking tool (Dan ran a family hotel for years afterwards). Ward Cunningham with the Wiki server idea and now the Smallest Federated Wiki idea. Mitch Kapor and the Chandler Project. The recent Dat Project by Max Ogden and others also connects with all this somewhat. There is this great book called "Design and Memory" from 1980 by Peter Huyck and Nellie Kremenak exploring such ideas. And of course "As We may Think" by Vannevar Bush. But the roots go further back though annotated scrolls (and the Talmud) and millennia of libraries -- and even back to the first "Standing Bear" cave paintings (see the book, "The Walking People"). This issue of (re)creating a decentralized alternative based around an email-ish feeling of interlinked locally-searchable information is increasingly important because it potentially is a way to move beyond negative aspects of centralized (anti-)social media. But, hey, my wife and I have more than 5000 books in our house, so all this might just be my hoarding tendencies trying to justify themselves? :-) Well, off to an unrelated job. And it is unrelated precisely because most of the big paying players in this space (e.g. Google and Facebook) want to get between people and their data -- and typical startups make deals with investors which limit their ability to do FOSS. Still, I can hope there will eventually be, say, an "Automattic" of this "intertwingling" equivalent of WordPress. I actually had hoped Automattic might itself become such a company, but they sadly seem to now see themselves just as a Wix/Squarespace competitor (e.g. adopting Slack for internal communications -- and not even Mattermost or Matrix.org if they want to let someone else own that space and cooperate with another FOSS system). At least it is nice to think about these things in my spare time -- although sadly at the cost of other experiences. Maybe it is all just a mistake to use my non-job time like that, like Piotr Solnica suggests: https://twitter.com/_solnic_/status/953578620617920513 https://twitter.com/_solnic_/status/953578620617920513