9 ms·

TMTP a Internet protocol combining elements of email and the web

77 points by modinfo 4y ago

cestith 4y ago
Oh, look, we've come full circle to private bulletin boards with a custom client.
towaway15463 4y ago
The demo is not really usable on mobile. Glad to see efforts to build new protocols, I think that’s going to be the only way to fix the web.
- deleted 4y ago
  [deleted]
- networkimprov 4y ago
  (Author here.) The client UI is designed for desktop. A mobile frontend is in the works. Tentatively planning to use Ionic Framework with Vue.
mxuribe 4y ago
Firstly, i love the recursive nature of the implementaion: "mnm is not mail"! Also, quite a fascinating approach; hoping it works out! Me being a fanboy of the matrix protocol, I've often wondered if there is a valid use-case for matrix to enable safe, secure communications between site user/customer, and site manager(s). Not just chat, but a sort of customer support mechanism, but also a safe, private mechanism for site managers and/or orgs to do reasanble outreach to user/customers. And, no, it need not be chatbots, though i suppose there would be a desire by some orgs for an implementation like that at the end of the day. My hope would be of course for a better chatbot than current standards. Obviously, there would need to be the delicate balance of ease of use, privacy for user/customer, ideally decentralization (for empowering liberty/freedoms), integrity of content conveyed (by way of non-tampering/security of content), etc. In any case, good luck to the folks behind this effort; the more internet protocols (and not centralizied systems/orgs) that can help people, the better!
- networkimprov 4y ago
  (Author here.) Thanks for your interest & support! Yes, the default concept for an Internet messaging service is "everyone can reach anyone". This isn't helpful for online services and businesses who need to reach their members/customers directly & securely, and with a rich content model (e.g. survey/form in message, and structured result in reply).
indymike 4y ago
Interesting. The protocol TMT, seems to have the idea of users having multiple nodes (which are referred to in comments as devices). I suspect if I were to use this in a product, I'd use nodes for application instances instead, so if a user had multiple apps (i.e. consumer payment app and merchant dashboard app) on the same device I could keep them straight. Seems promising.
- networkimprov 4y ago
  (Author here.) TMTP clients maintain concurrent (possibly intermittent) connections to any number of TMTP services, and each service gets its own inbox. So you don't need a separate app instance per service. Inboxes for a set of related services could conceivably be merged, tho I haven't done any work on that yet.
  - indymike 4y ago
    Thanks for clarifying.
duncan-donuts 4y ago
I’m struggling to understand what this actually does. I poked around looking for information on the actual protocol, read through the site, and I don’t understand what this solves. The mechanics of the protocol aren’t very well defined, and the site says a lot of things without giving much information. TMTP isn’t an idea I would invest in and it’s nice that people are working on new protocols.
vxNsr 4y ago
I don’t quite understand how this works. They say it’s a new protocol but it’s still using a browser which would imply it’s layered on top of UDP or TCP. Building a new protocol today is kinda like inventing a new engine that doesn’t require gas, it’s gonna need a giant motivator to be adopted and trillions of dollars. The days of grassroots bootstrapping are over, no one has any reason to adopt your thing because what we have now is Good Enough(tm). Your new thing needs to be revolutionary AND have government backing across multiple governments and administrations. There is no other way, consider the electric car vs hydrogen.
- cestith 4y ago
  Most Internet application protocols run on top of UDP or TCP. They mean a new application protocol, like SMTP, HTTP, HTTPS, L2TP, POP3, IMAP4, SNMP, LDAP, SSH, FTP, RDP, LPD, XMPP, Kerberos, remote syslog, etc etc etc. If you're on any flavor of Unix, take a look at /etc/services and you'll see there are lots of protocols in the world.
  - icedchai 4y ago
    The protocols you list are decades old. There hasn't been a new mainstream protocol in a very long time. "Innovation" kind of stopped when we started tunneling everything over HTTP(S).
    - cestith 4y ago
      HTTP2 and HTTP3 themselves are relatively recent and widely used. They also happen to work on top of TCP and UDP rather than supplanting them. ACME protocol is fairly recent and widely used. WebSocket protocol is barely over one decade old. DNSSEC is a little over a decade and a half old. NVMe over Ethernet is barely half a decade old and still gaining support. 400GBASE-LR4-6 is from last year. People are still working on terabit/second Ethernet. There are loads of protocols on top of the HTTP family. DNS over HTTP is a protocol. SOAP, JSON-RPC, and REST are protocols. IPFS is a protocol. To say there's no innovation in protocols when it's literally happening all the way up and down the stack is just silly and shortsighted.
RajT88 4y ago
What sites use this? Seems like Hilton is mostly what comes up when I try searching for "sites supporting tmtp"
squarefoot 4y ago
About time someone thought of replacing email protocols with something new but without adding the bloat that seems so inevitable today. I would also think of integrating functions to replace also NNTP (News, USENET), so that it would cover all needs between private one to one and public many to many communications.
- myself248 4y ago
  Compare also NNCP: http://www.nncpgo.org/ http://www.nncpgo.org/
- networkimprov 4y ago
  (Author here.) I imagine that existing web-based bulletin boards would simply copy messages to TMTP users if requested. Conversely, TMTP groups/lists could be copied to a read-only website.
Forge36 4y ago
Previous discussion 208 comments https://news.ycombinator.com/item?id=25804869 https://news.ycombinator.com/item?id=25804869
eis 4y ago
Seems like a poorly thought out protocol. 1. Inconsistences like the value for "op" being sometimes integer but then sometimes string. Why is this an integer anyways? We are already in inefficient JSON land and a lot of the rest is strings. 2. No mentioning of ordering or guarantees in the draft spec. Bad recommendation to use time for sequential message ids. 3. Says E2EE not in it right now but planned. Says E2EE not default because sites want to scan and archive messages but then the page says email is bad because it doesn't have E2EE and nodes can scan messages? 4. The usage of the word "node" is confusing and inconsistent. Sometimes it's described as a device, sometimes it's a password. "nodeid", "node", "newnode"... 5. Seems like one can spoof the alias for each message? What's the reason for this? The page specifically mentions phishing being an issue with email but wouldn't this help phishers? There is no way that I can see if a message came from the site itself or a random user on it. Then a quick look at the code. First noticed the unusual (for Go) coding convention like "tFoo" structs, "eErrFoo" errors. Calling a mutex "door", weird "//:" comments. I haven't tested it but I'm fairly sure the convoluted way of storing user info in flat files has path traversal issues. Seems like there's a cache for alias -> uid but anyone can pick any alias so they would collide? Just from a cursory glance. Also doesn't seem like it has moved much in a couple years.
- networkimprov 4y ago
  (Author here.) Development has been paused for ~1y as I've been at work creating a vehicle to take this forward. The protocol specifics (e.g. identifiers & types) aren't polished, because I focused on features in working code. Thanks for the feedback! You can't have alias collisions; it checks whether the string you propose is acceptable & available :) A sender's alias is (or will be) checked against their recorded aliases. The reason why lack of E2EE in email is a serious problem is that email traverses at least two "federated" services, which are typically third parties with whom you have no intrinsic need for a relationship. A TMTP message passes through a single service which you explicitly joined. I don't see how your feedback justifies "poorly thought out"; it's merely an unpolished preview.
  - georgia_peach 4y ago
    Wrong place to share. The harmless people here will nitpick you to death as the adult version of sucking-up to teacher & showing how smart they are. The dangerous ones will silently steal the better parts of your idea & run with them, but with better funding & connections.
vidarh 4y ago
It seems to be conflating a communications protocol, an application level list management protocol, and parts of a message format. It'd be far more likely to gain support if they unbundled this and made it work as an upgrade over e-mail, which would seem to make a lot of the spec redundant.
- networkimprov 4y ago
  (Author here.) The message format definition is included in the protocol doc only as a convenience. Ultimately it would be two standards. Trying to "upgrade" the email ecosystem is an impossible goal, and the requirements I'm working to can't be shoehorned into the email model.
  - vidarh 4y ago
    I don't see anything in your current page that can't be layered into the e-mail model, and my prediction is that this will never get serious traction for that reason, because anything people want from it will just be integrated into e-mail if it starts to get attention. Consider that SMTP was "shoehorned into" the email model, and so was HTML, and attachments. My first e-mail exchange involved UUCP rather than SMTP. For many years my e-mail clients did not understand this newfangled HTML thing. E-mail is extremely malleable because e-mail is extremely simple. Add a "To: " header to a text file and you have a basic e-mail. Copy the file to another machine and you have a mail transport (that's roughly how UUCP worked - by implementing basic file transfer between hosts, and layering distribution of that mail on top with a program that'd read a file of concatenated e-mail messages and distribute them to mailboxes). E-mail is at its core a simple key-value header format, a message body and an address. Everything else is auxiliary. Transport layers can coexist: UUCP coexisted with SMTP for many years. If you invent a new one you just need to decide if you care about having a gateway to let "outsiders" contact you. Message formats can evolve: As long as you provide the basic MIME headers, you're in full control of the format of the message past the first set of headers. You choose how much interop you want by whether or not you use multipart/alternative to provide version readable by "old style" clients. These features are why e-mail in its current form is still around. What you gain* from this is interop and an opportunity to have your system evangelise itself as you use it. To address your page more specifically: >+ choose the sites where you participate >+ select which members of a site can message you If you enforce DMARC (see below), you can do this in a mail server simply with an option to whitelist domains or users or both. >+ always know from which site a message originated This is supported in e-mail today via DMARC. The reason you still receive e-mail for which it's not the case is lack of universal deployment. You can solve this issue simply by configuring a mail server to reject all e-mail that fails DMARC. > + can block anyone with whom you’ve made contact > + may leave a site and never see traffic from it again If you enforce DMARC, you can certainly block sites at the server level or client level or both, or just block individuals. The upside of doing this with DMARC is that people with it correctly set up will only notice anything different if they try to send you unsolicited messages and get an error message back. > + message formatting & layout via Markdown (aka CommonMark) Solved with client support - just needs the appropriate MIME type set and a ideally a multipart/alternative to deal with any clients that fail to fall back on plain text rendering. > + hyperlinks to messages and other threads Solved with a URN spec and client support. > + hashtags and private tags > + slide deck layouts > + data-driven charts & graphs Solved with message formats and MIME types. > + forms/surveys whose results are collected into tables Needs server side support if you want something standardised, but works fine with links to the web today with appropriate client support and HTML, and if you want to provide an upgradeable path it's easily done with client support and a MIME type + requiring multipart alternative w/ a link to a web version for clients without support. So to summarise how I would have done this if I wanted to meet the goals stated on your page: * Pick a mail server that supports JMAP for the client. * Configure it to reject all mail that fails DMARC with a suitable message explaining why. Now you've created pressure to add DMARC for those who don't. * Add a JMAP profile for the client to tell the server if it wants whitelisting or blacklisting, and which users and domains it wants to whitelist and blacklist. I'll tell you right away that most users will reject a whitelisting option, so consider greylisting or quarantine as options. * Optionally define a MIME type to carry subscription/follow/unfollow requests to remote sites, and a DNS record to hold public keys to sign requests. * Define your desired message format and pick a MIME type. * Specify the rendering of a plain text version of the message formats for delivery via multipart/alternative, which includes a footer that specifies that this is a possibly imprecise rendering and links to an info page. Using JMAP means people would be able to connect with any JMAP capable client and "just" lose out on functionality not supported by that client. Using a mail server means most well configured mail servers can already talk to you if whitelisted/not blacklisted. Using multipart/alternative that way means clients can "upgrade" gradually and possibly after users see a link to an info page, and the footer in the plain text parts (or other rendered alternative) would help evangelise the system.
networkimprov 4y ago
Hi, author here, AMA!
- networkimprov 4y ago
  FWIW, most of the roadmap for this project is not described on the website nor defined in the protocol. This is a deep topic, and "messaging" is not the only goal, nor even the principal one :)
sbuk 4y ago
Not sure about the “features” at the end. Specifically Markdown support. This is a client-side issue. If MUAs parsed Markdown, then there is no reason that markdown couldn’t replace html in the relevant part of a MIME multipart message. The SMTP server only handles ASCII characters; it really doesn’t care how that is presented. It also afford the possibly of plaintext as a fall back. Arguably, you could have all three.

77 points by modinfo 4y ago

Oh, look, we've come full circle to private bulletin boards with a custom client.

The demo is not really usable on mobile. Glad to see efforts to build new protocols, I think that’s going to be the only way to fix the web.

deleted 4y ago
[deleted]
networkimprov 4y ago
(Author here.) The client UI is designed for desktop. A mobile frontend is in the works. Tentatively planning to use Ionic Framework with Vue.

mxuribe 4y ago

Firstly, i love the recursive nature of the implementaion: "mnm is not mail"! Also, quite a fascinating approach; hoping it works out! Me being a fanboy of the matrix protocol, I've often wondered if there is a valid use-case for matrix to enable safe, secure communications between site user/customer, and site manager(s). Not just chat, but a sort of customer support mechanism, but also a safe, private mechanism for site managers and/or orgs to do reasanble outreach to user/customers. And, no, it need not be chatbots, though i suppose there would be a desire by some orgs for an implementation like that at the end of the day. My hope would be of course for a better chatbot than current standards. Obviously, there would need to be the delicate balance of ease of use, privacy for user/customer, ideally decentralization (for empowering liberty/freedoms), integrity of content conveyed (by way of non-tampering/security of content), etc. In any case, good luck to the folks behind this effort; the more internet protocols (and not centralizied systems/orgs) that can help people, the better!

networkimprov 4y ago
(Author here.) Thanks for your interest & support! Yes, the default concept for an Internet messaging service is "everyone can reach anyone". This isn't helpful for online services and businesses who need to reach their members/customers directly & securely, and with a rich content model (e.g. survey/form in message, and structured result in reply).

indymike 4y ago

Interesting. The protocol TMT, seems to have the idea of users having multiple nodes (which are referred to in comments as devices). I suspect if I were to use this in a product, I'd use nodes for application instances instead, so if a user had multiple apps (i.e. consumer payment app and merchant dashboard app) on the same device I could keep them straight. Seems promising.

networkimprov 4y ago
(Author here.) TMTP clients maintain concurrent (possibly intermittent) connections to any number of TMTP services, and each service gets its own inbox. So you don't need a separate app instance per service. Inboxes for a set of related services could conceivably be merged, tho I haven't done any work on that yet.
- indymike 4y ago
  Thanks for clarifying.

duncan-donuts 4y ago

I’m struggling to understand what this actually does. I poked around looking for information on the actual protocol, read through the site, and I don’t understand what this solves. The mechanics of the protocol aren’t very well defined, and the site says a lot of things without giving much information. TMTP isn’t an idea I would invest in and it’s nice that people are working on new protocols.

vxNsr 4y ago

I don’t quite understand how this works. They say it’s a new protocol but it’s still using a browser which would imply it’s layered on top of UDP or TCP. Building a new protocol today is kinda like inventing a new engine that doesn’t require gas, it’s gonna need a giant motivator to be adopted and trillions of dollars. The days of grassroots bootstrapping are over, no one has any reason to adopt your thing because what we have now is Good Enough(tm). Your new thing needs to be revolutionary AND have government backing across multiple governments and administrations. There is no other way, consider the electric car vs hydrogen.

cestith 4y ago
Most Internet application protocols run on top of UDP or TCP. They mean a new application protocol, like SMTP, HTTP, HTTPS, L2TP, POP3, IMAP4, SNMP, LDAP, SSH, FTP, RDP, LPD, XMPP, Kerberos, remote syslog, etc etc etc. If you're on any flavor of Unix, take a look at /etc/services and you'll see there are lots of protocols in the world.
- icedchai 4y ago
  The protocols you list are decades old. There hasn't been a new mainstream protocol in a very long time. "Innovation" kind of stopped when we started tunneling everything over HTTP(S).
  - cestith 4y ago
    HTTP2 and HTTP3 themselves are relatively recent and widely used. They also happen to work on top of TCP and UDP rather than supplanting them. ACME protocol is fairly recent and widely used. WebSocket protocol is barely over one decade old. DNSSEC is a little over a decade and a half old. NVMe over Ethernet is barely half a decade old and still gaining support. 400GBASE-LR4-6 is from last year. People are still working on terabit/second Ethernet. There are loads of protocols on top of the HTTP family. DNS over HTTP is a protocol. SOAP, JSON-RPC, and REST are protocols. IPFS is a protocol. To say there's no innovation in protocols when it's literally happening all the way up and down the stack is just silly and shortsighted.

RajT88 4y ago

What sites use this? Seems like Hilton is mostly what comes up when I try searching for "sites supporting tmtp"

squarefoot 4y ago

About time someone thought of replacing email protocols with something new but without adding the bloat that seems so inevitable today. I would also think of integrating functions to replace also NNTP (News, USENET), so that it would cover all needs between private one to one and public many to many communications.

myself248 4y ago
Compare also NNCP: http://www.nncpgo.org/ http://www.nncpgo.org/
networkimprov 4y ago
(Author here.) I imagine that existing web-based bulletin boards would simply copy messages to TMTP users if requested. Conversely, TMTP groups/lists could be copied to a read-only website.

Forge36 4y ago

Previous discussion 208 comments https://news.ycombinator.com/item?id=25804869 https://news.ycombinator.com/item?id=25804869

eis 4y ago

Seems like a poorly thought out protocol. 1. Inconsistences like the value for "op" being sometimes integer but then sometimes string. Why is this an integer anyways? We are already in inefficient JSON land and a lot of the rest is strings. 2. No mentioning of ordering or guarantees in the draft spec. Bad recommendation to use time for sequential message ids. 3. Says E2EE not in it right now but planned. Says E2EE not default because sites want to scan and archive messages but then the page says email is bad because it doesn't have E2EE and nodes can scan messages? 4. The usage of the word "node" is confusing and inconsistent. Sometimes it's described as a device, sometimes it's a password. "nodeid", "node", "newnode"... 5. Seems like one can spoof the alias for each message? What's the reason for this? The page specifically mentions phishing being an issue with email but wouldn't this help phishers? There is no way that I can see if a message came from the site itself or a random user on it. Then a quick look at the code. First noticed the unusual (for Go) coding convention like "tFoo" structs, "eErrFoo" errors. Calling a mutex "door", weird "//:" comments. I haven't tested it but I'm fairly sure the convoluted way of storing user info in flat files has path traversal issues. Seems like there's a cache for alias -> uid but anyone can pick any alias so they would collide? Just from a cursory glance. Also doesn't seem like it has moved much in a couple years.

networkimprov 4y ago
(Author here.) Development has been paused for ~1y as I've been at work creating a vehicle to take this forward. The protocol specifics (e.g. identifiers & types) aren't polished, because I focused on features in working code. Thanks for the feedback! You can't have alias collisions; it checks whether the string you propose is acceptable & available :) A sender's alias is (or will be) checked against their recorded aliases. The reason why lack of E2EE in email is a serious problem is that email traverses at least two "federated" services, which are typically third parties with whom you have no intrinsic need for a relationship. A TMTP message passes through a single service which you explicitly joined. I don't see how your feedback justifies "poorly thought out"; it's merely an unpolished preview.
- georgia_peach 4y ago
  Wrong place to share. The harmless people here will nitpick you to death as the adult version of sucking-up to teacher & showing how smart they are. The dangerous ones will silently steal the better parts of your idea & run with them, but with better funding & connections.

vidarh 4y ago

It seems to be conflating a communications protocol, an application level list management protocol, and parts of a message format. It'd be far more likely to gain support if they unbundled this and made it work as an upgrade over e-mail, which would seem to make a lot of the spec redundant.

networkimprov 4y ago
(Author here.) The message format definition is included in the protocol doc only as a convenience. Ultimately it would be two standards. Trying to "upgrade" the email ecosystem is an impossible goal, and the requirements I'm working to can't be shoehorned into the email model.
- vidarh 4y ago
  I don't see anything in your current page that can't be layered into the e-mail model, and my prediction is that this will never get serious traction for that reason, because anything people want from it will just be integrated into e-mail if it starts to get attention. Consider that SMTP was "shoehorned into" the email model, and so was HTML, and attachments. My first e-mail exchange involved UUCP rather than SMTP. For many years my e-mail clients did not understand this newfangled HTML thing. E-mail is extremely malleable because e-mail is extremely simple. Add a "To: " header to a text file and you have a basic e-mail. Copy the file to another machine and you have a mail transport (that's roughly how UUCP worked - by implementing basic file transfer between hosts, and layering distribution of that mail on top with a program that'd read a file of concatenated e-mail messages and distribute them to mailboxes). E-mail is at its core a simple key-value header format, a message body and an address. Everything else is auxiliary. Transport layers can coexist: UUCP coexisted with SMTP for many years. If you invent a new one you just need to decide if you care about having a gateway to let "outsiders" contact you. Message formats can evolve: As long as you provide the basic MIME headers, you're in full control of the format of the message past the first set of headers. You choose how much interop you want by whether or not you use multipart/alternative to provide version readable by "old style" clients. These features are why e-mail in its current form is still around. What you gain* from this is interop and an opportunity to have your system evangelise itself as you use it. To address your page more specifically: >+ choose the sites where you participate >+ select which members of a site can message you If you enforce DMARC (see below), you can do this in a mail server simply with an option to whitelist domains or users or both. >+ always know from which site a message originated This is supported in e-mail today via DMARC. The reason you still receive e-mail for which it's not the case is lack of universal deployment. You can solve this issue simply by configuring a mail server to reject all e-mail that fails DMARC. > + can block anyone with whom you’ve made contact > + may leave a site and never see traffic from it again If you enforce DMARC, you can certainly block sites at the server level or client level or both, or just block individuals. The upside of doing this with DMARC is that people with it correctly set up will only notice anything different if they try to send you unsolicited messages and get an error message back. > + message formatting & layout via Markdown (aka CommonMark) Solved with client support - just needs the appropriate MIME type set and a ideally a multipart/alternative to deal with any clients that fail to fall back on plain text rendering. > + hyperlinks to messages and other threads Solved with a URN spec and client support. > + hashtags and private tags > + slide deck layouts > + data-driven charts & graphs Solved with message formats and MIME types. > + forms/surveys whose results are collected into tables Needs server side support if you want something standardised, but works fine with links to the web today with appropriate client support and HTML, and if you want to provide an upgradeable path it's easily done with client support and a MIME type + requiring multipart alternative w/ a link to a web version for clients without support. So to summarise how I would have done this if I wanted to meet the goals stated on your page: * Pick a mail server that supports JMAP for the client. * Configure it to reject all mail that fails DMARC with a suitable message explaining why. Now you've created pressure to add DMARC for those who don't. * Add a JMAP profile for the client to tell the server if it wants whitelisting or blacklisting, and which users and domains it wants to whitelist and blacklist. I'll tell you right away that most users will reject a whitelisting option, so consider greylisting or quarantine as options. * Optionally define a MIME type to carry subscription/follow/unfollow requests to remote sites, and a DNS record to hold public keys to sign requests. * Define your desired message format and pick a MIME type. * Specify the rendering of a plain text version of the message formats for delivery via multipart/alternative, which includes a footer that specifies that this is a possibly imprecise rendering and links to an info page. Using JMAP means people would be able to connect with any JMAP capable client and "just" lose out on functionality not supported by that client. Using a mail server means most well configured mail servers can already talk to you if whitelisted/not blacklisted. Using multipart/alternative that way means clients can "upgrade" gradually and possibly after users see a link to an info page, and the footer in the plain text parts (or other rendered alternative) would help evangelise the system.

networkimprov 4y ago

Hi, author here, AMA!

networkimprov 4y ago
FWIW, most of the roadmap for this project is not described on the website nor defined in the protocol. This is a deep topic, and "messaging" is not the only goal, nor even the principal one :)

sbuk 4y ago

Not sure about the “features” at the end. Specifically Markdown support. This is a client-side issue. If MUAs parsed Markdown, then there is no reason that markdown couldn’t replace html in the relevant part of a MIME multipart message. The SMTP server only handles ASCII characters; it really doesn’t care how that is presented. It also afford the possibly of plaintext as a fall back. Arguably, you could have all three.