6 ms·

Matrix was invented because XMPP was getting too complex. Are we once again at a point where it's time to start over? A federated asynchronous group chat prot

by Tepix 2y ago

Matrix was invented because XMPP was getting too complex. Are we once again at a point where it's time to start over? A federated asynchronous group chat protocol with modern e2e encryption for desktop and mobile use (thus, not always online) is impossible to build without a faire share of complicated corner cases.

Gormo 2y ago
> Are we once again at a point where it's time to start over? Ask Sisyphus. In the meantime, while everything else keeps going through newer iterations of XKCD 927 (https://xkcd.com/927/ https://xkcd.com/927/), IRC keeps chugging along, and gaining new features and functionality as it enters its fourth decade despite the previous commenter's gripes.
- hnarn 2y ago
  > Ask Sisyphus. that doesn't make any sense.
  - Gormo 2y ago
    What doesn't make sense about it?
emmanueloga_ 2y ago
I guess I did not make the scope of the project clear. Both Matrix and XMPP specs add up to 100s of pages, 500 or so give or take. IRCv3 seems to be over 100 pages too. In comparison, Gemini is about 10 pages long or so. A bunch of people here talk about setting up a small server for family and friends, so I think a smaller protocol, akin to Gemini in size, could be a lot of fun to work with for small scale deployments.
- MattJ100 2y ago
  The protocol isn't really an issue for the use-case you talk about. I founded the Snikket project, which aims squarely at the family-and-friends use case you mention (after all, it was made to scratch my own itch - my family's excessive use of WhatsApp for communicating with each other). I can tell you that my family don't care a bit whether Snikket uses IRC, XMPP or Matrix or some real-time Gemini equivalent. There may be some scalability differences between different protocols/implementations for the admin of the service, but Snikket fits comfortably on even low-end Raspberry Pi devices, and literally over half of the typical resource usage is by the web dashboard (yay Python). So what difference does the protocol make? It can make a difference to the developer experience. If all you want to do is exchange text messages, then yeah, XMPP and Matrix are absolutely overkill. But - especially for a family-and-friends use case - people also want file sharing, audio/video calls, and all that stuff. It very quickly gets quite complex to support all this stuff, especially in a way that allows you to evolve the protocol over time (trust me, what you think of as core messaging features today, were not a thing 10+ years ago, and messaging in 10+ years will also involve a new set of features). There will always be a set of users for whom plain text messaging is enough (90% of my own daily communication is via messaging in a terminal app). However that set does not intersect significantly with the general population, and practically none of my family members would accept such a solution as a replacement for WhatsApp.
  - linsomniac 2y ago
    TIL: https://snikket.org/ https://snikket.org/
  - imiric 2y ago
    Hey, your project looks interesting, thanks for building and sharing it. One question: are you aware of Jami[1], f.k.a. Ring? If so, how does it compare to Snikket? I see that Snikket requires a server, whereas Jami is P2P. The benefit of a server is probably that messages can be stored centrally and not on each device. But I can see pros and cons of either approach. [1]: https://jami.net/ https://jami.net/
    - MattJ100 2y ago
      Hey, thanks! I've been into messaging for quite a long time - network protocols and particularly those involving online communication are among my favourite tech topics :) So yeah, I follow various projects. You're right that there are pros and cons. Obviously, not having to run a server is a big pro for many. However, the first thing to remember when researching messaging solutions - no matter what anyone tells you - there are always servers! What differs between projects/platforms is what the servers do, and who runs them. Jami uses a network of public servers that form a distributed hash table (see https://github.com/savoirfairelinux/opendht https://github.com/savoirfairelinux/opendht ). It's a neat design, and they have done a good job tackling the challenges of P2P messaging. Last time I looked in, it still required both users to be connected at the same time for message delivery/sync to work (the devices use the DHT to discover each other, and then exchange messages). This is a fairly common issue for P2P systems, and can be frustrating in a mobile-dominated world. Their DHT software does support push notifications, which helps with that though. Another project in this category is Briar, which uses the existing network of Tor servers - and therefore adds IP address masking and a layer of metadata protection (as always, there are limitations, e.g. https://code.briarproject.org/briar/briar/-/wikis/FAQ#does-briar-provide-anonymity https://code.briarproject.org/briar/briar/-/wikis/FAQ#does-b... ). Briar built the concept of "mailbox" nodes you can run ( https://briarproject.org/download-briar-mailbox/ https://briarproject.org/download-briar-mailbox/ ) to overcome some of the problems with P2P messaging. With Snikket, instead of using existing publicly shared infrastructure, you just run your own server (e.g. VPS or Raspberry Pi or whatever) which is responsible just for your users, and your users connect directly to it, improving (meta)data locality. This makes the design very simple, reliable and efficient (e.g. with battery/bandwidth). It also enables some important (for our use case) convenience/UX features, such as the ability to add restrictions on certain accounts (e.g. for children), and server-managed contact lists so all your family members don't have to manually add each other as contacts one-by-one. Things like that. No approach is universally better than every other, but I much prefer the Snikket model for the family-and-friends use case. Not that we don't have our own challenges. Our iOS app is probably the weakest part right now (in terms of UX and general polish). Something I'm working hard to get fixed in 2025.
- RadiozRadioz 2y ago
  Well, all the XEPs add up to 100s of pages, but the minimal requirements for basic chat are pretty small. It gets more complex as you add more features, but importantly the complexity is managed well. You don't have to do the crazy fragile hacks that IRC needs, because XMPP was designed expressly as an extensible protocol.
ezst 2y ago
To me, it's slowly become obvious that Matrix was invented because certain people wanted to ride the trendy wave of VC money propping up Slack and friends back in those days. Matrix has been all about superlative marketing and over-promises with little substance and theoretical underpinnings to back it up. The fact that they were spreading FUD and disinformation against XMPP and the alternatives in their early days should have been a red herring. The fact that they haven't stabilized their protocol and made it scalable to mid-level deployments after more than a decade should be a reason for pause. The fact that after so long, due to the sheer complexity of it, only one entity has emerged as willing to sink the costs of Matrix should be a reason for worries. I think that we are due for an XMPP resurgence. Not that it is perfect by any stretch of mind, but it is no-BS, mature, lightweight on resources and bandwidth and verifiably fit for purpose, whether you want to host small-scale for your family & friends, or for the whole world with the ambition to be the next GTalk, facebook messenger, or whatsapp (all of them XMPP services at one point or another of their histories)
- zaik 2y ago
  > red herring I think you meant to say red flag, not red herring: https://en.wikipedia.org/wiki/Red_herring https://en.wikipedia.org/wiki/Red_herring
  - ezst 2y ago
    Oh, TIL! Thanks :-)
- tcfhgj 2y ago
  > Matrix has been all about superlative marketing and over-promises with little substance and theoretical underpinnings to back it up. I don't think so. The underpinnings were there, may it be the concepts for E2EE, P2P, Verification, Bridging, decentralized rooms and conferencing, synchronized history, etc. Not everything is production ready let alone finished with perfect UX, but it doesn't lack theoretical underpinnings. > The fact that they were spreading FUD and disinformation against XMPP In the past, I have experienced it more often vice versa than this way. > The fact that they haven't stabilized their protocol and made it scalable to mid-level deployments after more than a decade should be a reason for pause. There are already pretty large deployments (1+ million users), so I think it's scalable to mid-level deployments. > only one entity has emerged as willing to sink the costs of Matrix should be a reason for worries. idk what you mean > all of them XMPP services at one point or another of their histories histories is an important keyword

by Tepix 2y ago

Gormo 2y ago

> Are we once again at a point where it's time to start over? Ask Sisyphus. In the meantime, while everything else keeps going through newer iterations of XKCD 927 (https://xkcd.com/927/ https://xkcd.com/927/), IRC keeps chugging along, and gaining new features and functionality as it enters its fourth decade despite the previous commenter's gripes.

hnarn 2y ago
> Ask Sisyphus. that doesn't make any sense.
- Gormo 2y ago
  What doesn't make sense about it?

emmanueloga_ 2y ago

I guess I did not make the scope of the project clear. Both Matrix and XMPP specs add up to 100s of pages, 500 or so give or take. IRCv3 seems to be over 100 pages too. In comparison, Gemini is about 10 pages long or so. A bunch of people here talk about setting up a small server for family and friends, so I think a smaller protocol, akin to Gemini in size, could be a lot of fun to work with for small scale deployments.

MattJ100 2y ago
The protocol isn't really an issue for the use-case you talk about. I founded the Snikket project, which aims squarely at the family-and-friends use case you mention (after all, it was made to scratch my own itch - my family's excessive use of WhatsApp for communicating with each other). I can tell you that my family don't care a bit whether Snikket uses IRC, XMPP or Matrix or some real-time Gemini equivalent. There may be some scalability differences between different protocols/implementations for the admin of the service, but Snikket fits comfortably on even low-end Raspberry Pi devices, and literally over half of the typical resource usage is by the web dashboard (yay Python). So what difference does the protocol make? It can make a difference to the developer experience. If all you want to do is exchange text messages, then yeah, XMPP and Matrix are absolutely overkill. But - especially for a family-and-friends use case - people also want file sharing, audio/video calls, and all that stuff. It very quickly gets quite complex to support all this stuff, especially in a way that allows you to evolve the protocol over time (trust me, what you think of as core messaging features today, were not a thing 10+ years ago, and messaging in 10+ years will also involve a new set of features). There will always be a set of users for whom plain text messaging is enough (90% of my own daily communication is via messaging in a terminal app). However that set does not intersect significantly with the general population, and practically none of my family members would accept such a solution as a replacement for WhatsApp.
- linsomniac 2y ago
  TIL: https://snikket.org/ https://snikket.org/
- imiric 2y ago
  Hey, your project looks interesting, thanks for building and sharing it. One question: are you aware of Jami[1], f.k.a. Ring? If so, how does it compare to Snikket? I see that Snikket requires a server, whereas Jami is P2P. The benefit of a server is probably that messages can be stored centrally and not on each device. But I can see pros and cons of either approach. [1]: https://jami.net/ https://jami.net/
  - MattJ100 2y ago
    Hey, thanks! I've been into messaging for quite a long time - network protocols and particularly those involving online communication are among my favourite tech topics :) So yeah, I follow various projects. You're right that there are pros and cons. Obviously, not having to run a server is a big pro for many. However, the first thing to remember when researching messaging solutions - no matter what anyone tells you - there are always servers! What differs between projects/platforms is what the servers do, and who runs them. Jami uses a network of public servers that form a distributed hash table (see https://github.com/savoirfairelinux/opendht https://github.com/savoirfairelinux/opendht ). It's a neat design, and they have done a good job tackling the challenges of P2P messaging. Last time I looked in, it still required both users to be connected at the same time for message delivery/sync to work (the devices use the DHT to discover each other, and then exchange messages). This is a fairly common issue for P2P systems, and can be frustrating in a mobile-dominated world. Their DHT software does support push notifications, which helps with that though. Another project in this category is Briar, which uses the existing network of Tor servers - and therefore adds IP address masking and a layer of metadata protection (as always, there are limitations, e.g. https://code.briarproject.org/briar/briar/-/wikis/FAQ#does-briar-provide-anonymity https://code.briarproject.org/briar/briar/-/wikis/FAQ#does-b... ). Briar built the concept of "mailbox" nodes you can run ( https://briarproject.org/download-briar-mailbox/ https://briarproject.org/download-briar-mailbox/ ) to overcome some of the problems with P2P messaging. With Snikket, instead of using existing publicly shared infrastructure, you just run your own server (e.g. VPS or Raspberry Pi or whatever) which is responsible just for your users, and your users connect directly to it, improving (meta)data locality. This makes the design very simple, reliable and efficient (e.g. with battery/bandwidth). It also enables some important (for our use case) convenience/UX features, such as the ability to add restrictions on certain accounts (e.g. for children), and server-managed contact lists so all your family members don't have to manually add each other as contacts one-by-one. Things like that. No approach is universally better than every other, but I much prefer the Snikket model for the family-and-friends use case. Not that we don't have our own challenges. Our iOS app is probably the weakest part right now (in terms of UX and general polish). Something I'm working hard to get fixed in 2025.
RadiozRadioz 2y ago
Well, all the XEPs add up to 100s of pages, but the minimal requirements for basic chat are pretty small. It gets more complex as you add more features, but importantly the complexity is managed well. You don't have to do the crazy fragile hacks that IRC needs, because XMPP was designed expressly as an extensible protocol.

ezst 2y ago

To me, it's slowly become obvious that Matrix was invented because certain people wanted to ride the trendy wave of VC money propping up Slack and friends back in those days. Matrix has been all about superlative marketing and over-promises with little substance and theoretical underpinnings to back it up. The fact that they were spreading FUD and disinformation against XMPP and the alternatives in their early days should have been a red herring. The fact that they haven't stabilized their protocol and made it scalable to mid-level deployments after more than a decade should be a reason for pause. The fact that after so long, due to the sheer complexity of it, only one entity has emerged as willing to sink the costs of Matrix should be a reason for worries. I think that we are due for an XMPP resurgence. Not that it is perfect by any stretch of mind, but it is no-BS, mature, lightweight on resources and bandwidth and verifiably fit for purpose, whether you want to host small-scale for your family & friends, or for the whole world with the ambition to be the next GTalk, facebook messenger, or whatsapp (all of them XMPP services at one point or another of their histories)

zaik 2y ago
> red herring I think you meant to say red flag, not red herring: https://en.wikipedia.org/wiki/Red_herring https://en.wikipedia.org/wiki/Red_herring
- ezst 2y ago
  Oh, TIL! Thanks :-)
tcfhgj 2y ago
> Matrix has been all about superlative marketing and over-promises with little substance and theoretical underpinnings to back it up. I don't think so. The underpinnings were there, may it be the concepts for E2EE, P2P, Verification, Bridging, decentralized rooms and conferencing, synchronized history, etc. Not everything is production ready let alone finished with perfect UX, but it doesn't lack theoretical underpinnings. > The fact that they were spreading FUD and disinformation against XMPP In the past, I have experienced it more often vice versa than this way. > The fact that they haven't stabilized their protocol and made it scalable to mid-level deployments after more than a decade should be a reason for pause. There are already pretty large deployments (1+ million users), so I think it's scalable to mid-level deployments. > only one entity has emerged as willing to sink the costs of Matrix should be a reason for worries. idk what you mean > all of them XMPP services at one point or another of their histories histories is an important keyword