21 ms·

Is XMPP any good? Also, let's write a client in Tcl, maybe

101 points by wusspuss 4y ago

Freie_Messenger 4y ago
> Is XMPP any good? Sure - here are lots of informations: The chat standard XMPP (IETF): https://www.freie-messenger.de/sys_xmpp https://www.freie-messenger.de/sys_xmpp (german) Thoughts on interoperabiliy: https://www.freie-messenger.de/en/begriffe/interoperabilitaet/gedanken https://www.freie-messenger.de/en/begriffe/interoperabilitae... Quick overview of messenger systems (DE, EN, FR, ES, NL, ...): https://www.freie-messenger.de/en/systemvergleich https://www.freie-messenger.de/en/systemvergleich Because there are so many comments comparing XMPP with Matrix: https://www.freie-messenger.de/en/systemvergleich/xmpp-matrix https://www.freie-messenger.de/en/systemvergleich/xmpp-matri...
MattJ100 4y ago
As someone who has been working with XMPP for a long time, I can safely say that most of the criticisms here simply have no material effects in practice. Yes, an ack packet can be 19 characters long. Is that terrible? It would be just as easy to write a post like this about JSON, Matrix and so on. Both have many warts. But I really don't much care. No technology is perfect, and there will always be people who dislike choices that other people have made. What matters is what we do with the technology. XMPP has proven itself able to evolve over more than two decades, during which the internet and the way we use it has shifted dramatically, and yet it continues to be relevant as new people discover and adopt it. It's unfair to describe things like multi-device sync as "afterthoughts" when XMPP was one of the first real-time messaging protocols to have support for multiple concurrent connections of the same account built in, at a time before the iPhone was released, and most people had a single device with wired access to the internet only. Support for multi-device sync is part of the suite of protocol features required by modern XMPP implementations, these requirements are updated every year to account for the changing technology landscape: https://xmpp.org/about/compliance-suites/ https://xmpp.org/about/compliance-suites/ I personally like the simple delivery model of XMPP compared to the "distributed database" model that Matrix pursues. I think there are use-cases for both approaches. Both XMPP and Matrix have bridges between each other and third-party protocols (proprietary and open ones), which makes any attempt to create false competition between them an unhelpful effort. What's important is that both provide openly-documented protocols with open-source reusable implementations, and (in my opinion) most important of all: decentralized open communication networks. The problems facing the broader adoption of open networks for modern messaging are not protocol problems.
- dmitriid 4y ago
  > Support for multi-device sync is part of the suite of protocol features required by modern XMPP implementations, these requirements are updated every year The problems is that many of these modern features were unbelievably late. I still remember "The State of Mobile XMPP in 2016", https://gultsch.de/xmpp_2016.html https://gultsch.de/xmpp_2016.html At the time, 9 years after iPhone had come on the scene, most modern features were still experimental draft proposals implemented by no one. 5 more years later there's maybe one client that implements the modern features (Conversations for Android IIRC), and that's about it. And it's an unknown how many servers implement those features.
  - seba_dos1 4y ago
    > 5 more years later there's maybe one client that implements the modern features (Conversations for Android IIRC), and that's about it. ...and yet I'm using three different modern clients that work well with each other. > And it's an unknown how many servers implement those features. Enough to make me not have to care about it.
  - MattJ100 4y ago
    > And it's an unknown how many servers implement those features It's not: https://compliance.conversations.im/tests/ https://compliance.conversations.im/tests/ Obviously a new user is not expected to fathom what all this means before getting started, which is why there are curated lists of public XMPP services, e.g. https://providers.xmpp.net/ https://providers.xmpp.net/ and https://joinjabber.org/ https://joinjabber.org/ and for private servers there are projects like Snikket where the server software is developed in parallel with apps to ensure a consistent feature set.
- ajconway 4y ago
  Yet Matrix is gaining traction while XMPP is not. Maybe the key was to have funding, build and run a reference server and quickly iterate with a reference client for all major platforms.
  - olah_1 4y ago
grishka 4y ago
The thing with the Nokia 5800 (I also have one) and other phones not designed in the US is that outside of the US, most people paid for SMS per message. So it made sense to present the messages in that UI, you're paying for the whole thing anyway, could as well use all 160 (or 70 around here) characters. No one in their right mind would split into several messages something that fit into one. For this style of instant messaging, we used ICQ. There were unofficial ICQ clients for just about everything. On XMPP itself — I'm looking into it as the protocol for messaging for my federated social network. I first contemplated using Matrix, but then I looked at the spec, saw the mathematical notation for the room state synchronization algorithm, and noped outta there really fast.
- Arathorn 4y ago
  mental note to link https://matrix.org/docs/guides/implementing-stateres https://matrix.org/docs/guides/implementing-stateres from the matrix spec :p
  - grishka 4y ago
    Oh, thanks for this. I did look for a tutorial about implementing a minimally functional Matrix server like there are for ActivityPub[1][2], but didn't find any. [1] https://blog.joinmastodon.org/2018/06/how-to-implement-a-basic-activitypub-server/ https://blog.joinmastodon.org/2018/06/how-to-implement-a-bas... [2] https://blog.joinmastodon.org/2018/07/how-to-make-friends-and-verify-requests/ https://blog.joinmastodon.org/2018/07/how-to-make-friends-an...
- grishka 4y ago
  And after reading the whole article, I'd like to add: it really helps to treat these protocols like APIs. As in, if you're building an XMPP server, you wouldn't want to expose the actual clients as resources, for example. You'd instead just send each message to each connected client and present them all as one single "client" to other users and for the purpose of s2s federation. That would eliminate an entire class of issues, like messages getting lost, AND improve the user experience at the cost of a slight, barely noticeable spec noncompliance. I do a variation of this for ActivityPub in my project already. Except I don't implement c2s ActivityPub at all.
upofadown 4y ago
How old is this article? It talks about client to server encryption like it is some sort of optional thing. >Today you expect a messenger to be a database. I do? Not all of us want the levels of resource utilization this approach requires. Why would I want to be able to edit/delete old messages? I can just send new ones.
- IYasha 4y ago
  You've never (fully) used Telegram, have you?
deleted 4y ago
[deleted]
icedchai 4y ago
About 10 years ago, I worked on an XMPP client and experienced many of these issues. Fun times.
buttocks 4y ago
I said this on a past XMPP thread. It is a dead protocol due to mobile. I loved XMPP but without a push system it is useless on modern mobile operating systems.
- MattJ100 4y ago
  XMPP is widely used on mobile, it has supported push notifications for many years. Without it, you are right, using XMPP on mobile would be pretty painful, but that's not the case.
  - bertman 4y ago
    And yet, after all these years, mod_cloud_notify is still a community module and not even mentioned in prosody.cfg.lua.dist.
    - MattJ100 4y ago
      Indeed, mod_cloud_notify is high on our list of modules to merge for the next Prosody release, if not the highest. A lot of effort went into Prosody 0.12, which we released recently. It's unfortunate that mod_cloud_notify didn't quite make the cut for that version, but it was better than delaying the long overdue release even further. The vast majority of work on Prosody is unpaid volunteer time, and as with most open-source work there are certain kinds of tasks which are hard to find funding for, leading to the old "it'll be done when it's done" because we (the developers) simultaneously need to pay our bills. Also, in recognition of many desirable things being in community modules, one thing we did prioritize in 0.12 was the plugin installer, which should make it easier for people to add additional modules to their Prosody installation.
      - bertman 4y ago
        Sorry, my comment probably came off way too snarky. Love Prosody and the work you do with Snikket.
- Nasreddin_Hodja 4y ago
kseistrup 4y ago
There alread _was_ an xmpp client in tcl/tk: Coccinella ⌘ https://en.wikipedia.org/wiki/Coccinella_%28software%29 https://en.wikipedia.org/wiki/Coccinella_%28software%29
- Nasreddin_Hodja 4y ago
  There was Tkabber also.
  - kseistrup 4y ago
    Ah, yes!
  - libx 4y ago
    Unfortunately, none of them have support for OMEMO.
arendtio 4y ago
I have the impression that the discussion around XMPP isn't going anywhere. On the one side are the people who say it is outdated and can't handle mobile/multi-device/energy-efficient communication and on the other side are the people who argue, that XMPP got extensions to handle all those cases. I am part of the later group. However, instead of arguing I think it might be better to give a short summary about the state of XMPP from a users perpective. I use XMPP every day for several years now. In my opinion the best clients are the following: - Linux -> Gajim (also available for Windows) - iOS -> Siskin - Android -> Quicksy/Conversations Especially Siskin has made great progress over the past year. While these clients can do most of the things you would expect from a modern IM client, there are still a few weak spots: - The combination of OMEMO (End-to-end encryption) while using multiple (clients over longer periods of time) can lead to situations where some clients can't decrypt messages. This is rarely dramatic (because one of your devices can decrypt the message), but can be annoying. - It seems video calls across different clients are work-in-progress. Conversations to Conversations works fine, but with Siskin I had some trouble. So the bottom line is: Most things work fine, but some things don't.
- IYasha 4y ago
  Yeah, it's all pretty doable. For me it's: - Linux -> Psi+ (also available for Windows) - Android -> Conversations All FOSS.
Andrew_nenakhov 4y ago
> Everything that we expect today - chat history, reliable message delivery, multi-device synchronization - is an afterthought in XMPP. The author grossly misunderstands XMPP. The first letter X stands for "eXtensible", and extensible it is. Chat history, milti-device sync, reliable message delivery is not an afterthought, it is extension (and btw they all are solved problems as of now) The protocol is by design modular, and it is a strength, not weakness. You can't realistically expect all the world to share the same version of your monolithic messaging protocol. Different instances will support different features, some implementations will lag on with newer features, etc. The only possible answer to this is modularity and legacy fallbacks for some features that are not supported by remote party. Sure, the protocol could develop more rapidly. I actually believe that XMPP survives despite efforts of the XSF coincils, not thanks to them. However, the core of the protocol is strong and allows to build communication products with UX not worse than Telegram or Discord. And if you lack some crucial feature like a properly working group chat, you can make your own XMPP extension - because eXtensible it is.
- rcxdude 4y ago
  In most people's mind extension = optional = afterthought. Especially this is the case if you cannot guarantee that something which supports the protocol actually supports the feature then it's difficult to use that feature. The main point is that the mentioned features are no longer optional for a 'chat protocol', any protocol which does not have them as core or required features is not going to be considered meeting the requirement of 'chat protocol'.
  - Andrew_nenakhov 4y ago
    Most people's mind do not care about protocols, they think of products. And speaking of products, every major xmpp server implementation I know (and I know at least five) supports all these features. Also. You don't really need any of them to send messages between servers. All this stuff matters only for communication between your clients and your own server.
codr7 4y ago
I tried to write a client back in the days. XML is always painful but XMPP brings it to another level by not playing nice with most parsers which expect complete documents, I ended up having to roll my own parser. But perhaps the situation in XML land has improved since, I sincerely hope so for the people writing today's clients. Professionally I've been involved in one project that used XMPP as a data protocol between back end services. It was eventually phased out in favor of REST because of buggy libraries which meant capturing messages in queues and replaying whenever the thing fell over.
bigChris 4y ago

101 points by wusspuss 4y ago

> Is XMPP any good? Sure - here are lots of informations: The chat standard XMPP (IETF): https://www.freie-messenger.de/sys_xmpp https://www.freie-messenger.de/sys_xmpp (german) Thoughts on interoperabiliy: https://www.freie-messenger.de/en/begriffe/interoperabilitaet/gedanken https://www.freie-messenger.de/en/begriffe/interoperabilitae... Quick overview of messenger systems (DE, EN, FR, ES, NL, ...): https://www.freie-messenger.de/en/systemvergleich https://www.freie-messenger.de/en/systemvergleich Because there are so many comments comparing XMPP with Matrix: https://www.freie-messenger.de/en/systemvergleich/xmpp-matrix https://www.freie-messenger.de/en/systemvergleich/xmpp-matri...

MattJ100 4y ago

As someone who has been working with XMPP for a long time, I can safely say that most of the criticisms here simply have no material effects in practice. Yes, an ack packet can be 19 characters long. Is that terrible? It would be just as easy to write a post like this about JSON, Matrix and so on. Both have many warts. But I really don't much care. No technology is perfect, and there will always be people who dislike choices that other people have made. What matters is what we do with the technology. XMPP has proven itself able to evolve over more than two decades, during which the internet and the way we use it has shifted dramatically, and yet it continues to be relevant as new people discover and adopt it. It's unfair to describe things like multi-device sync as "afterthoughts" when XMPP was one of the first real-time messaging protocols to have support for multiple concurrent connections of the same account built in, at a time before the iPhone was released, and most people had a single device with wired access to the internet only. Support for multi-device sync is part of the suite of protocol features required by modern XMPP implementations, these requirements are updated every year to account for the changing technology landscape: https://xmpp.org/about/compliance-suites/ https://xmpp.org/about/compliance-suites/ I personally like the simple delivery model of XMPP compared to the "distributed database" model that Matrix pursues. I think there are use-cases for both approaches. Both XMPP and Matrix have bridges between each other and third-party protocols (proprietary and open ones), which makes any attempt to create false competition between them an unhelpful effort. What's important is that both provide openly-documented protocols with open-source reusable implementations, and (in my opinion) most important of all: decentralized open communication networks. The problems facing the broader adoption of open networks for modern messaging are not protocol problems.

dmitriid 4y ago
> Support for multi-device sync is part of the suite of protocol features required by modern XMPP implementations, these requirements are updated every year The problems is that many of these modern features were unbelievably late. I still remember "The State of Mobile XMPP in 2016", https://gultsch.de/xmpp_2016.html https://gultsch.de/xmpp_2016.html At the time, 9 years after iPhone had come on the scene, most modern features were still experimental draft proposals implemented by no one. 5 more years later there's maybe one client that implements the modern features (Conversations for Android IIRC), and that's about it. And it's an unknown how many servers implement those features.
- seba_dos1 4y ago
  > 5 more years later there's maybe one client that implements the modern features (Conversations for Android IIRC), and that's about it. ...and yet I'm using three different modern clients that work well with each other. > And it's an unknown how many servers implement those features. Enough to make me not have to care about it.
- MattJ100 4y ago
  > And it's an unknown how many servers implement those features It's not: https://compliance.conversations.im/tests/ https://compliance.conversations.im/tests/ Obviously a new user is not expected to fathom what all this means before getting started, which is why there are curated lists of public XMPP services, e.g. https://providers.xmpp.net/ https://providers.xmpp.net/ and https://joinjabber.org/ https://joinjabber.org/ and for private servers there are projects like Snikket where the server software is developed in parallel with apps to ensure a consistent feature set.
ajconway 4y ago
Yet Matrix is gaining traction while XMPP is not. Maybe the key was to have funding, build and run a reference server and quickly iterate with a reference client for all major platforms.
- olah_1 4y ago

grishka 4y ago

The thing with the Nokia 5800 (I also have one) and other phones not designed in the US is that outside of the US, most people paid for SMS per message. So it made sense to present the messages in that UI, you're paying for the whole thing anyway, could as well use all 160 (or 70 around here) characters. No one in their right mind would split into several messages something that fit into one. For this style of instant messaging, we used ICQ. There were unofficial ICQ clients for just about everything. On XMPP itself — I'm looking into it as the protocol for messaging for my federated social network. I first contemplated using Matrix, but then I looked at the spec, saw the mathematical notation for the room state synchronization algorithm, and noped outta there really fast.

Arathorn 4y ago
mental note to link https://matrix.org/docs/guides/implementing-stateres https://matrix.org/docs/guides/implementing-stateres from the matrix spec :p
- grishka 4y ago
  Oh, thanks for this. I did look for a tutorial about implementing a minimally functional Matrix server like there are for ActivityPub[1][2], but didn't find any. [1] https://blog.joinmastodon.org/2018/06/how-to-implement-a-basic-activitypub-server/ https://blog.joinmastodon.org/2018/06/how-to-implement-a-bas... [2] https://blog.joinmastodon.org/2018/07/how-to-make-friends-and-verify-requests/ https://blog.joinmastodon.org/2018/07/how-to-make-friends-an...
grishka 4y ago
And after reading the whole article, I'd like to add: it really helps to treat these protocols like APIs. As in, if you're building an XMPP server, you wouldn't want to expose the actual clients as resources, for example. You'd instead just send each message to each connected client and present them all as one single "client" to other users and for the purpose of s2s federation. That would eliminate an entire class of issues, like messages getting lost, AND improve the user experience at the cost of a slight, barely noticeable spec noncompliance. I do a variation of this for ActivityPub in my project already. Except I don't implement c2s ActivityPub at all.

upofadown 4y ago

How old is this article? It talks about client to server encryption like it is some sort of optional thing. >Today you expect a messenger to be a database. I do? Not all of us want the levels of resource utilization this approach requires. Why would I want to be able to edit/delete old messages? I can just send new ones.

IYasha 4y ago
You've never (fully) used Telegram, have you?

deleted 4y ago

[deleted]

icedchai 4y ago

About 10 years ago, I worked on an XMPP client and experienced many of these issues. Fun times.

buttocks 4y ago

I said this on a past XMPP thread. It is a dead protocol due to mobile. I loved XMPP but without a push system it is useless on modern mobile operating systems.

MattJ100 4y ago
XMPP is widely used on mobile, it has supported push notifications for many years. Without it, you are right, using XMPP on mobile would be pretty painful, but that's not the case.
- bertman 4y ago
  And yet, after all these years, mod_cloud_notify is still a community module and not even mentioned in prosody.cfg.lua.dist.
  - MattJ100 4y ago
    Indeed, mod_cloud_notify is high on our list of modules to merge for the next Prosody release, if not the highest. A lot of effort went into Prosody 0.12, which we released recently. It's unfortunate that mod_cloud_notify didn't quite make the cut for that version, but it was better than delaying the long overdue release even further. The vast majority of work on Prosody is unpaid volunteer time, and as with most open-source work there are certain kinds of tasks which are hard to find funding for, leading to the old "it'll be done when it's done" because we (the developers) simultaneously need to pay our bills. Also, in recognition of many desirable things being in community modules, one thing we did prioritize in 0.12 was the plugin installer, which should make it easier for people to add additional modules to their Prosody installation.
    - bertman 4y ago
      Sorry, my comment probably came off way too snarky. Love Prosody and the work you do with Snikket.
Nasreddin_Hodja 4y ago

kseistrup 4y ago

There alread _was_ an xmpp client in tcl/tk: Coccinella ⌘ https://en.wikipedia.org/wiki/Coccinella_%28software%29 https://en.wikipedia.org/wiki/Coccinella_%28software%29

Nasreddin_Hodja 4y ago
There was Tkabber also.
- kseistrup 4y ago
  Ah, yes!
- libx 4y ago
  Unfortunately, none of them have support for OMEMO.

arendtio 4y ago

I have the impression that the discussion around XMPP isn't going anywhere. On the one side are the people who say it is outdated and can't handle mobile/multi-device/energy-efficient communication and on the other side are the people who argue, that XMPP got extensions to handle all those cases. I am part of the later group. However, instead of arguing I think it might be better to give a short summary about the state of XMPP from a users perpective. I use XMPP every day for several years now. In my opinion the best clients are the following: - Linux -> Gajim (also available for Windows) - iOS -> Siskin - Android -> Quicksy/Conversations Especially Siskin has made great progress over the past year. While these clients can do most of the things you would expect from a modern IM client, there are still a few weak spots: - The combination of OMEMO (End-to-end encryption) while using multiple (clients over longer periods of time) can lead to situations where some clients can't decrypt messages. This is rarely dramatic (because one of your devices can decrypt the message), but can be annoying. - It seems video calls across different clients are work-in-progress. Conversations to Conversations works fine, but with Siskin I had some trouble. So the bottom line is: Most things work fine, but some things don't.

IYasha 4y ago
Yeah, it's all pretty doable. For me it's: - Linux -> Psi+ (also available for Windows) - Android -> Conversations All FOSS.

Andrew_nenakhov 4y ago

> Everything that we expect today - chat history, reliable message delivery, multi-device synchronization - is an afterthought in XMPP. The author grossly misunderstands XMPP. The first letter X stands for "eXtensible", and extensible it is. Chat history, milti-device sync, reliable message delivery is not an afterthought, it is extension (and btw they all are solved problems as of now) The protocol is by design modular, and it is a strength, not weakness. You can't realistically expect all the world to share the same version of your monolithic messaging protocol. Different instances will support different features, some implementations will lag on with newer features, etc. The only possible answer to this is modularity and legacy fallbacks for some features that are not supported by remote party. Sure, the protocol could develop more rapidly. I actually believe that XMPP survives despite efforts of the XSF coincils, not thanks to them. However, the core of the protocol is strong and allows to build communication products with UX not worse than Telegram or Discord. And if you lack some crucial feature like a properly working group chat, you can make your own XMPP extension - because eXtensible it is.

rcxdude 4y ago
In most people's mind extension = optional = afterthought. Especially this is the case if you cannot guarantee that something which supports the protocol actually supports the feature then it's difficult to use that feature. The main point is that the mentioned features are no longer optional for a 'chat protocol', any protocol which does not have them as core or required features is not going to be considered meeting the requirement of 'chat protocol'.
- Andrew_nenakhov 4y ago
  Most people's mind do not care about protocols, they think of products. And speaking of products, every major xmpp server implementation I know (and I know at least five) supports all these features. Also. You don't really need any of them to send messages between servers. All this stuff matters only for communication between your clients and your own server.

codr7 4y ago

I tried to write a client back in the days. XML is always painful but XMPP brings it to another level by not playing nice with most parsers which expect complete documents, I ended up having to roll my own parser. But perhaps the situation in XML land has improved since, I sincerely hope so for the people writing today's clients. Professionally I've been involved in one project that used XMPP as a data protocol between back end services. It was eventually phased out in favor of REST because of buggy libraries which meant capturing messages in queues and replaying whenever the thing fell over.

bigChris 4y ago