6 ms·

What an earnest and well-meaning BS post.... I mean seriously, XMPP is fundamentally flawed, and it's not coming back. What's more, we have matrix which is the

What an earnest and well-meaning BS post.... I mean seriously, XMPP is fundamentally flawed, and it's not coming back. What's more, we have matrix which is the protocol re-written from the ground up with the knowledge of XMPP and better design. Matrix IS XMPP 2.0.

datenarsch 5y ago
> XMPP is fundamentally flawed how is it fundamentally flawed, can you elaborate?
- forty 5y ago
  I really wanted to like XMPP back in the days, but honestly I always ended up feeling that the protocol is just bad. This idea of opening an XML document at the beginning of the stream, then only allowing a subset of XML and all the mess with the xmlns, all that bringing really nothing to the table except complexity. I think ideally in a good protocol, the server should not have to parse the content for the messages that are not targetted to itself (only the metadata useful for routing). the XML mess makes it impossible to do that since you have to validate the full document. At the time I think this page was a good summary of the issues https://about.psyc.eu/XMPP https://about.psyc.eu/XMPP No idea if this is still relevant though.
  - icedchai 5y ago
    I've built an XMPP client for an internal application. My impression was the protocol was overly complex, starting with the lower level problems you describe ("streaming XML.") It's been years since I've looked at. Maybe things are better now.
  - Yoric 5y ago
    Well, I guess it kinda makes sense? Beats having to reinvent a tokenization format from scratch? Of course, smaller messages (à la Matrix) probbably make more sense.
    - jandrese 5y ago
      Ultimately the issue is that XML is a document markup language, not a purpose built streaming protocol. You can kind of force it into that role, but the parser ends up being overly complex and issue prone. It's like basing a chat app around creating a MS Word document of every message and sending it across the network.
  - dwaite 5y ago
    There were several members of the core team, pre-XMPP effort in IETF, who wanted to change it to have framing. If not a length-prefixed model, a nil-separated one. There were ideas to separate out the addressing/routing from the actual messaging, so that servers did not need to process XML and so that messages did not necessarily need to be XML. There were even some very preliminary ideas on using the servers to potentially negotiate peer connections for arbitrary traffic. Back in the early 2000s there were a _lot_ of self-hosted Jabber servers though, and there was push-back from early commercial interests on any protocol-breaking changes resetting adoption to zero. This resulted in Jabber 1.0 pretty much becoming the basis of the XMPP RFC, with XMPP adding new authentication techniques and internationalized JIDs. Later on, there were efforts to establish alternative transports to accomplish some of these alternative transports and forms - HTTP endpoints to poll for messages, JSON mappings of the core messages, etc. I would argue against PSYC's claims (or would have, back in the day) that the usage of XML in XMPP is not proper, however. Its proper, it just wasn't the best idea.
- pohl 5y ago
  I'm not going claim that it's fundamentally flawed, but here's an anecdote. Many years ago, when XML was having its day in the sun, long before it was sidelined by the simplicity of REST and JSON, I was at a Java One convention listening to a speaker present on some new XML parsing API. After the talk, I approached the presenter for some post-talk Q&A to ask how one might use the API to parse the Jabber protocol, which may or may not be relevant to what XMPP is today (I haven't been keeping up.) The presenter was unfamiliar with the protocol, so I had to describe how the xml document was opened when you establish a connection, and how elements keep getting appended to it, and how the "xml document" isn't really completed until you're all done and the connection is terminated. They looked at me like I had two heads. To them, XML didn't make any sense at all unless you have the entire document available all at once. After all, how on earth could one ever apply an XSLT transform to it, right!? Good times.
  - pkulak 5y ago
    That's what Jabber is? Streaming XML? Whoa boy...
  - zmix 5y ago
    > After all, how on earth could one ever apply an XSLT transform to it, right!? There is streaming APIs for XML. Just as XSLT 3.0 can do streaming. Saxon has implemented it, for example[1]. I am aware, that you are talking about the past, but also the XML world moves forward, albeit slowly, since the community has gotten much smaller. [1]: https://www.saxonica.com/html/documentation10/sourcedocs/streaming/xslt-streaming.html https://www.saxonica.com/html/documentation10/sourcedocs/str...
  - dwaite 5y ago
    It was common to have to parse below XML (angle bracket counting) to convert each stanza to an XML element, then parse those as separate XML documents. You'd also have to explicitly turn XML namespace support _off_, since so many systems didn't actually support them. XML defines well-formedness and namespace-well-formedness as two different things, and you didn't want to completely drop communication because the other side was sending a message that didn't meet the more stringent requirement. Some implementations would figure out ways to incrementally disrupt and extract elements from the DOM - but this would sometimes cause resource leaks due to the design of the W3C DOM itself. The expat had explicit support for parsing Jabber/XMPP messages very early, and was by far the most often XML component used for making libraries.
- trasz 5y ago
  I still find it amusing that XMPP by design violates XML spec, thus requiring its own custom XML parser.
  - SamWhited 5y ago
    It uses a subset of the XML spec, it does not need a custom parser, any existing parser will do.
  - dwaite 5y ago
    It does not violate the spec, but it violates the expectations of a lot of XML tooling for sure.
Andrew_nenakhov 5y ago
Matrix is not even a protocol. It is by far inferior to xmpp, especially regarding extendability, and the only flaw in XMPP is a rather lethargic council of elders that are not really interested in its development. This council is too be ignored, the base of the protocol is healthy.
- rapnie 5y ago
  That is interesting. Do you mean that the xmpp.org foundation is not a good custodian of the specs?
  - Andrew_nenakhov 5y ago
    I think they are in some kind of lethargy, waking up each year and updating the compliance suite xeps.
    - tannhaeuser 5y ago
      TBH, (federated) chat/messaging is a solved problem since 1988 (IRC) or 1999 (XMPP), so I don't know what you expect
      - Andrew_nenakhov 5y ago
        - group chats are shit - message formatting is a mess - device management is absent - iOS apps can't really work with stock xeps And the list is long, these are just the top problems. Luckily, this will likely change soon.
        snvzz 5y ago
        >Luckily, this will likely change soon. Please elaborate.
        Andrew_nenakhov 5y ago
        Group chats, nicely working and with fallback for legacy clients. They are usable (without all features, of course) in Gajim/etc without doing anything: Web client: https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/xYmkVFkG/image.png https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f... iOS client: https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/4McZ5fSq/02FD4E41-E09F-45D9-99BE-E40BAA03B179.png https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f... Message formatting (from web client, but is supported by all, actually): https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/SHNoo5ht/image.png https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f... Device management. Newly connected device is given a token, with a mechanism to prevent token duplication, and tokens can be revoked, resulting in device disconnection: https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/4stcWlA4/1A7A8E71-3A05-4E20-BDB7-F1E53C44AA4E.png https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f... https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/ggMyGWlA/3B814FC7-5C20-489C-AC6B-854CE5BD95FA.png https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f... This all is a work in progress, and we're aiming for an iOS version release in december or early next year.
NoGravitas 5y ago
I use both XMPP and Matrix regularly, and run servers for both. I wouldn't say Matrix is better than XMPP across the board, but that they have different trade-offs. The main architectural difference between them is that Matrix provides "eventually-consistent cryptographically secure synchronisation of room state across a global open network of federated servers and services", whereas XMPP just passes messages from point to point (where one of the endpoints may be a multiuser room). The main practical benefit of Matrix is that multi-user rooms don't have a "home" server; they're distributed across the homeservers of every member of the room. This means that though everyone may have signed up for a room identified as #shitposting:matrix.org, if the matrix.org homeserver goes down, the chat still works. People can't join it until a new name is published for it, but it works for everyone already in it. In XMPP, chats are hosted on a particular server, and if the server a chat is on goes down, the chat goes down. The main practical benefit of XMPP is that it is much more lightweight than Matrix. Being based on synchronization of room state means that Matrix stores a lot of data, generally all messages and attachments back to when the room was created (or possibly the first time someone on your homeserver joined the room). Apparently it's possible to prune room history, but it's not done by default, and as far as I know, it's not officially supported. It's much easier to control how much data your XMPP server stores. XMPP is also a simpler protocol, which means there is a wider variety of clients; while there are several vaguely viable Matrix clients, if you're not using Element, you are much more likely to have problems, especially with encrypted rooms. Of course, the flip side of this is that a lot of XMPP clients don't support all the extensions which make modern XMPP useful, either. Both of them have trouble with multi-device E2EE key management, though Matrix has the edge. But again, if you're not using Element, you are likely to have problems.
- zaik 5y ago
  > whereas XMPP just passes messages from point to point So does Matrix. All the other qualities are build on top of that. XMPP also enables e2ee, message synchronisation and an "global open network of federated servers and services".
  - dwaite 5y ago
    I'll say that we did have efforts early (pre 1.0 Jabber) to do leaderless communication and build rooms on top of it. Back around 2000 getting information on building eventually consistent systems was much harder than it is today, and leaderless versions of that doubly so. Those who worked on it felt like they were inventing half the techniques themselves, had difficulty recruiting (teaching) new people, and eventually burnt out. If those involved had gotten it to work, it would have just been core infrastructure for eventually consistent ordered events, with UX concepts like "rooms" built on top as common business logic. Or in other words, distributed apps.
durandal1 5y ago
It'd take the flawed XMPP protocol any day over the ad-hoc implementation of Matrix, which hardly even deserves to be called a protocol. Matrix feels like something made by making things up as the wrote the code, there is no coherency at all.
- jakecopp 5y ago
  > ad-hoc implementation of Matrix, which hardly even deserves to be called a protocol. Please explain why you believe this. This is the spec: https://spec.matrix.org/latest/ https://spec.matrix.org/latest/ These are the spec change proposals: https://spec.matrix.org/unstable/proposals/ https://spec.matrix.org/unstable/proposals/
- dwaite 5y ago
  Jabber was written by making things up as we wrote the code, and then pitching a prototype to a core group for revisions. XMPP didn't really pitch any client or server protocol breaking changes. Since the messages themselves were just addressed XML elements, we were able to do revisions over time, such as the first group chat being replaced by MUC or adding in the ability for XHTML-formatted rich text.

datenarsch 5y ago

> XMPP is fundamentally flawed how is it fundamentally flawed, can you elaborate?

forty 5y ago
I really wanted to like XMPP back in the days, but honestly I always ended up feeling that the protocol is just bad. This idea of opening an XML document at the beginning of the stream, then only allowing a subset of XML and all the mess with the xmlns, all that bringing really nothing to the table except complexity. I think ideally in a good protocol, the server should not have to parse the content for the messages that are not targetted to itself (only the metadata useful for routing). the XML mess makes it impossible to do that since you have to validate the full document. At the time I think this page was a good summary of the issues https://about.psyc.eu/XMPP https://about.psyc.eu/XMPP No idea if this is still relevant though.
- icedchai 5y ago
  I've built an XMPP client for an internal application. My impression was the protocol was overly complex, starting with the lower level problems you describe ("streaming XML.") It's been years since I've looked at. Maybe things are better now.
- Yoric 5y ago
  Well, I guess it kinda makes sense? Beats having to reinvent a tokenization format from scratch? Of course, smaller messages (à la Matrix) probbably make more sense.
  - jandrese 5y ago
    Ultimately the issue is that XML is a document markup language, not a purpose built streaming protocol. You can kind of force it into that role, but the parser ends up being overly complex and issue prone. It's like basing a chat app around creating a MS Word document of every message and sending it across the network.
- dwaite 5y ago
  There were several members of the core team, pre-XMPP effort in IETF, who wanted to change it to have framing. If not a length-prefixed model, a nil-separated one. There were ideas to separate out the addressing/routing from the actual messaging, so that servers did not need to process XML and so that messages did not necessarily need to be XML. There were even some very preliminary ideas on using the servers to potentially negotiate peer connections for arbitrary traffic. Back in the early 2000s there were a _lot_ of self-hosted Jabber servers though, and there was push-back from early commercial interests on any protocol-breaking changes resetting adoption to zero. This resulted in Jabber 1.0 pretty much becoming the basis of the XMPP RFC, with XMPP adding new authentication techniques and internationalized JIDs. Later on, there were efforts to establish alternative transports to accomplish some of these alternative transports and forms - HTTP endpoints to poll for messages, JSON mappings of the core messages, etc. I would argue against PSYC's claims (or would have, back in the day) that the usage of XML in XMPP is not proper, however. Its proper, it just wasn't the best idea.
pohl 5y ago
I'm not going claim that it's fundamentally flawed, but here's an anecdote. Many years ago, when XML was having its day in the sun, long before it was sidelined by the simplicity of REST and JSON, I was at a Java One convention listening to a speaker present on some new XML parsing API. After the talk, I approached the presenter for some post-talk Q&A to ask how one might use the API to parse the Jabber protocol, which may or may not be relevant to what XMPP is today (I haven't been keeping up.) The presenter was unfamiliar with the protocol, so I had to describe how the xml document was opened when you establish a connection, and how elements keep getting appended to it, and how the "xml document" isn't really completed until you're all done and the connection is terminated. They looked at me like I had two heads. To them, XML didn't make any sense at all unless you have the entire document available all at once. After all, how on earth could one ever apply an XSLT transform to it, right!? Good times.
- pkulak 5y ago
  That's what Jabber is? Streaming XML? Whoa boy...
- zmix 5y ago
  > After all, how on earth could one ever apply an XSLT transform to it, right!? There is streaming APIs for XML. Just as XSLT 3.0 can do streaming. Saxon has implemented it, for example[1]. I am aware, that you are talking about the past, but also the XML world moves forward, albeit slowly, since the community has gotten much smaller. [1]: https://www.saxonica.com/html/documentation10/sourcedocs/streaming/xslt-streaming.html https://www.saxonica.com/html/documentation10/sourcedocs/str...
- dwaite 5y ago
  It was common to have to parse below XML (angle bracket counting) to convert each stanza to an XML element, then parse those as separate XML documents. You'd also have to explicitly turn XML namespace support _off_, since so many systems didn't actually support them. XML defines well-formedness and namespace-well-formedness as two different things, and you didn't want to completely drop communication because the other side was sending a message that didn't meet the more stringent requirement. Some implementations would figure out ways to incrementally disrupt and extract elements from the DOM - but this would sometimes cause resource leaks due to the design of the W3C DOM itself. The expat had explicit support for parsing Jabber/XMPP messages very early, and was by far the most often XML component used for making libraries.
trasz 5y ago
I still find it amusing that XMPP by design violates XML spec, thus requiring its own custom XML parser.
- SamWhited 5y ago
  It uses a subset of the XML spec, it does not need a custom parser, any existing parser will do.
- dwaite 5y ago
  It does not violate the spec, but it violates the expectations of a lot of XML tooling for sure.

Andrew_nenakhov 5y ago

Matrix is not even a protocol. It is by far inferior to xmpp, especially regarding extendability, and the only flaw in XMPP is a rather lethargic council of elders that are not really interested in its development. This council is too be ignored, the base of the protocol is healthy.

rapnie 5y ago
That is interesting. Do you mean that the xmpp.org foundation is not a good custodian of the specs?
- Andrew_nenakhov 5y ago
  I think they are in some kind of lethargy, waking up each year and updating the compliance suite xeps.
  - tannhaeuser 5y ago
    TBH, (federated) chat/messaging is a solved problem since 1988 (IRC) or 1999 (XMPP), so I don't know what you expect
    - Andrew_nenakhov 5y ago
      - group chats are shit - message formatting is a mess - device management is absent - iOS apps can't really work with stock xeps And the list is long, these are just the top problems. Luckily, this will likely change soon.
      - snvzz 5y ago
        >Luckily, this will likely change soon. Please elaborate.
        Andrew_nenakhov 5y ago
        Group chats, nicely working and with fallback for legacy clients. They are usable (without all features, of course) in Gajim/etc without doing anything: Web client: https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/xYmkVFkG/image.png https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f... iOS client: https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/4McZ5fSq/02FD4E41-E09F-45D9-99BE-E40BAA03B179.png https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f... Message formatting (from web client, but is supported by all, actually): https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/SHNoo5ht/image.png https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f... Device management. Newly connected device is given a token, with a mechanism to prevent token duplication, and tokens can be revoked, resulting in device disconnection: https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/4stcWlA4/1A7A8E71-3A05-4E20-BDB7-F1E53C44AA4E.png https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f... https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/ggMyGWlA/3B814FC7-5C20-489C-AC6B-854CE5BD95FA.png https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f... This all is a work in progress, and we're aiming for an iOS version release in december or early next year.

NoGravitas 5y ago

I use both XMPP and Matrix regularly, and run servers for both. I wouldn't say Matrix is better than XMPP across the board, but that they have different trade-offs. The main architectural difference between them is that Matrix provides "eventually-consistent cryptographically secure synchronisation of room state across a global open network of federated servers and services", whereas XMPP just passes messages from point to point (where one of the endpoints may be a multiuser room). The main practical benefit of Matrix is that multi-user rooms don't have a "home" server; they're distributed across the homeservers of every member of the room. This means that though everyone may have signed up for a room identified as #shitposting:matrix.org, if the matrix.org homeserver goes down, the chat still works. People can't join it until a new name is published for it, but it works for everyone already in it. In XMPP, chats are hosted on a particular server, and if the server a chat is on goes down, the chat goes down. The main practical benefit of XMPP is that it is much more lightweight than Matrix. Being based on synchronization of room state means that Matrix stores a lot of data, generally all messages and attachments back to when the room was created (or possibly the first time someone on your homeserver joined the room). Apparently it's possible to prune room history, but it's not done by default, and as far as I know, it's not officially supported. It's much easier to control how much data your XMPP server stores. XMPP is also a simpler protocol, which means there is a wider variety of clients; while there are several vaguely viable Matrix clients, if you're not using Element, you are much more likely to have problems, especially with encrypted rooms. Of course, the flip side of this is that a lot of XMPP clients don't support all the extensions which make modern XMPP useful, either. Both of them have trouble with multi-device E2EE key management, though Matrix has the edge. But again, if you're not using Element, you are likely to have problems.

zaik 5y ago
> whereas XMPP just passes messages from point to point So does Matrix. All the other qualities are build on top of that. XMPP also enables e2ee, message synchronisation and an "global open network of federated servers and services".
- dwaite 5y ago
  I'll say that we did have efforts early (pre 1.0 Jabber) to do leaderless communication and build rooms on top of it. Back around 2000 getting information on building eventually consistent systems was much harder than it is today, and leaderless versions of that doubly so. Those who worked on it felt like they were inventing half the techniques themselves, had difficulty recruiting (teaching) new people, and eventually burnt out. If those involved had gotten it to work, it would have just been core infrastructure for eventually consistent ordered events, with UX concepts like "rooms" built on top as common business logic. Or in other words, distributed apps.

durandal1 5y ago

It'd take the flawed XMPP protocol any day over the ad-hoc implementation of Matrix, which hardly even deserves to be called a protocol. Matrix feels like something made by making things up as the wrote the code, there is no coherency at all.

jakecopp 5y ago
> ad-hoc implementation of Matrix, which hardly even deserves to be called a protocol. Please explain why you believe this. This is the spec: https://spec.matrix.org/latest/ https://spec.matrix.org/latest/ These are the spec change proposals: https://spec.matrix.org/unstable/proposals/ https://spec.matrix.org/unstable/proposals/
dwaite 5y ago
Jabber was written by making things up as we wrote the code, and then pitching a prototype to a core group for revisions. XMPP didn't really pitch any client or server protocol breaking changes. Since the messages themselves were just addressed XML elements, we were able to do revisions over time, such as the first group chat being replaced by MUC or adding in the ability for XHTML-formatted rich text.