5 ms·

The thing is... with XMPP, it's things that are really, utterly critical like the concept of messages having IDs that are left in the "extensions". You can't b

The thing is... with XMPP, it's things that are really, utterly critical like the concept of messages having IDs that are left in the "extensions". You can't build a sane protocol when something like "IDs" are a complete fend-for-yourself wilderness. I've written about this previously: https://news.ycombinator.com/item?id=12908619 https://news.ycombinator.com/item?id=12908619 Long story short, if you ever try to implement something in XMPP/XEPs, you'll learn the hard way why it never seems to gain any traction. Very, very quickly. There are real technical reasons. Extensibility is good. I believe the Matrix developers when they say it's extensible. (I've been a user for several years now. Features regularly get added in backwards-compatible and smooth ways.) Extensibility in the way XMPP tried to go about it -- for patching over core inadequacies of the basic protocol -- is not the same beast.

NoGravitas 8y ago
What would really help with XMPP is some idea of profiles - like a Jabber2018 profile that requires all of the XEPs used by Conversations[0]. That way, client and server implementers would know exactly what they need to be compliant with other implementations using the same profile. [0]: https://conversations.im/ https://conversations.im/
- MattJ100 8y ago
  That exists. Here are the 2018 profiles (they are split into basic/advanced): https://xmpp.org/extensions/xep-0387.html https://xmpp.org/extensions/xep-0387.html They are only part of the solution however because: - Open-source projects are developed by volunteers, and implement what their users (actually typically their developers) want implemented. Having a document that says "you should implement these things" is good for guidance, but it doesn't magically get them implemented. - Commercial projects will implement the things that they can get revenue from. Having a document that says "you should implement these things" doesn't magically make those things earn the company money. E.g. in the open-source category, Pidgin is the classic example of a once-popular client that has fallen behind (with feature requests for some "new" protocol features open on their bug tracker for many many years). The document doesn't fix this part. But yes, I do agree it's an essential part of solving the problem.
  - dnel 8y ago
    It's still so common to see clients tout a list of supported XEPs like they expect users to know what any of it means. It's great there is a list of importantant XEPs but this must be communicated to users better so that it's easier to find suitable clients. It might even be good to refer to compliant clients as something other than Jabber/XMPP then you can easily search for an "XTalk" (for example) client and know this means the full list of XEPs are supported. Part of this might include some new XEPs to indicate to users when they are communicating with non-XTalk users so the experience might differ. A certification programme even might help motivate devs to meet compliance quickly. One can dream anyway.
dnel 8y ago
I doubt any platform uses vanilla XMPP, that wouldn't be sane, but like using XML, XMPP is a relic of its era. Whose to say what super-critical "how can you not have this as standard!?!" feature won't come up in the next 5 years that Matrix devs hadn't even known of in 2018. That doesn't mean we should flip over the table and start again every now and again and hope the users will follow, this is why extensibility is an important part of the recipe and make ways to ensure the standards of the day are always implemented before being considered ready for prime time.
- heavenlyhash 8y ago
  IDs, though. Like, in the abstract, I agree with you completely. But IDs.
  - rakoo 8y ago
    I don't know, IRC in its current form doesn't have any message-ids and yet seems to be working fine.
    - heavenlyhash 8y ago
      It also does not and never did support consistent history or multi-device. And it never will, because having some way to identify a message is a prerequisite for even phrasing the question "does device_foo have message_x synced yet?" Bouncers don't count. They reintroduce centralization as a way to solve the problem. It's a solution, of a sort, but it's a fairly poor one if your original aim was decentralization. And "working fine" is also frankly a bit of a stretch, isn't it? I can't count how many times I've seen a "netsplit" in freenode or the other IRC networks. That means messages aren't delivered. It's so common we have a word for it: "netsplit". I shouldn't dunk on IRC too much, because at least it's honest about itself for being simple, and there's a virtue in that. But if we were to construct a statement like "IRC manages reliable message delivery without any concept of message IDs", then that would be false (or technically, vacuous), because it doesn't manage reliable message delivery.
seba_dos1 8y ago
I don't think you get the idea of extensions in XMPP. If something is published as a XEP, it doesn't mean "it's a completely optional thing you can ignore". It does mean "it's a thing that might not be useful in some cases, so it doesn't belong to the core that everyone and their pets must implement". If you build an IM client for XMPP (which is just one of the use cases of XMPP), you absolutely have to implement at least a few of those extensions in order to have basic functionality. Heck, there's even a separate RFC for XMPP-IM on top of the XMPP-Core RFC. That's expected, by design and very fine. Of course, what's considered "basic" changes over time and it's easy to find outdated clients out there. That's why Compliance Profiles exist, so you can look around for clients that implement, for instance, 2018 desktop IM profile. It also imposes some burden on new implementations to consider backwards compatibility with those clients. No solution is perfect, but while I think Matrix' way allows them to move forward faster now, in the long term it's probably XMPP that will age more gracefully.
- heavenlyhash 8y ago
  > I don't think you get the idea of extensions in XMPP I don't think you read the other comments I linked to. My fault for not copy-pasting the entire thing every time the "y u not be exactly XMPP" train comes on HN again... --- > - unique message IDs? Absent. XEPs kind of provide; but I can't tell you which of the three or for relevant ones are the most relevant (AMP IDs from XEP 0079? Stream Management from XEP 0198? Acking from XEP 0184? Something from Carbons or MAM in 0313 or 0280? You know, if you wanted some light reading...). > - multi device? Oh. My. God. It's bananas. The spec behavior is that whenever a client sends a message, the server is supposed to consider that one the most alive, and then route all future messages exclusively to that one. So you send a message on your phone? Yeah, your desktop is just going to silently stop receiving messages. > - there's a concept of "message carbons" to deal with this. This involves re-sending all your messages back to the server after you receive them, with special instructions to send them back again to your other clients. The amount of redundantly redundant XML involved is eyewatering. > Combine that multidevice behavior (messages can get randomly routed anywhere at any time) with the wild-west nature of message delivery acks, and you can see how ridiculously difficult this makes the basic idea of "all clients should see the same picture". > Overall, the XEP process, conceptually, is a great example of open extensibility. The trouble is, so much of this stuff is core to sane message delivery semantics that it really, practically speaking, causes huge problems when it's all considered "extensions". Stuff like message IDs fundamentally shouldn't be an extension because it's just too critical that all minimum-viable clients agree. You just can't build higher level stuff without that. XEPs are great. A community process for extensions should exist. It just needs to exist for extensions, not subsume the total set of realistic minimum viable features. (All of this is a shameless repost from https://news.ycombinator.com/item?id=9772968 https://news.ycombinator.com/item?id=9772968 ; please forgive my humble attempt to save the dear reader from the need to traverse any links.) --- I get the idea of extensions in XMPP. I have tried to implement them. It is based on this first-hand experience that I am now arguing that the XEPs are a swamp. It is time to move on. Yes, even with Compliance Profiles. It's too late. The horses have left the barn, developed tools, formed their own complex civilizations, invented light speed travel, and in fact left the galaxy. That's how far out of the barn they are. Message ID and multi-device messaging is in fact fundamentally broken in such a unique, fascinating way in XMPP that all other clients will cease to operate correctly in the presence of a single bad client. That's right. Any bad client can cause other clients which are following the spec to the letter to begin to drop messages, forward them incorrectly, and generally lose history and form desynchronized views of the world. I'm not kidding. If you haven't read the XEPs, and don't believe me, go read the XEPs. The only way to implement a reasonably correct multi-device behavior requires ignoring significant parts of XMPP because it is wrong. Start. Over. XMPP is not your savior.

NoGravitas 8y ago

What would really help with XMPP is some idea of profiles - like a Jabber2018 profile that requires all of the XEPs used by Conversations[0]. That way, client and server implementers would know exactly what they need to be compliant with other implementations using the same profile. [0]: https://conversations.im/ https://conversations.im/

MattJ100 8y ago
That exists. Here are the 2018 profiles (they are split into basic/advanced): https://xmpp.org/extensions/xep-0387.html https://xmpp.org/extensions/xep-0387.html They are only part of the solution however because: - Open-source projects are developed by volunteers, and implement what their users (actually typically their developers) want implemented. Having a document that says "you should implement these things" is good for guidance, but it doesn't magically get them implemented. - Commercial projects will implement the things that they can get revenue from. Having a document that says "you should implement these things" doesn't magically make those things earn the company money. E.g. in the open-source category, Pidgin is the classic example of a once-popular client that has fallen behind (with feature requests for some "new" protocol features open on their bug tracker for many many years). The document doesn't fix this part. But yes, I do agree it's an essential part of solving the problem.
- dnel 8y ago
  It's still so common to see clients tout a list of supported XEPs like they expect users to know what any of it means. It's great there is a list of importantant XEPs but this must be communicated to users better so that it's easier to find suitable clients. It might even be good to refer to compliant clients as something other than Jabber/XMPP then you can easily search for an "XTalk" (for example) client and know this means the full list of XEPs are supported. Part of this might include some new XEPs to indicate to users when they are communicating with non-XTalk users so the experience might differ. A certification programme even might help motivate devs to meet compliance quickly. One can dream anyway.

dnel 8y ago

I doubt any platform uses vanilla XMPP, that wouldn't be sane, but like using XML, XMPP is a relic of its era. Whose to say what super-critical "how can you not have this as standard!?!" feature won't come up in the next 5 years that Matrix devs hadn't even known of in 2018. That doesn't mean we should flip over the table and start again every now and again and hope the users will follow, this is why extensibility is an important part of the recipe and make ways to ensure the standards of the day are always implemented before being considered ready for prime time.

heavenlyhash 8y ago
IDs, though. Like, in the abstract, I agree with you completely. But IDs.
- rakoo 8y ago
  I don't know, IRC in its current form doesn't have any message-ids and yet seems to be working fine.
  - heavenlyhash 8y ago
    It also does not and never did support consistent history or multi-device. And it never will, because having some way to identify a message is a prerequisite for even phrasing the question "does device_foo have message_x synced yet?" Bouncers don't count. They reintroduce centralization as a way to solve the problem. It's a solution, of a sort, but it's a fairly poor one if your original aim was decentralization. And "working fine" is also frankly a bit of a stretch, isn't it? I can't count how many times I've seen a "netsplit" in freenode or the other IRC networks. That means messages aren't delivered. It's so common we have a word for it: "netsplit". I shouldn't dunk on IRC too much, because at least it's honest about itself for being simple, and there's a virtue in that. But if we were to construct a statement like "IRC manages reliable message delivery without any concept of message IDs", then that would be false (or technically, vacuous), because it doesn't manage reliable message delivery.

seba_dos1 8y ago

I don't think you get the idea of extensions in XMPP. If something is published as a XEP, it doesn't mean "it's a completely optional thing you can ignore". It does mean "it's a thing that might not be useful in some cases, so it doesn't belong to the core that everyone and their pets must implement". If you build an IM client for XMPP (which is just one of the use cases of XMPP), you absolutely have to implement at least a few of those extensions in order to have basic functionality. Heck, there's even a separate RFC for XMPP-IM on top of the XMPP-Core RFC. That's expected, by design and very fine. Of course, what's considered "basic" changes over time and it's easy to find outdated clients out there. That's why Compliance Profiles exist, so you can look around for clients that implement, for instance, 2018 desktop IM profile. It also imposes some burden on new implementations to consider backwards compatibility with those clients. No solution is perfect, but while I think Matrix' way allows them to move forward faster now, in the long term it's probably XMPP that will age more gracefully.

heavenlyhash 8y ago
> I don't think you get the idea of extensions in XMPP I don't think you read the other comments I linked to. My fault for not copy-pasting the entire thing every time the "y u not be exactly XMPP" train comes on HN again... --- > - unique message IDs? Absent. XEPs kind of provide; but I can't tell you which of the three or for relevant ones are the most relevant (AMP IDs from XEP 0079? Stream Management from XEP 0198? Acking from XEP 0184? Something from Carbons or MAM in 0313 or 0280? You know, if you wanted some light reading...). > - multi device? Oh. My. God. It's bananas. The spec behavior is that whenever a client sends a message, the server is supposed to consider that one the most alive, and then route all future messages exclusively to that one. So you send a message on your phone? Yeah, your desktop is just going to silently stop receiving messages. > - there's a concept of "message carbons" to deal with this. This involves re-sending all your messages back to the server after you receive them, with special instructions to send them back again to your other clients. The amount of redundantly redundant XML involved is eyewatering. > Combine that multidevice behavior (messages can get randomly routed anywhere at any time) with the wild-west nature of message delivery acks, and you can see how ridiculously difficult this makes the basic idea of "all clients should see the same picture". > Overall, the XEP process, conceptually, is a great example of open extensibility. The trouble is, so much of this stuff is core to sane message delivery semantics that it really, practically speaking, causes huge problems when it's all considered "extensions". Stuff like message IDs fundamentally shouldn't be an extension because it's just too critical that all minimum-viable clients agree. You just can't build higher level stuff without that. XEPs are great. A community process for extensions should exist. It just needs to exist for extensions, not subsume the total set of realistic minimum viable features. (All of this is a shameless repost from https://news.ycombinator.com/item?id=9772968 https://news.ycombinator.com/item?id=9772968 ; please forgive my humble attempt to save the dear reader from the need to traverse any links.) --- I get the idea of extensions in XMPP. I have tried to implement them. It is based on this first-hand experience that I am now arguing that the XEPs are a swamp. It is time to move on. Yes, even with Compliance Profiles. It's too late. The horses have left the barn, developed tools, formed their own complex civilizations, invented light speed travel, and in fact left the galaxy. That's how far out of the barn they are. Message ID and multi-device messaging is in fact fundamentally broken in such a unique, fascinating way in XMPP that all other clients will cease to operate correctly in the presence of a single bad client. That's right. Any bad client can cause other clients which are following the spec to the letter to begin to drop messages, forward them incorrectly, and generally lose history and form desynchronized views of the world. I'm not kidding. If you haven't read the XEPs, and don't believe me, go read the XEPs. The only way to implement a reasonably correct multi-device behavior requires ignoring significant parts of XMPP because it is wrong. Start. Over. XMPP is not your savior.