6 ms·

Ask HN: What should replace XMPP?

75 points by tylorr 11y ago

Let's assume for the sake of discussion that XMPP has died or is at least on its last legs. What could we learn from XMPP that would help us find its replacement? There are a few key questions that need answering. What did XMPP do right and could be copied? Why did so many big players ditch XMPP? What would it take to get big players to adopt it? What are the existing resources for creating protocols similar to XMPP? Is it possible to simplify a chat protocol or is it complex in nature? I briefly tried working on an existing XMPP server implementation but found the protocol and all the extension overbearing and very hard to grok.

detaro 11y ago
Encryption would be a massive thing to figure out. More precisely, end-to-end encryption with proper multi-device support. That's also something all the other established players haven't solved, so it would be a great advantage to get users.
lwf 11y ago
https://matrix.org/ https://matrix.org/ — Matrix is increasingly looking like the only sensible player; they get so many things right: * Federated * Fault-tolerant * Cryptographic attestations of messages
- giancarlostoro 11y ago
  This is the only other alternative to date I've found that gets those things which I want. It's quite amazing all they're trying to accomplish as it is.
- Perceptes 11y ago
  I'm a fan of Matrix.org and very slowly working on an implementation of the client-server API in Rust[1]. In the process of doing so I'm also doing my best to help them improve and clarify the spec. I'd love to see nice native clients for Matrix implemented for the various major platforms. Right now there is vector.im, but I'm personally not a fan of using a web browser for applications I leave running all the time, like chat. [1] https://github.com/ruma https://github.com/ruma
daurnimator 11y ago
What XMPP did right: - Any chat protocol must be extendable - Federation is good What could have been better - Regular users don't want to run their own server - Overcomplicated specs with optional bits (MUC, Pubsub) - Encryption should be mandatory from the start - Logging/history is important Other lessons - Device sync is important - If you have multiple devices or clients; commit log style is best (see new MIX spec vs MUC spec). - In the modern world, people are afraid of XML - 95% of people still don't understand XML namespaces ------------------- But XMPP isn't dead (and doesn't have to be). New interesting XEPs are being working on: - MAM - MIX aka MUC2 - OMEMO
- tylorr 11y ago
  I really don't want XMPP to die, but I'm worried that it can only go down hill after the big players ditched it. Is it possible for XMPP to make a major come back without their support? Would big players ever come back or are they too concerned with walled gardens?
- vpkaihla 11y ago
  - In the modern world, people are afraid of XML - 95% of people still don't understand XML namespaces I guess that's one way of saying that something is too complicated :)
  - hvidgaard 11y ago
    It isn't much different from namespaces in programming languages, which most developers do not have a problem understanding. They are in XML to avoid naming clash, and they do that well. I haven't really seen a different mechanic to solve that problem.
    - daurnimator 11y ago
      I think the problem with xml namespaces enters in a few ways: - Non XML namespace supporting parsers: these are xml parsers people throw together without paying attention to the fact that attributes starting with "xml" are special and *must* be rejected. - This results in people trying to shim namespaces in after the fact; which rarely goes well. - There are two syntaxes: `<foo xmlns="urn:example"/>` vs `<myns:foo xmlns:myns="urn:example"/>`. Most people get caught up in the latter without understanding it, and you end up with *lots* of implementations that depend on `myns`, which is meant to be arbitary and can change between serialisation/deserialisation - The W3C style for namespace inheritence forces each element to keep a reference to it's parent: a nightmare for simple data structures.
qrmn 11y ago
XML was probably a mistake. Strong, deniable, end-to-end encryption should be mandatory. The Axolotl ratchet is the current state-of-the-art: maybe it does asymmetric things we don't need, or maybe that's helpful. Looking forward: metadata protection. This is a much more difficult-to-solve problem, but existing tools such as Tor are partially successful.
- andrewflnr 11y ago
  It you need that, why wouldn't you just tunnel your new protocol over Tor? Edit: metadata protection, I mean. I'm not disputing the need for encryption.
- JoshTriplett 11y ago
  > Strong, deniable, end-to-end encryption should be mandatory. Strong end-to-end encryption with perfect forward secrecy should be mandatory. Deniable authentication (https://en.wikipedia.org/wiki/Deniable_authentication https://en.wikipedia.org/wiki/Deniable_authentication), however, seems like a potentially interesting option but not one that the protocol should mandate. Sometimes you do want authentication that remains valid after the conversation ends, so you can subsequently authenticate the messages in it.
- lambau 11y ago
  Pond uses Tor for metadata protection. Vuvuzela is another recent messenger that protects metadata.
deleted 11y ago
[deleted]
maximmilius_rob 11y ago
SIP?
- zamalek 11y ago
  The IM protocol that runs over SIP is called SIMPLE and it's anything but.
Todd 11y ago
As you know, there's a lot of good in XMPP. The part that I think causes the most difficulty is that it's based on XML. I don't mean this in the typical knee-jerk XML bashing sense. It's just that XML is ill suited for a stream-based protocol. XML is amenable to extensibility, with its innate namespacing features. But this also made it difficult for beginners to use well and was easy to get wrong. The XML decision was really just a consequence of the times in which it was developed. It was a natural choice. If it were done again, it would be REST-based and use JSON as the data format.
- tylorr 11y ago
  Would it be worth it to try to replace XMPP with something very similar but using a JSON based protocol and clean up its worts in the process?
  - Todd 11y ago
    I absolutely think it would be worth it. The data format change would be one thing. There are many other factors which are worth considering. For example, the long lived connections of XMPP don't work well in the mobile environment. There are XEPs which address things like this. I think to do it right, you have to build a company around it, and build it in the open. Also, no one is going to jump on a protocol just because it's open. It has to be built around a service which is compelling in its own right, and gains user traction regardless of the protocol. Geeks can get it going, but it has to be self sustaining.
  - estefan 11y ago
    I'm using JSON for my new socketIO server. You can put anything on the wire with socketIO, so messages are small and simple. Plus the clients seem really good too. They'll auto reconnect, etc. if the server goes away. I haven't tested whether they'll queue messages while offline yet... guess I should. I'd definitely recommend it. This is a problem I've had multiple stabs at, but I finally feel like I've got a long-term solution now. But yeah, no federation, etc. My need is single-site realtime chat.
  - rakoo 11y ago
ksec 11y ago
May not be 100% replacement, but JMAP ?
JoshTriplett 11y ago
Can any of the new secure-text-messaging protocols, such as the one used by Signal, work in a federated manner? Many of those protocols currently use a central server; which ones actually require that, and which ones don't?
pmlnr 11y ago
IRC - because dead simple & extendable Tox - because fully decentralized & encrypted SIP - because rock solid for voice & video Chat & IM is not the web. It shouldn't be running on HTTP with HTTP APIs.
- jgh 11y ago
  SIP is a signaling protocol (and XMPP can also be used that way). It doesn't really have much to do with voice & video other than it gets used to initialize a connection. Also SIP is horrible.
  - deleted 11y ago
    [deleted]
sbeckeriv 11y ago
BEEP on TCP - RFC 3081 Any day now... https://en.wikipedia.org/wiki/BEEP https://en.wikipedia.org/wiki/BEEP
moonshinefe 11y ago
It's of course possible to simplify a chat protocol. For instance, I'm not sure why the XMPP designers decided to go with XML; it's much more verbose than most formats, and typically slower and more complicated to parse in my experience say versus JSON. What advantage did it have? Unless someone is sniffing unencrypted XMPP packets and finds that XML readability handy, I don't really see the point. And that's an edge case. Not to mention XML's main advantage over many other formats is having hierarchical relationships--but how does that help in a chat protocol? I'm sure someone could very well have counter points to the above, but those are the things off the top of my head that made me question why XML was the top choice here for an IM protocol.
evilotto 11y ago
I was under the impression that XMPP was alive and well, it's federation that's dead. Because as it turns out, the big players aren't really interested in interoperability, they want their own walled gardens because that's where you can actually make money.
babebridou 11y ago
Putting the format and security aside, isn't the main issue that XMPP was so good that we tried to bite more than it could chew? - Local Federation is great, but Global Federation doesn't benefit the big players - Extensibility is great for domain-specific problems, but it leads to more fragmentation - It tried to handle data streaming, but it's inherently suboptimal at it I mean, it was a great protocol to explore the IM design space, but it was bound to be replaced once we clarified our needs (for the most part, multiple devices and media streaming). As a side note, I often get showerthoughts about mixing IM and blockchain techs. I'm not sure where that would lead us.
estefan 11y ago
A few weeks ago I replaced a XMPP chat architecture with socketio. It's so much better for my needs (website with realtime chat). There were 2 main reasons in the end: - No way to log all messages sent to a database so that users could retrieve them on subsequent logins - No way to edit/delete messages, especially in multi-user chats. Maybe these things are possible, but I couldn't find how to do it. Now, with socket IO on nodejs, I have a < 400 line chat server with DB integration that is much more suitable for my needs. I guess XMPP's focus was too narrow.
- daurnimator 11y ago
  > - No way to log all messages sent to a database so that users could retrieve them on subsequent logins This is what MAM does.
  - estefan 11y ago
    Perhaps the ejabberd support is bad or poorly documented? Either way I just couldn't find how to do it. All I could find for MAM was logging to flat files.
    - rakoo 11y ago
      This post [1] seems to indicate that ejabberd's support of MAM is complete. Note that you also (obviously) need support from the client, which most but not all clients do. Conversations [2], on mobile, does. [1] https://blog.process-one.net/ejabberd-15-06/ https://blog.process-one.net/ejabberd-15-06/ [2] https://play.google.com/store/apps/details?id=eu.siacs.conversations https://play.google.com/store/apps/details?id=eu.siacs.conve...
      - estefan 11y ago
        That sucks. I searched for ages but found nothing. Oh well, it only took me a couple of days to replace it, so it's not the end of the world.
modos189 11y ago
We are developing an open protocol, which take the best of XMPP and improve it :) We are for decentralization, the extension of the protocol, but without harm for the users. For web and native clients use thin client connecting to the server via websocket and CBOR. Why not JSON? This allows you not to have problems with sending binary files unlike XMPP. Also thinking about using WebRTC for the transmission of all messages, files and audio/video, but today WebRTC is not supported everywhere. The server has the ability to transport other protocols.
tylorr 11y ago
Would something like ProtocolBuffers[0] or FlatBuffers[1] be a good replacement for XML? The data could then be transmitted in either binary or JSON formats. [0] https://developers.google.com/protocol-buffers/ https://developers.google.com/protocol-buffers/ [1] https://google.github.io/flatbuffers/ https://google.github.io/flatbuffers/

jallmann 11y ago

Nice list. > What XMPP did right: > - Any chat protocol must be extendable That is a theoretically desirable technical property, but there is a strong argument to be made that extensibility, and the interoperability issues that result, is the Achilles heel of XMPP. Of course, there is risk the other way around too: by over-specifying, you constrain your use cases, which may hurt adoption. I'm skeptical that there will (or should) ever be "one protocol to rule them all." There is such a diversity of use-cases, anything that delegates its raison d'etre across multiple specs is going to encounter the same flavor of problems that XMPP has. > - Federation is good Definitely another desirable property. Unfortunately, the choice whether or not to federate is often a policy decision, not a technical one. And those policy decisions are not always made simply to keep users siloed: there can be a sound technical basis for them. For example, to mitigate abuse, to immediately extend the service with new features, and to have end-to-end control of user experience without concern for interoperability. Interoperability with arbitrary network elements is often the most frustrating and most expensive part of standards implementation. For federation to be taken as the default, you need critical mass, and no single IM service has ever achieved critical mass. While it would certainly be nice to have IM standardized like E-mail, telephone calls or IPv4, that comes at the cost of glacial progress forward, and you will be stuck supporting the lowest common denominator for a long time. So there will always be operators trying to short-circuit that baggage by introducing new (and often proprietary) services. The problems of standardization (or lack thereof), in their broadest strokes, are never going away. > What could have been better > - Regular users don't want to run their own server > - Logging/history is important Those seem like implementation details and not anything intrinsic to the protocol.

daurnimator 11y ago
> That is a theoretically desirable technical property, but there is a strong argument to be made that extensibility, and the interoperability issues that result, is the Achilles heel of XMPP. > Of course, there is risk the other way around too: by over-specifying, you constrain your use cases, which may hurt adoption. IMO anything that doesn't have extensibility built in, will gain it after the fact whether you want it or not. Better to have clear deliniation of duties and a standards body than a hodge-podge of uninteroperable things.
- jallmann 11y ago
  > Better to have a standards body than a hodge-podge of uninteroperable things. XMPP has both in spades -- that's where the theory doesn't meet the reality. And XMPP is hardly alone; this is a chronic issue with messaging, because the problem space is so ill-defined. The most successful protocols have been extremely focused in scope. IPv4 is essentially unchanged from RFC 791. TCP has a very carefully curated set of options that has stayed small. For ~15 years, RFC 2616 was the only spec you needed for HTTP. Notice the layering here -- those protocols don't try to be everything for everyone, unlike XMPP's catch-all design.