40 ms·

Easy XMPP: The Challenges

68 points by ge0rg 9y ago

DiThi 9y ago
After so many years fighting for XMPP, in my humble opinion it will be better to use matrix instead. XMPP was mostly killed by Google.
- SamWhited 9y ago
  Why would matrix adoption succeed where XMPP adoption failed? They've done a few good things (Olm), and don't have to worry about backwards compatibility yet, but for the most part it seems like they're just reinventing the wheel and re-discovering problems XMPP fixed 15–20 years ago, except with a giant distributed graph protocol that doesn't scale very well (not that this matters for most people that aren't, eg. Google).
  - Torgo 9y ago
    Matrix Riot is really good.
    - SamWhited 9y ago
      Does that have anything to do with Matrix though, or just that one company implemented a nice client? Conversations for Android/XMPP is also very good. There aren't any good desktop options, unfortunately, but I don't think this is necessarily an XMPP problem (although it is a problem that should be fixed), just that all the companies that write good desktop clients for XMPP make them proprietary or locked to their specific chat service.
      - nickexyz 9y ago
        https://github.com/dino/dino https://github.com/dino/dino Shows a lot of promise. Would love a nice Windows client as well though. With Mix coming, I think XMPP will become/remain pretty awesome. https://docs.ejabberd.im/tutorials/mix-010/ https://docs.ejabberd.im/tutorials/mix-010/
        SamWhited 9y ago
        Thanks for the links; I wasn't aware ejabberd had an experimental MIX impementation! I switched to Prosody for my personal server a while back just because it was easier to setup/configure, but maybe it's time to look at ejabberd again (although I hate to run a full beam VM just for a single small server; as nice as beam is, it's still a lot of extra stuff to run on my tiny server).
        nickexyz 9y ago
        I actually switched to Ejabberd from Prosody a few days ago! I hear you, but Ejabberd seems snappier now than from what I remember. Well worth a try, importing users from Prosody was a breeze. There aren't any client's that can handle mix yet, but I think I saw something about it at Conversations GitHub.
        ryukafalz 9y ago
        Dino looks really cool, thanks for the link! Looks like it doesn't yet support MAM[0], but aside from that this may be the closest thing to a Conversations equivalent I've seen for the desktop. https://github.com/dino/dino/tree/master/xmpp-vala/src/module/xep https://github.com/dino/dino/tree/master/xmpp-vala/src/modul...
      - DiThi 9y ago
        > or just that one company implemented a nice client? That is a huge deal. Having a de-facto reference client for all platforms with most of the features is very important. XMPP never had that. Google could have done it but they didn't. I used XMPP for more than a decade and tried everyone else to switch, but it has always been lacking in many ways in all platforms, without a safe fallback option.
      - goffi 9y ago
        > There aren't any good desktop options, unfortunately, but I don't think this is necessarily an XMPP problem (although it is a problem that should be fixed), just that all the companies that write good desktop clients for XMPP make them proprietary or locked to their specific chat service. For the record, we have a working Desktop/Android client on the way, which supports file sharing, carbon, MAM, Jingle (for file transfer only so far, but voice/video is planed at some point), with unique interface, and more features to come. It is also working on web and console (TUI + CLI). UI may need improvement, help is welcome. https://www.goffi.org/blog/goffi/9ff9bb64-1e1f-4334-80e5-a86f0b6b795b https://www.goffi.org/blog/goffi/9ff9bb64-1e1f-4334-80e5-a86... and https://salut-a-toi.org/ https://salut-a-toi.org/
  - sweden 9y ago
    Easy file sharing? Shared chat logs across devices? Easy encryption? Easy voice chats? Video chats? Group voice/video chats? These are all things that I have on Matrix out-of-the-box and that I have some trouble getting on XMPP. I run my own XMPP server, I have been doing it for a couple of years, but getting some modern features is very difficult, if not impossible. But with Matrix everything is quite easy.
    - SamWhited 9y ago
      I feel like clients and servers have fixed most of these. I've never had to do anything to have chat logs, all devices just have it thanks to MAM, file sharing is done with HTTP Upload, encryption is done with OMEMO (axolotl/Olm, although this one is still making its way into clients many support it already), voice and video is still a problem but lots of XMPP based services exist to make it easier (Jitsi Meet). It's only not a problem with Matrix yet because they have a single client; if they ever get as popular as XMPP and have many, they'll have the same fragmentation issues. What the XSF needs is one really good desktop client to compete; something with a feature set comparable to Convesations on Android, and good looks to boot.
      - ryukafalz 9y ago
        >I've never had to do anything to have chat logs, all devices just have it thanks to MAM Android has Conversations, which has pretty good XEP support. But I can count on one hand the number of desktop clients that support this, and none of them are Pidgin[0] or Adium, which are probably the two most popular desktop XMPP clients (though it's hard to know for sure). Gajim supports it but not for multi-user chats[1]. Chat history is certainly not a solved problem overall in XMPP. >It's only not a problem with Matrix yet because they have a single client Matrix has lots of clients, the difference is it has an official reference client that supports (almost) all the protocol's features. [0] https://developer.pidgin.im/ticket/15653 https://developer.pidgin.im/ticket/15653 [1] https://dev.gajim.org/gajim/gajim/issues/8036 https://dev.gajim.org/gajim/gajim/issues/8036
    - ge0rg 9y ago
      File sharing, cross-device logs and encryption have been addressed in the last one-two years, pushed forward by Conversations on Android. However, getting that into other clients (e.g. on desktop/web) requires dedicated developers, and this is where XMPP currently lacks a little bit - not for lack of motivation but for lack of time.
- saagarjha 9y ago
  Sigh…the struggles of dealing with Google Talk’s limited Jabber interface…
  - SamWhited 9y ago
    Google Talk is an interesting case; it was great having non-technical friends use a federated system (obviously they didn't know that, nor do they need to know that; this is one of the big problems with most small XMPP-based services, they only focus on technical people and use federation as a marketing point), but it was aweful dealing with, eg. Google refusing to follow certain standards, refusing to do TLS over S2S connections, etc. sigh indeed.
jd007 9y ago
This post comes at an interesting timing for us. Our product has a real time chat component, and currently is done over XMPP. We implemented using custom components on top of Tigase, a Java based XMPP server. Due to a variety of reasons our chat services overall have fallen out of shape and we are in the process of considering a full re-write. Part of the consideration is ditching XMPP altogether. It seemed to me that for products where chat is only one of the features, XMPP feels like an awkward add-on, requiring its own set of IDs and auth protocols. AFAIK many popular chat services these days are no longer based on XMPP, for one reason or another (e.g. scalability). Would like to hear any recommendations on what the best ways to do real time chat services nowadays.
- SamWhited 9y ago
  If you're making a service that's primarily chat, just use XMPP (disclaimer: I used to work for a large chat system based on XMPP and got involved with the protocol development when I did so). It scales very well (if it worked for Google Talk and HipChat…), but it does require that you understand the protocol. Network protocols have tons of subtle edge cases that will cause problems; inventing your own is just a bad idea, even if the existing ones require a bit more work to learn and get going with.
  - jd007 9y ago
    Did you guys write your own implementation or use an existing server?
    - SamWhited 9y ago
      We wrote our own server based on Python/Twisted (which has an XMPP implementation in Twisted Words). The Twisted implementation of XMPP is a bit outdated now, but it wasn't too terrible to work with. There may be more up to date libraries; in Python land aioxmpp seems nice.
      - jd007 9y ago
        Thanks for the info!
dmitriid 9y ago
XMPP: Captain Obvious edition Add this article to the one above: https://gultsch.de/xmpp_2016.html https://gultsch.de/xmpp_2016.html It doesn't help that most of the features expected of a modern chat are "warning: experimental" XEPs and supported by at most one chat client and at most one XMPP server. Bazaar indeed
- microcolonel 9y ago
  Yeah, frankly I think the technical specifics of XMPP are why I need about ten incompatible messaging apps with mostly the same features. It simply has never been straightforward enough to do the right thing. Unlike with functioning protocols like HTTP and SMTP. XML was a fundamentally poor choice for wire and processing efficiency, XEPs don't get ratified quickly enough, and there's a chicken-and-egg problem between server and client software in implementing them even if they do.
blfr 9y ago
Improving UX helps somewhat but how do you convince users when most have unlimited texts with their plan and Facebook Messenger, Whatsapp, Viber, Telegram, Signal... to choose from? I love the federation. I love the JID (so much better than a phone number). I love the handling of multiple clients connected to the same account. But it didn't take. Even my tech friends ditched it. IRC held strong against all of the above and even Slack to some extent but XMPP just faded away. Signal has a gif search and SMS/MMS fallback. What does XMPP offer to normies?
- SamWhited 9y ago
  Maybe the problem here is that we advertise XMPP as if it were a service, when it's a protocol that "normies" shouldn't ever have to hear about and care about. They should just be using the "ChatPro Company Fancy Name" app on their phone (which has fancy gif search), and not have to understand that it's federated under the hood. XMPP doesn't have to offer these things, the service using it does. That's the part I think the XMPP community is missing right now, a good public chat service that's federated and not strictly for businesses.
  - blfr 9y ago
    OK but now you're targeting the provider. Why should ChatPro use XMPP? Whisper Systems wanted to get (their clever version of) strong end to end encryption in the hands of regulars users so they created Signal. Moxie has blog posts[1] and long discussions why XMPP was unsuitable in which he answers your idea: if vast majority of users will coalesce around ChatPro anyway, why not get the benefits of a centralized communicator? Others want to silo their users, or offer this and that fancy feature. Why would they use XMPP? And what is the benefit of using XMPP if it's really just ChatPro with a mostly unused s2s federation port open on their server somewhere? [1] https://whispersystems.org/blog/the-ecosystem-is-moving/ https://whispersystems.org/blog/the-ecosystem-is-moving/
    - ge0rg 9y ago
      I think that ChatPro (or any other SV startup) is motivated in silo lock-in of their users indeed. Normal people don't care much about it, they just move to the next silo at some point. I see an interesting niche for XMPP in business deployments, basically what's currently covered by Slack, where you (as a user) often need to have multiple accounts just to participate in different teams of your BigCorp. XMPP allows a company to host their IM infrastructure on-premise or to use a service like ChatPro, and there is a business interest in requiring federation to allow easier interop with suppliers and customers. We just failed to execute on that mission so far... ;-)
dade_ 9y ago
I am surprised this made it to the front page. XMPP is no longer relevant, please focus your efforts on a protocol that better meets your requirements. No matter what those requirements are, I would have hard time believing that XMPP is it.
Rjevski 9y ago
XMPP is obsolete and irrelevant - please stop wasting your time with it.
5ilv3r 9y ago
Getting ejabberd set up to do encryption and file sharing was a huge pain in the butt. Worth it because I don't like SMS, but still freaking crazy hard to get running through a firewall.
starsinspace 9y ago
Recently I spent an entire evening trying to get Gajim and ChatSecure work together with OMEMO and delivery of messages to multiple devices. I'm a technical person, but I failed, I got into all kinds of weird error modes, but ultimately no success. What a frustrating evening that was, a waste of time. I have stopped to recommend Jabber/XMPP to people. Too many times I did that over the past 15 years and even when I succeeded in making them curious to try out... in the end there were too many problems and things didn't work right, and ultimately I too often looked like an idiot for recommending what is to other people just a broken tool. In the end, people don't care about "it's the choice that gives you freedom". People use the things that just work. And honestly, I can't even blame them (even if I'm personally not willing to give in to the privacy compromises that come with the other tools).
hobarrera 9y ago
Another very important thing is some form of free hosting. I used to have my own prosody server a long long time ago. Eventually, I got fed up, since very few people used XMPP. I just got rid of it and barely noticed any difference. If I had some place where I could register my own user@domain that matches my own email, I'd use it for sure. But shelling out 5-10 USD a month for that isn't worth while, not for me, nor any other early adopter (especially when almost any other IM service is "free").
- ge0rg 9y ago
  There are many free community servers, but you can't host your own domain with them. If this is a requirement, you can have a paid hosting for 1€/address/mo at https://account.conversations.im/domain/ https://account.conversations.im/domain/
  - hobarrera 9y ago
    That's exactly my point; having a paid-for entry barrier is already to high for something lacking adoption. And since JIDs look email-like, people prefer them to BE exactly their email.

68 points by ge0rg 9y ago

After so many years fighting for XMPP, in my humble opinion it will be better to use matrix instead. XMPP was mostly killed by Google.

SamWhited 9y ago
Why would matrix adoption succeed where XMPP adoption failed? They've done a few good things (Olm), and don't have to worry about backwards compatibility yet, but for the most part it seems like they're just reinventing the wheel and re-discovering problems XMPP fixed 15–20 years ago, except with a giant distributed graph protocol that doesn't scale very well (not that this matters for most people that aren't, eg. Google).
- Torgo 9y ago
  Matrix Riot is really good.
  - SamWhited 9y ago
    Does that have anything to do with Matrix though, or just that one company implemented a nice client? Conversations for Android/XMPP is also very good. There aren't any good desktop options, unfortunately, but I don't think this is necessarily an XMPP problem (although it is a problem that should be fixed), just that all the companies that write good desktop clients for XMPP make them proprietary or locked to their specific chat service.
    - nickexyz 9y ago
      https://github.com/dino/dino https://github.com/dino/dino Shows a lot of promise. Would love a nice Windows client as well though. With Mix coming, I think XMPP will become/remain pretty awesome. https://docs.ejabberd.im/tutorials/mix-010/ https://docs.ejabberd.im/tutorials/mix-010/
      - SamWhited 9y ago
        Thanks for the links; I wasn't aware ejabberd had an experimental MIX impementation! I switched to Prosody for my personal server a while back just because it was easier to setup/configure, but maybe it's time to look at ejabberd again (although I hate to run a full beam VM just for a single small server; as nice as beam is, it's still a lot of extra stuff to run on my tiny server).
        nickexyz 9y ago
        I actually switched to Ejabberd from Prosody a few days ago! I hear you, but Ejabberd seems snappier now than from what I remember. Well worth a try, importing users from Prosody was a breeze. There aren't any client's that can handle mix yet, but I think I saw something about it at Conversations GitHub.
      - ryukafalz 9y ago
        Dino looks really cool, thanks for the link! Looks like it doesn't yet support MAM[0], but aside from that this may be the closest thing to a Conversations equivalent I've seen for the desktop. https://github.com/dino/dino/tree/master/xmpp-vala/src/module/xep https://github.com/dino/dino/tree/master/xmpp-vala/src/modul...
    - DiThi 9y ago
      > or just that one company implemented a nice client? That is a huge deal. Having a de-facto reference client for all platforms with most of the features is very important. XMPP never had that. Google could have done it but they didn't. I used XMPP for more than a decade and tried everyone else to switch, but it has always been lacking in many ways in all platforms, without a safe fallback option.
    - goffi 9y ago
      > There aren't any good desktop options, unfortunately, but I don't think this is necessarily an XMPP problem (although it is a problem that should be fixed), just that all the companies that write good desktop clients for XMPP make them proprietary or locked to their specific chat service. For the record, we have a working Desktop/Android client on the way, which supports file sharing, carbon, MAM, Jingle (for file transfer only so far, but voice/video is planed at some point), with unique interface, and more features to come. It is also working on web and console (TUI + CLI). UI may need improvement, help is welcome. https://www.goffi.org/blog/goffi/9ff9bb64-1e1f-4334-80e5-a86f0b6b795b https://www.goffi.org/blog/goffi/9ff9bb64-1e1f-4334-80e5-a86... and https://salut-a-toi.org/ https://salut-a-toi.org/
- sweden 9y ago
  Easy file sharing? Shared chat logs across devices? Easy encryption? Easy voice chats? Video chats? Group voice/video chats? These are all things that I have on Matrix out-of-the-box and that I have some trouble getting on XMPP. I run my own XMPP server, I have been doing it for a couple of years, but getting some modern features is very difficult, if not impossible. But with Matrix everything is quite easy.
  - SamWhited 9y ago
    I feel like clients and servers have fixed most of these. I've never had to do anything to have chat logs, all devices just have it thanks to MAM, file sharing is done with HTTP Upload, encryption is done with OMEMO (axolotl/Olm, although this one is still making its way into clients many support it already), voice and video is still a problem but lots of XMPP based services exist to make it easier (Jitsi Meet). It's only not a problem with Matrix yet because they have a single client; if they ever get as popular as XMPP and have many, they'll have the same fragmentation issues. What the XSF needs is one really good desktop client to compete; something with a feature set comparable to Convesations on Android, and good looks to boot.
    - ryukafalz 9y ago
      >I've never had to do anything to have chat logs, all devices just have it thanks to MAM Android has Conversations, which has pretty good XEP support. But I can count on one hand the number of desktop clients that support this, and none of them are Pidgin[0] or Adium, which are probably the two most popular desktop XMPP clients (though it's hard to know for sure). Gajim supports it but not for multi-user chats[1]. Chat history is certainly not a solved problem overall in XMPP. >It's only not a problem with Matrix yet because they have a single client Matrix has lots of clients, the difference is it has an official reference client that supports (almost) all the protocol's features. [0] https://developer.pidgin.im/ticket/15653 https://developer.pidgin.im/ticket/15653 [1] https://dev.gajim.org/gajim/gajim/issues/8036 https://dev.gajim.org/gajim/gajim/issues/8036
  - ge0rg 9y ago
    File sharing, cross-device logs and encryption have been addressed in the last one-two years, pushed forward by Conversations on Android. However, getting that into other clients (e.g. on desktop/web) requires dedicated developers, and this is where XMPP currently lacks a little bit - not for lack of motivation but for lack of time.
saagarjha 9y ago
Sigh…the struggles of dealing with Google Talk’s limited Jabber interface…
- SamWhited 9y ago
  Google Talk is an interesting case; it was great having non-technical friends use a federated system (obviously they didn't know that, nor do they need to know that; this is one of the big problems with most small XMPP-based services, they only focus on technical people and use federation as a marketing point), but it was aweful dealing with, eg. Google refusing to follow certain standards, refusing to do TLS over S2S connections, etc. sigh indeed.

jd007 9y ago

This post comes at an interesting timing for us. Our product has a real time chat component, and currently is done over XMPP. We implemented using custom components on top of Tigase, a Java based XMPP server. Due to a variety of reasons our chat services overall have fallen out of shape and we are in the process of considering a full re-write. Part of the consideration is ditching XMPP altogether. It seemed to me that for products where chat is only one of the features, XMPP feels like an awkward add-on, requiring its own set of IDs and auth protocols. AFAIK many popular chat services these days are no longer based on XMPP, for one reason or another (e.g. scalability). Would like to hear any recommendations on what the best ways to do real time chat services nowadays.

SamWhited 9y ago
If you're making a service that's primarily chat, just use XMPP (disclaimer: I used to work for a large chat system based on XMPP and got involved with the protocol development when I did so). It scales very well (if it worked for Google Talk and HipChat…), but it does require that you understand the protocol. Network protocols have tons of subtle edge cases that will cause problems; inventing your own is just a bad idea, even if the existing ones require a bit more work to learn and get going with.
- jd007 9y ago
  Did you guys write your own implementation or use an existing server?
  - SamWhited 9y ago
    We wrote our own server based on Python/Twisted (which has an XMPP implementation in Twisted Words). The Twisted implementation of XMPP is a bit outdated now, but it wasn't too terrible to work with. There may be more up to date libraries; in Python land aioxmpp seems nice.
    - jd007 9y ago
      Thanks for the info!

dmitriid 9y ago

XMPP: Captain Obvious edition Add this article to the one above: https://gultsch.de/xmpp_2016.html https://gultsch.de/xmpp_2016.html It doesn't help that most of the features expected of a modern chat are "warning: experimental" XEPs and supported by at most one chat client and at most one XMPP server. Bazaar indeed

microcolonel 9y ago
Yeah, frankly I think the technical specifics of XMPP are why I need about ten incompatible messaging apps with mostly the same features. It simply has never been straightforward enough to do the right thing. Unlike with functioning protocols like HTTP and SMTP. XML was a fundamentally poor choice for wire and processing efficiency, XEPs don't get ratified quickly enough, and there's a chicken-and-egg problem between server and client software in implementing them even if they do.

blfr 9y ago

Improving UX helps somewhat but how do you convince users when most have unlimited texts with their plan and Facebook Messenger, Whatsapp, Viber, Telegram, Signal... to choose from? I love the federation. I love the JID (so much better than a phone number). I love the handling of multiple clients connected to the same account. But it didn't take. Even my tech friends ditched it. IRC held strong against all of the above and even Slack to some extent but XMPP just faded away. Signal has a gif search and SMS/MMS fallback. What does XMPP offer to normies?

SamWhited 9y ago
Maybe the problem here is that we advertise XMPP as if it were a service, when it's a protocol that "normies" shouldn't ever have to hear about and care about. They should just be using the "ChatPro Company Fancy Name" app on their phone (which has fancy gif search), and not have to understand that it's federated under the hood. XMPP doesn't have to offer these things, the service using it does. That's the part I think the XMPP community is missing right now, a good public chat service that's federated and not strictly for businesses.
- blfr 9y ago
  OK but now you're targeting the provider. Why should ChatPro use XMPP? Whisper Systems wanted to get (their clever version of) strong end to end encryption in the hands of regulars users so they created Signal. Moxie has blog posts[1] and long discussions why XMPP was unsuitable in which he answers your idea: if vast majority of users will coalesce around ChatPro anyway, why not get the benefits of a centralized communicator? Others want to silo their users, or offer this and that fancy feature. Why would they use XMPP? And what is the benefit of using XMPP if it's really just ChatPro with a mostly unused s2s federation port open on their server somewhere? [1] https://whispersystems.org/blog/the-ecosystem-is-moving/ https://whispersystems.org/blog/the-ecosystem-is-moving/
  - ge0rg 9y ago
    I think that ChatPro (or any other SV startup) is motivated in silo lock-in of their users indeed. Normal people don't care much about it, they just move to the next silo at some point. I see an interesting niche for XMPP in business deployments, basically what's currently covered by Slack, where you (as a user) often need to have multiple accounts just to participate in different teams of your BigCorp. XMPP allows a company to host their IM infrastructure on-premise or to use a service like ChatPro, and there is a business interest in requiring federation to allow easier interop with suppliers and customers. We just failed to execute on that mission so far... ;-)

dade_ 9y ago

I am surprised this made it to the front page. XMPP is no longer relevant, please focus your efforts on a protocol that better meets your requirements. No matter what those requirements are, I would have hard time believing that XMPP is it.

Rjevski 9y ago

XMPP is obsolete and irrelevant - please stop wasting your time with it.

5ilv3r 9y ago

Getting ejabberd set up to do encryption and file sharing was a huge pain in the butt. Worth it because I don't like SMS, but still freaking crazy hard to get running through a firewall.

starsinspace 9y ago

Recently I spent an entire evening trying to get Gajim and ChatSecure work together with OMEMO and delivery of messages to multiple devices. I'm a technical person, but I failed, I got into all kinds of weird error modes, but ultimately no success. What a frustrating evening that was, a waste of time. I have stopped to recommend Jabber/XMPP to people. Too many times I did that over the past 15 years and even when I succeeded in making them curious to try out... in the end there were too many problems and things didn't work right, and ultimately I too often looked like an idiot for recommending what is to other people just a broken tool. In the end, people don't care about "it's the choice that gives you freedom". People use the things that just work. And honestly, I can't even blame them (even if I'm personally not willing to give in to the privacy compromises that come with the other tools).

hobarrera 9y ago

Another very important thing is some form of free hosting. I used to have my own prosody server a long long time ago. Eventually, I got fed up, since very few people used XMPP. I just got rid of it and barely noticed any difference. If I had some place where I could register my own user@domain that matches my own email, I'd use it for sure. But shelling out 5-10 USD a month for that isn't worth while, not for me, nor any other early adopter (especially when almost any other IM service is "free").

ge0rg 9y ago
There are many free community servers, but you can't host your own domain with them. If this is a requirement, you can have a paid hosting for 1€/address/mo at https://account.conversations.im/domain/ https://account.conversations.im/domain/
- hobarrera 9y ago
  That's exactly my point; having a paid-for entry barrier is already to high for something lacking adoption. And since JIDs look email-like, people prefer them to BE exactly their email.