9 ms·

The Sad State of Mobile XMPP in 2014

102 points by ge0rg 13y ago

NSAID 13y ago
Maybe someday I'll be able to use my iMessage-to-XMPP scripts to communicate with several devices instead of just my Android phone. And maybe someday I can build a bridge between FaceTime and XMPP once all the media protocols get sorted out. It may currently be a sad state, but it's exciting to see the direction things are/can be going.
ChikkaChiChi 13y ago
I don't deal in the iOS ecosystem, but the fact that the only way to get data to the device using push is insecure is just pathetic. In a perfect world, the public would care. Instead, we scream into our own echo chamber.
- kalleboo 13y ago
  The public cares that their battery lasts longer since there's only a single persistent network connection.
- atmosx 13y ago
  The public doesn't know. In a perfect world the public would had make choices which would lead everyone to open platforms, away from walled gardens, but as we don't know how to pick up leaders, most of us don't understand how and why privacy works.
- danudey 13y ago
  iOS 7 supports background wake-up-and-do-things for apps, so this problem is actually solved now.
  - ge0rg 13y ago
    Would you care to elaborate? The only thing I found in the apple docs is that calling your app callback is a "best effort" service not guaranteed to do anything if your app is not running already.
danielrhodes 13y ago
Why would anybody want to use XMPP with mobile anyways? The protocol is very noisy and as a result it will keep turning your radio on/off in the background, effectively killing your battery.
- jerf 13y ago
  The point is precisely to make XMPP work with mobile. There's no fundamental reason why this is impossible, it just doesn't work very well now. Of course if it is tweaked to work on mobile, well, it will work on mobile after that.
- josteink 13y ago
  Why would anybody want to use XMPP with mobile anyways? Anyone with an Android-device is already using it. Android's push-notification depends on XMPP to work, and Gtalk/Hangouts is already piggybacking on that, making it a free ride. So, unless you are on the patent-troll side of mobile, you have no reason not to use it.
  - nutjob123 13y ago
    >Android's push-notification depends on XMPP to work Do you have a source for this info?
    - ge0rg 13y ago
      In the early Android versions (pre 1.0), the Android API actually included XMPP methods. I also remember reading debug messages containing "XMPP" in the function names related to GCM. These are just hints, and the protocol might have been changed since then, but the GCM uplink feature also indicates that it still at least relates to XMPP.
    - josteink 13y ago
      Not any hard reference-data, but whenever I am on a network where Android-push notification doesn't work, it seems to be because the device cannot connect to talk.google.com on the regular XMPP port, or at least so I seem to remember my logcat telling me. Note: I've observed this without being signed into Google Talk or Hangouts.
morsch 13y ago
The state of instant messaging is sad overall. There was a steady evolution to my own IM usage: from a proprietary, walled-garden IM solution (ICQ) via a reverse engineered multi-network client (a la Trillian) to a federated, free protocol, XMPP. I figured everybody would be using XMPP of some sorts and we'd be on our merry way to solve more interesting problems on that basis. Instead, everybody is back to walled gardens that can't interoperate, and I'm using 5 or more services to talk to acquaintances -- SMS, Jabber, Google Talk, Skype, the list goes on -- and fall back to good old email for people where I don't have access to their IM of choice (such as Facebook chat or Whatsapp). It's such a pain in the ass. It's 2014, I expect it to be matter of course to be able to chat with everyone I know with all the bells and whistles -- OTR-support, offline messages, synced history where desired, file transfer. It's an odd twist of technical history that we're not there yet.
- MichaelGG 13y ago
  What is he benefit or point in Facebook or Google doing interop with other IM systems? Interop seems like a desperate last-chance measure when you realise you're losing users. I don't think there's anything technical involved. Unless spam is considered a technical problem, but denying federation seems to solve that problem considerably,
  - Jtsummers 13y ago
    Goodwill? I don't think it's really a "desperate last-chance measure", so much as a tool. FB and Google both offer a lot of other services beyond just chat. FB offers photo sharing, the newsfeed, potential for correct address books (at least with people that share that info with their friends), event organizing, games. I'm not that into it, but I know a lot of people that are. Then Google offers Google Mail, Docs, Hangouts, Calendar, Drive. I think they have their own image sharing system (I don't take many photos, so those features are nearly useless to me). Their chat protocols are more of a side-effect of having something already associated with your identity. Opening them up to federation likely won't cost them many users (and those users it does cost them will likely be as inactive on all their other services as I am), but has the potential to gain goodwill and endorsement from other circles (particularly those people that others -- family, friends -- go to for tech advice).
mbq 13y ago
For me the worst thing was the way it handles multiple devices -- AFAIK when there are few clients with the same priority connected, the server delivers messages completely arbitrarily, usually only to a one of them and based on some crazy criterion like the last connection time; not to mention that the context is always lost. I ended up using SSH client and irssi :(
- danudey 13y ago
  My experience has been: * Google: Delivers incoming messages to whichever client most recently talked; occasionally 'resets' and delivers messages everywhere until a client talks again. * Facebook: Delivers incoming and outgoing messages to web client; incoming messages only to XMPP clients (i.e. no outgoing message sync); occasionally outgoing XMPP messages are never sent or recorded at Facebook (this seems to happen and then stay a problem forever), resulting in your messages being silently lost.
zdw 13y ago
iOS user here. I have Colloquy Mobile, with a backend of ZNC/Bitlbee, which bounces IRC and various IM protocols to my phone. But it's by no means a layman's solution...
- dhimes 13y ago
  Does that require your mac to bounce the messages to you?
  - zdw 13y ago
    Nope, ZNC has a colloquy mobile bouncer: http://wiki.znc.in/Colloquy http://wiki.znc.in/Colloquy
  - dijit 13y ago
    no, you can do push notifications from linux. (my friend wrote some push stuff for ZNC to his iOS irc client - which I use.)
- mathrawka 13y ago
  I'd be interested in hearing a bit more how this works... I currently just use tmux on my server instead of an IRC bouncer. And this is the first time I have heard about Bitlbee...
tehabe 13y ago
I don't like that most of the XMPP client on Android are presenting the roster of contacts instead of my conversations I'm currently having. Those are important to me. That is why I prefer systems like Hangouts or WhatsApp. So most XMPP clients looks odd for me, they try to look like desktop clients. Which doesn't work on Android I think.
shmerl 13y ago
What is the state of power saving related XEPs which are critical for battery life on mobile? Looks like nothing is moving anywhere. For example XEP-0273 is still deferred: http://xmpp.org/extensions/xep-0273.html http://xmpp.org/extensions/xep-0273.html I'd worry about this even before anything else when it comes to XMPP on mobile, since otherwise your client can be a hungry monster eating up your battery in no time.
- sheraz 13y ago
  I'm on the edge of my knowledge here, but your might look at the MQTT protocol [1]. Facebook uses this for Facebook Messenger [2]. MQTT, I believe can solve part of the power issue, as well others. However, I do not know how easy it is run XMPP over MQTT. I've been evaluating this internally at my company as we explore M2M (machine to machine) communications in mobile. [1] - http://mqtt.org http://mqtt.org [2] - http://mqtt.org/2011/08/mqtt-used-by-facebook-messenger http://mqtt.org/2011/08/mqtt-used-by-facebook-messenger
  - shmerl 13y ago
    Interesting, but it requires an overhead of routing one high end protocol (XMPP) over another one. Not sure if it's the best approach. XMPP itself can offer mobile optimizations (like the XEP I brought above), but for some reason the whole thing seems stalled.
    - sheraz 13y ago
      XMPP is XML, which is most probably sent over TCP. I think if you had a XMPP to MQTT gateway this might solve the "last mile" for XMPP and power.
      - shmerl 13y ago
        It still looks more like a workaround rather than a straight solution. I.e. chatty nature of XMPP is masked / mitigated with MQTT. A proper way is really to reduce the chattiness of XMPP itself in cases when it's needed.
msie 13y ago
I suspect some companies don't implement XMPP because it's too complex for simple messaging with very little upside. For a startup why worry about potential future interoperability with Google services(?), which would add months of engineering to get things right? YAGNI.
atmosx 13y ago
As we move to mobile platform, privacy becomes more and more exotic.
jgale 13y ago
Layer (http://layer.com http://layer.com) is building a successor to XMPP that's optimized for mobile and solves device coordination problems. The inventor of XMPP, Jeremy Miller, works there.
- ytjohn 13y ago
  This looks interesting. I looked at their site and documentation for implementing a client (once they are up and running). Do they provide any other information about their goals? Do you know if they have a goal to let anyone run a Layer server with interopability like xmpp does?
  - ytjohn 13y ago
    Ahh.. found their blog. https://layer.com/blog/post/why-I-joined-layer-jeremie https://layer.com/blog/post/why-I-joined-layer-jeremie
soapdog 13y ago
An interesting thing is the Push Notification system used by Firefox OS which does not maintain a connection with the server like Android or iOS, instead the carrier has a notification server so that when you send a push notification, the carrier sends a signal to the phone and awakes the app. Making IM apps that don't need to keep a connection open is a great way to reduce battery. This way whenever you receive any message the carrier pings the phone, the app awakes and open the connection to retrieve the messages. More info at https://developer.mozilla.org/en-US/docs/WebAPI/Simple_Push https://developer.mozilla.org/en-US/docs/WebAPI/Simple_Push I am sure that this can be used to make a XMPP client that makes good use of the battery by only keeping the connection open while people are talking and effectively closing it once the chat goes idle.
- ge0rg 13y ago
  As I wrote, it is hard to keep the conversation private in such an offline+push context. On the other hand, the additional battery load induced by a properly implemented XMPP connection is low. Apple's main fear is probably badly designed apps which really drain the battery.
kamobit 13y ago
"If we can manage that, we can also convince all the users of WhatsApp, Facebook and Google Hangouts to switch to an open protocol that is ready for the challenges of 2014." Not sure if serious. Let's hope not.
- ge0rg 13y ago
  It was tongue-in-cheek, though my agenda is to make mobile xmpp as reliable and user-friendly as the proprietary apps. Can you elaborate your reasons for hoping I'm not serious?
- newman314 13y ago
  IIRC, Facebook Chat was using XMPP but switched to MQTT. Not sure if it has changed since. Disclaimer: I could be completely wrong.
nasalgoat 13y ago
When we were developing a messaging system for our app, we looked at XMPP, and it wasn't a hard choice to not use it. As it's meant to be a constant-connected protocol, it's a poor choice for mobile. But beyond that, the various plug-ins are overly complicated, difficult to configure and most of the existing servers are "black boxes" that tend to not be easily hackable. It was easier to make a chat protocol over HTTP from scratch than scale XMPP. YMMV.
johnchristopher 13y ago
I remember a post mortem for XMPP/jabber from 5 or 7 years ago. The author described what went wrong and the title was something like "the state of xmpp and why it's dead" or something. Does anyone here remember that ? I have been looking over the internet but my google-fu failed me and I really wished I could read it again.

102 points by ge0rg 13y ago

NSAID 13y ago

Maybe someday I'll be able to use my iMessage-to-XMPP scripts to communicate with several devices instead of just my Android phone. And maybe someday I can build a bridge between FaceTime and XMPP once all the media protocols get sorted out. It may currently be a sad state, but it's exciting to see the direction things are/can be going.

ChikkaChiChi 13y ago

I don't deal in the iOS ecosystem, but the fact that the only way to get data to the device using push is insecure is just pathetic. In a perfect world, the public would care. Instead, we scream into our own echo chamber.

kalleboo 13y ago
The public cares that their battery lasts longer since there's only a single persistent network connection.
atmosx 13y ago
The public doesn't know. In a perfect world the public would had make choices which would lead everyone to open platforms, away from walled gardens, but as we don't know how to pick up leaders, most of us don't understand how and why privacy works.
danudey 13y ago
iOS 7 supports background wake-up-and-do-things for apps, so this problem is actually solved now.
- ge0rg 13y ago
  Would you care to elaborate? The only thing I found in the apple docs is that calling your app callback is a "best effort" service not guaranteed to do anything if your app is not running already.

danielrhodes 13y ago

Why would anybody want to use XMPP with mobile anyways? The protocol is very noisy and as a result it will keep turning your radio on/off in the background, effectively killing your battery.

jerf 13y ago
The point is precisely to make XMPP work with mobile. There's no fundamental reason why this is impossible, it just doesn't work very well now. Of course if it is tweaked to work on mobile, well, it will work on mobile after that.
josteink 13y ago
Why would anybody want to use XMPP with mobile anyways? Anyone with an Android-device is already using it. Android's push-notification depends on XMPP to work, and Gtalk/Hangouts is already piggybacking on that, making it a free ride. So, unless you are on the patent-troll side of mobile, you have no reason not to use it.
- nutjob123 13y ago
  >Android's push-notification depends on XMPP to work Do you have a source for this info?
  - ge0rg 13y ago
    In the early Android versions (pre 1.0), the Android API actually included XMPP methods. I also remember reading debug messages containing "XMPP" in the function names related to GCM. These are just hints, and the protocol might have been changed since then, but the GCM uplink feature also indicates that it still at least relates to XMPP.
  - josteink 13y ago
    Not any hard reference-data, but whenever I am on a network where Android-push notification doesn't work, it seems to be because the device cannot connect to talk.google.com on the regular XMPP port, or at least so I seem to remember my logcat telling me. Note: I've observed this without being signed into Google Talk or Hangouts.

morsch 13y ago

The state of instant messaging is sad overall. There was a steady evolution to my own IM usage: from a proprietary, walled-garden IM solution (ICQ) via a reverse engineered multi-network client (a la Trillian) to a federated, free protocol, XMPP. I figured everybody would be using XMPP of some sorts and we'd be on our merry way to solve more interesting problems on that basis. Instead, everybody is back to walled gardens that can't interoperate, and I'm using 5 or more services to talk to acquaintances -- SMS, Jabber, Google Talk, Skype, the list goes on -- and fall back to good old email for people where I don't have access to their IM of choice (such as Facebook chat or Whatsapp). It's such a pain in the ass. It's 2014, I expect it to be matter of course to be able to chat with everyone I know with all the bells and whistles -- OTR-support, offline messages, synced history where desired, file transfer. It's an odd twist of technical history that we're not there yet.

MichaelGG 13y ago
What is he benefit or point in Facebook or Google doing interop with other IM systems? Interop seems like a desperate last-chance measure when you realise you're losing users. I don't think there's anything technical involved. Unless spam is considered a technical problem, but denying federation seems to solve that problem considerably,
- Jtsummers 13y ago
  Goodwill? I don't think it's really a "desperate last-chance measure", so much as a tool. FB and Google both offer a lot of other services beyond just chat. FB offers photo sharing, the newsfeed, potential for correct address books (at least with people that share that info with their friends), event organizing, games. I'm not that into it, but I know a lot of people that are. Then Google offers Google Mail, Docs, Hangouts, Calendar, Drive. I think they have their own image sharing system (I don't take many photos, so those features are nearly useless to me). Their chat protocols are more of a side-effect of having something already associated with your identity. Opening them up to federation likely won't cost them many users (and those users it does cost them will likely be as inactive on all their other services as I am), but has the potential to gain goodwill and endorsement from other circles (particularly those people that others -- family, friends -- go to for tech advice).

mbq 13y ago

For me the worst thing was the way it handles multiple devices -- AFAIK when there are few clients with the same priority connected, the server delivers messages completely arbitrarily, usually only to a one of them and based on some crazy criterion like the last connection time; not to mention that the context is always lost. I ended up using SSH client and irssi :(

danudey 13y ago
My experience has been: * Google: Delivers incoming messages to whichever client most recently talked; occasionally 'resets' and delivers messages everywhere until a client talks again. * Facebook: Delivers incoming and outgoing messages to web client; incoming messages only to XMPP clients (i.e. no outgoing message sync); occasionally outgoing XMPP messages are never sent or recorded at Facebook (this seems to happen and then stay a problem forever), resulting in your messages being silently lost.

zdw 13y ago

iOS user here. I have Colloquy Mobile, with a backend of ZNC/Bitlbee, which bounces IRC and various IM protocols to my phone. But it's by no means a layman's solution...

dhimes 13y ago
Does that require your mac to bounce the messages to you?
- zdw 13y ago
  Nope, ZNC has a colloquy mobile bouncer: http://wiki.znc.in/Colloquy http://wiki.znc.in/Colloquy
- dijit 13y ago
  no, you can do push notifications from linux. (my friend wrote some push stuff for ZNC to his iOS irc client - which I use.)
mathrawka 13y ago
I'd be interested in hearing a bit more how this works... I currently just use tmux on my server instead of an IRC bouncer. And this is the first time I have heard about Bitlbee...

tehabe 13y ago

I don't like that most of the XMPP client on Android are presenting the roster of contacts instead of my conversations I'm currently having. Those are important to me. That is why I prefer systems like Hangouts or WhatsApp. So most XMPP clients looks odd for me, they try to look like desktop clients. Which doesn't work on Android I think.

shmerl 13y ago

What is the state of power saving related XEPs which are critical for battery life on mobile? Looks like nothing is moving anywhere. For example XEP-0273 is still deferred: http://xmpp.org/extensions/xep-0273.html http://xmpp.org/extensions/xep-0273.html I'd worry about this even before anything else when it comes to XMPP on mobile, since otherwise your client can be a hungry monster eating up your battery in no time.

sheraz 13y ago
I'm on the edge of my knowledge here, but your might look at the MQTT protocol [1]. Facebook uses this for Facebook Messenger [2]. MQTT, I believe can solve part of the power issue, as well others. However, I do not know how easy it is run XMPP over MQTT. I've been evaluating this internally at my company as we explore M2M (machine to machine) communications in mobile. [1] - http://mqtt.org http://mqtt.org [2] - http://mqtt.org/2011/08/mqtt-used-by-facebook-messenger http://mqtt.org/2011/08/mqtt-used-by-facebook-messenger
- shmerl 13y ago
  Interesting, but it requires an overhead of routing one high end protocol (XMPP) over another one. Not sure if it's the best approach. XMPP itself can offer mobile optimizations (like the XEP I brought above), but for some reason the whole thing seems stalled.
  - sheraz 13y ago
    XMPP is XML, which is most probably sent over TCP. I think if you had a XMPP to MQTT gateway this might solve the "last mile" for XMPP and power.
    - shmerl 13y ago
      It still looks more like a workaround rather than a straight solution. I.e. chatty nature of XMPP is masked / mitigated with MQTT. A proper way is really to reduce the chattiness of XMPP itself in cases when it's needed.

msie 13y ago

I suspect some companies don't implement XMPP because it's too complex for simple messaging with very little upside. For a startup why worry about potential future interoperability with Google services(?), which would add months of engineering to get things right? YAGNI.

atmosx 13y ago

As we move to mobile platform, privacy becomes more and more exotic.

jgale 13y ago

Layer (http://layer.com http://layer.com) is building a successor to XMPP that's optimized for mobile and solves device coordination problems. The inventor of XMPP, Jeremy Miller, works there.

ytjohn 13y ago
This looks interesting. I looked at their site and documentation for implementing a client (once they are up and running). Do they provide any other information about their goals? Do you know if they have a goal to let anyone run a Layer server with interopability like xmpp does?
- ytjohn 13y ago
  Ahh.. found their blog. https://layer.com/blog/post/why-I-joined-layer-jeremie https://layer.com/blog/post/why-I-joined-layer-jeremie

soapdog 13y ago

An interesting thing is the Push Notification system used by Firefox OS which does not maintain a connection with the server like Android or iOS, instead the carrier has a notification server so that when you send a push notification, the carrier sends a signal to the phone and awakes the app. Making IM apps that don't need to keep a connection open is a great way to reduce battery. This way whenever you receive any message the carrier pings the phone, the app awakes and open the connection to retrieve the messages. More info at https://developer.mozilla.org/en-US/docs/WebAPI/Simple_Push https://developer.mozilla.org/en-US/docs/WebAPI/Simple_Push I am sure that this can be used to make a XMPP client that makes good use of the battery by only keeping the connection open while people are talking and effectively closing it once the chat goes idle.

ge0rg 13y ago
As I wrote, it is hard to keep the conversation private in such an offline+push context. On the other hand, the additional battery load induced by a properly implemented XMPP connection is low. Apple's main fear is probably badly designed apps which really drain the battery.

kamobit 13y ago

"If we can manage that, we can also convince all the users of WhatsApp, Facebook and Google Hangouts to switch to an open protocol that is ready for the challenges of 2014." Not sure if serious. Let's hope not.

ge0rg 13y ago
It was tongue-in-cheek, though my agenda is to make mobile xmpp as reliable and user-friendly as the proprietary apps. Can you elaborate your reasons for hoping I'm not serious?
newman314 13y ago
IIRC, Facebook Chat was using XMPP but switched to MQTT. Not sure if it has changed since. Disclaimer: I could be completely wrong.

nasalgoat 13y ago

When we were developing a messaging system for our app, we looked at XMPP, and it wasn't a hard choice to not use it. As it's meant to be a constant-connected protocol, it's a poor choice for mobile. But beyond that, the various plug-ins are overly complicated, difficult to configure and most of the existing servers are "black boxes" that tend to not be easily hackable. It was easier to make a chat protocol over HTTP from scratch than scale XMPP. YMMV.

johnchristopher 13y ago

I remember a post mortem for XMPP/jabber from 5 or 7 years ago. The author described what went wrong and the title was something like "the state of xmpp and why it's dead" or something. Does anyone here remember that ? I have been looking over the internet but my google-fu failed me and I really wished I could read it again.