5 ms·

I worked on a large XMPP system for a while and realized it's actually not a very good protocol, for one reason: presence. "Presence" is the feature that shows

by oofbey 5y ago

I worked on a large XMPP system for a while and realized it's actually not a very good protocol, for one reason: presence. "Presence" is the feature that shows a little indicator next to each one of your contacts indicating whether they're online or away. The problem with presence is that it generates a LOT of traffic. It's N^2 with the average number of contacts each user has - every time a user changes presence, you have to send a message to every one of their contacts to update them about the presence. This happens whether they're paying attention to your client or not. So on a large system, the big majority of the traffic is just presence messages flying back and forth saying who is at their keyboard and who isn't. This is really wasteful in general. Worse, it's not a super important feature for a lot of people, so it's definitely not worth the effort. And ever since mobile computing became ubiquitous, it's not even clear what this is supposed to represent. So, I say let XMPP die. It was a fine idea for its time. But the protocol has no place in the modern world.

detaro 5y ago
That's afaik a problem with pretty much any system that supports presence, even though it can be somewhat optimized potentially (where I don't know how many options exist in XMPP, but I don't think would be fundamentally impossible to add)?
- fshbbdssbbgdd 5y ago
  It seems like it would be much more efficient for the presence to be stored somewhere where clients can fetch it if needed.
  - secondaryacct 5y ago
    if needed=each time presence change. So you d have an enormous graph with one node in the center refreshing presence for everyone. And end up worse.
  - detaro 5y ago
    Fetch has the downside that with the usual idea of presence you want to know immediately if presence changes, so you'd be polling constantly. What could make some sense is caching server-side and allowing the subscription to be quickly enabled/disabled, so that a client could save traffic while it's not on screen etc. When it returns, it asks for the last message for each contact to be delivered from cache and the subscription to be resumed (in exchange for having to wait for the data to come in, and until then not showing the correct information).
  - oofbey 5y ago
    Yeah, any kind of aggregation would help. But XMPP dictates that you "MUST" broadcast presence messages to all subscribers every time it changes. (Many open standards are very forceful in their language about such things.) Most chat systems have done away with it. Slack has it, but that's about it. For anything you use from your phone, what does it even mean?
    - georgyo 5y ago
      > Most chat systems have done away with it. Slack has it, but that's about it. Slack, Discord, Matrix, FB Messenger, Instagram all have it to name a few. The only ones that don't have it are SMS replacement apps (Whatsapp, Signal) and IRC. I for one greatly rely on presence. It very often informs if I should or should not send a message to someone.
      - detaro 5y ago
        IRC has away-status at least.
      - mulmen 5y ago
        > I for one greatly rely on presence. It very often informs if I should or should not send a message to someone. Do you have an example? I certainly don’t operate this way. I just fire off a message. The status only informs my expectation of when a reply might arrive. In the absence of a status I have no expectation of a reply. With mobile devices I still get immediate replies frequently because of notifications. So I don’t rely on status.
      - GoblinSlayer 5y ago
        In my experience away status is uninformative. It's often inaccurate - transitions too early, you can also send the message in the gap between afk and status transition. On the other hand when I'm at the keyboard, it doesn't mean I'm free for chat and can easily reply 10 minutes after I receive the message.
      - namdnay 5y ago
        + Teams, Jabber, Sametime, Hipchat, FB workplace i.e. basically every corporate chat system
f1refly 5y ago
Actually, it is an important feature for a lot of people, with virtually every popular messenger implementing it. Telegram and Whatsapp both go one step further and not only show when a user is at his keyboard, but also if he's currently typing. In the grand scheme of things, a few strings being routed regarding presence information don't pose a big problem for server providers (I know two of those and they didn't complain nor think of it as an issue whatsoever). You shouldn't impose your use case to "a lot of people", and more importantly also shouldn't conclude that it's "definitely not worth it" from that. The industry disagrees with you, and so does my mom.
- oofbey 5y ago
  When somebody is typing or not does not get broadcast to every contact. It’s only sent to the individual who is about to receive the typed message. O(1) communication vs O(N). That is a different feature which is not XMPP presence and I agree way more relevant in the modern world. Even if the server load from presence is acceptable (we clearly disagree here) it’s not okay to bombard somebody’s mobile device with these messages all the time. If your phone’s CPU and radio need to wake from sleep every time any one of your contacts picks up or puts down their phone, your battery life is ruined. You may think these messages important, but they are not urgent.
  - f1refly 5y ago
    Honestly, I can't disagree with you on technical terms, because what you're saying makes sense and on other topics this discussion would be over. But in this case, I have Conversations on my phone and can say from experience that it didn't degrade my battery life whatsoever. I know that android restricts this behaviour heavily by default and for good reason, because lots of apps will drain the battery in a matter of hours. No idea how the conversations devs do it, there must be a quirk to it I don't know which makes modem usage acceptable. I'm not a zealot in this case and I accept the flaws, but I can't leave you thinking mobile xmpp hurts the battery in a relavant way because that's (in my experience) not true. I'm not deep into xmpp xep standards, but I'd imagine theres a way to tell the server "I don't care about event type x please don't send those until I ask for them again"
IanSanders 5y ago
What do you suggest replacing it with?
Flowdalic 5y ago
I wonder why people jump fast to conclusion that to "let XMPP die", when the protocol can also be iteratively improved. Presence is not required in XMPP, its an optional feature. Everything you said has been considered in newer XMPP extension protocols (like XEP-0396: MIX).
- oofbey 5y ago
  Why improve when you can start over? Modern tools for pub sub and websockets are so much better. We don’t need to use XML any more. (For the love of god please let XML die.) The old code and implementations are more of a liability than an asset, especially if you want to take out major functionality. Unless interop with AOL instant messenger’s install base is a key feature or something. (Don’t know if AIM uses XMPP but the timing is about right and representative of what you’d get.)
  - Zash 5y ago
    Why start over when you can improve? You'll most likely end up with the second system effect, probably slowly rediscover all the valid reasons for the things you disliked, probably re-make all the mistakes.

by oofbey 5y ago

detaro 5y ago

That's afaik a problem with pretty much any system that supports presence, even though it can be somewhat optimized potentially (where I don't know how many options exist in XMPP, but I don't think would be fundamentally impossible to add)?

fshbbdssbbgdd 5y ago
It seems like it would be much more efficient for the presence to be stored somewhere where clients can fetch it if needed.
- secondaryacct 5y ago
  if needed=each time presence change. So you d have an enormous graph with one node in the center refreshing presence for everyone. And end up worse.
- detaro 5y ago
  Fetch has the downside that with the usual idea of presence you want to know immediately if presence changes, so you'd be polling constantly. What could make some sense is caching server-side and allowing the subscription to be quickly enabled/disabled, so that a client could save traffic while it's not on screen etc. When it returns, it asks for the last message for each contact to be delivered from cache and the subscription to be resumed (in exchange for having to wait for the data to come in, and until then not showing the correct information).
- oofbey 5y ago
  Yeah, any kind of aggregation would help. But XMPP dictates that you "MUST" broadcast presence messages to all subscribers every time it changes. (Many open standards are very forceful in their language about such things.) Most chat systems have done away with it. Slack has it, but that's about it. For anything you use from your phone, what does it even mean?
  - georgyo 5y ago
    > Most chat systems have done away with it. Slack has it, but that's about it. Slack, Discord, Matrix, FB Messenger, Instagram all have it to name a few. The only ones that don't have it are SMS replacement apps (Whatsapp, Signal) and IRC. I for one greatly rely on presence. It very often informs if I should or should not send a message to someone.
    - detaro 5y ago
      IRC has away-status at least.
    - mulmen 5y ago
      > I for one greatly rely on presence. It very often informs if I should or should not send a message to someone. Do you have an example? I certainly don’t operate this way. I just fire off a message. The status only informs my expectation of when a reply might arrive. In the absence of a status I have no expectation of a reply. With mobile devices I still get immediate replies frequently because of notifications. So I don’t rely on status.
    - GoblinSlayer 5y ago
      In my experience away status is uninformative. It's often inaccurate - transitions too early, you can also send the message in the gap between afk and status transition. On the other hand when I'm at the keyboard, it doesn't mean I'm free for chat and can easily reply 10 minutes after I receive the message.
    - namdnay 5y ago
      + Teams, Jabber, Sametime, Hipchat, FB workplace i.e. basically every corporate chat system

f1refly 5y ago

Actually, it is an important feature for a lot of people, with virtually every popular messenger implementing it. Telegram and Whatsapp both go one step further and not only show when a user is at his keyboard, but also if he's currently typing. In the grand scheme of things, a few strings being routed regarding presence information don't pose a big problem for server providers (I know two of those and they didn't complain nor think of it as an issue whatsoever). You shouldn't impose your use case to "a lot of people", and more importantly also shouldn't conclude that it's "definitely not worth it" from that. The industry disagrees with you, and so does my mom.

oofbey 5y ago
When somebody is typing or not does not get broadcast to every contact. It’s only sent to the individual who is about to receive the typed message. O(1) communication vs O(N). That is a different feature which is not XMPP presence and I agree way more relevant in the modern world. Even if the server load from presence is acceptable (we clearly disagree here) it’s not okay to bombard somebody’s mobile device with these messages all the time. If your phone’s CPU and radio need to wake from sleep every time any one of your contacts picks up or puts down their phone, your battery life is ruined. You may think these messages important, but they are not urgent.
- f1refly 5y ago
  Honestly, I can't disagree with you on technical terms, because what you're saying makes sense and on other topics this discussion would be over. But in this case, I have Conversations on my phone and can say from experience that it didn't degrade my battery life whatsoever. I know that android restricts this behaviour heavily by default and for good reason, because lots of apps will drain the battery in a matter of hours. No idea how the conversations devs do it, there must be a quirk to it I don't know which makes modem usage acceptable. I'm not a zealot in this case and I accept the flaws, but I can't leave you thinking mobile xmpp hurts the battery in a relavant way because that's (in my experience) not true. I'm not deep into xmpp xep standards, but I'd imagine theres a way to tell the server "I don't care about event type x please don't send those until I ask for them again"

IanSanders 5y ago

What do you suggest replacing it with?

Flowdalic 5y ago

I wonder why people jump fast to conclusion that to "let XMPP die", when the protocol can also be iteratively improved. Presence is not required in XMPP, its an optional feature. Everything you said has been considered in newer XMPP extension protocols (like XEP-0396: MIX).

oofbey 5y ago
Why improve when you can start over? Modern tools for pub sub and websockets are so much better. We don’t need to use XML any more. (For the love of god please let XML die.) The old code and implementations are more of a liability than an asset, especially if you want to take out major functionality. Unless interop with AOL instant messenger’s install base is a key feature or something. (Don’t know if AIM uses XMPP but the timing is about right and representative of what you’d get.)
- Zash 5y ago
  Why start over when you can improve? You'll most likely end up with the second system effect, probably slowly rediscover all the valid reasons for the things you disliked, probably re-make all the mistakes.