11 ms·

Meh. None of these are claims I really see particularly often. The claims I do see are: 1. XMPP is not battery-friendly on mobile devices (this primarily has/h

Meh. None of these are claims I really see particularly often. The claims I do see are: 1. XMPP is not battery-friendly on mobile devices (this primarily has/had to do with the absence of push notifications, it seems). 2. XML is unnecessarily complex for this kind of usecase, and too hard and time-consuming to work with (and I agree). That is most likely what 'slow' refers to - not slow in parsing it, but in developing for it. Neither of these are addressed. I also don't really understand the comparison with "JSON over HTTP" - why would you ever use that for a realtime application? The correct technology is JSON-over-WebSockets, and it doesn't really make sense to compare XMPP to a known-bad implementation and say "see? XMPP isn't that bad!" Another issue with XMPP is the strange implementation of both resources and the existing options for sending to all resources (especially when taking into account tools like OTR). People want device synchronization, and it just isn't viable with XMPP right now.

ape4 11y ago
> absence of push If you are connected and your buddy sends a message you get it right away.
- niklasni1 11y ago
  On mobile you want messages to come through without each app maintaining a connection.
  - emias 11y ago
    Because?
    - joepie91_ 11y ago
      Because TCP connections are expensive on your battery life. Especially if your network is flaky, and all those connections need to constantly repeat their handshakes.
      - ultramancool 11y ago
        Not really, TCP connections are how GCM does things even. Unless you're using keep alives or otherwise preventing sleep, I believe these devices are capable of sending an interrupt when there's activity on a network connection though I could be mistaken.
      - Flowdalic 11y ago
        Idle TCP connections do not consume any battery. And on a data connectivity change, e.g. GSM <-> WiFi switch, there is a good chance that a dozen other components start trying to re-establish their connections, which means the radio is awake anyway (for, usually, at most a few seconds).
        rakoo 11y ago
        > Idle TCP connections do not consume any battery. An idle connection doesn't consume anything, but a useful idle connection will have some keepalive every X minutes. Multiply that by the number of connections your application will have and by the number of background application you run on your phone, and the radio will never truly sleep or enter the "low" mode. The solution wanted by Google and Apple is to maintain a single connection to their server on which every push is aggregated and sent, and that the OS itself manages.
        Flowdalic 11y ago
        I use my mobile XMPP connections without a TCP keepalive but send a server ping if there has been stanzas received in the last 30 minutes and get a useful and reliable XMPP connection without a noticeable impact on battery. If you use Android's AlarmManager to trigger the check, then Android will even take care of scheduling the "alarm" with other alarms for efficiency. > Multiply that by the number of connections your application will have and by the number of background application you run on your phone, and the radio will never truly sleep or enter the "low" mode. Why would X x Y idle connections (X: TCP connections per app, Y: applictions) prevent your radio from sleeping?
        11y ago
    - rakoo 11y ago
      Because iOS doesn't let you do that (see http://op-co.de/blog/posts/mobile_xmpp_in_2014/#index6h2 http://op-co.de/blog/posts/mobile_xmpp_in_2014/#index6h2)
      - pjc50 11y ago
        This is the real answer - long-standing connections are banned by the platform monopolist - and should be much closer to the top.
      - emias 11y ago
        Yes, but I think the claim was that a persistent TCP connection has a negative effect on battery life, which is not obvious to me. An idle connection should have no impact whatsoever.
sz4kerto 11y ago
> XML is unnecessarily complex for this kind of usecase, and too hard and time-consuming to work with (and I agree) What is 'difficult' in XML?
- inversionOf 11y ago
  The notion that XML is difficult comes from the most profoundly lazy evaluations possible, where the most minimal of up front understanding are effort are eschewed if you can just launch mongodb and insert some giant mystery meat bags of json. We've seen this same mistake play out over, and over, and over again, and it is incredible that in a field where projects often end up taking thousands or more of man hours, people sweat and complain about 15 minutes up front gaining a basic understanding of a structure or API, and instead pursue solutions that, for instance, doesn't even have a date type or even consistent hack standard of how to handle it. This is how we have the rapid trends in this field, where the new panacea is tomorrow's nightmare.
  - stonogo 11y ago
    This is the most breathtakingly ironic defense of XML I have ever seen.
    - inversionOf 11y ago
      In what way is my post "ironic"? I would dearly love to know. Do you think my defense of XML is somehow that I am too lazy to know alternatives. You would be painfully wrong, dear friend, though such is a go to among the intellectually lazy (how many failed NoSQL projects were justified on notions of "people who criticize elements of it are just old and fear change". It's the laziest, most unprofessional retort possible, and fueled about a year of content on Hacker News, and was the meat of a million failed adventures) My post will be pummeled by the defensive, and that's okay.
      - stonogo 11y ago
        All of your invective was exactly what programmers were saying to XML proponents in the early days of XML. I still can't entirely tell if you're deliberately doing this or if you really are just wholly unaware of yourself. > The notion that binary serialization is difficult comes from the most profoundly lazy evaluations possible, where the most minimal of up front understanding are effort are eschewed if you can just open a text file and insert some giant mystery meat bags of XML. etc etc
vog 11y ago
While I agree that XMPP is not the future for app-to-app communication, I do so for a slightly different reasons. > XML is unnecessarily complex for this kind of usecase XMPP is too complex, but the XML stuff is really the smallest issue, compared with other complexities in the protocol. For example, every client that successfully connected to an XMPP server must first send a "<presence>" message. Otherwise, it won't receive any messages despite their successful connection. And there are more such surprises in XMPP. Good luck in understanding which "devices" (#xxxx) of the same Jabber ID will receive a message targeted at that ID. It's really annoying and discouraging to debug that kind of issues - especially since these aren't bugs. These work as specified, but the specifications were obviously designed with direct human chatting or direct human collaboration in mind, programs being an afterthought. > The correct technology is JSON-over-WebSockets Not really. Whenever I saw someone trying this "in the wild", I see them quickly switching to HTTP long-polling. (Not to be confused with polling! We are still talking about instant push notifications with very low network overhead.) For example, there are some libraries that used to provide wrappers around WebSockets, which switched to HTTP long-polling as fallback for old browsers. Nowadays, it seems they use long-polling by default, even though browser support becomes better and better. Don't underestimate the role of firewalls and HTTP proxies! Many firewalls are aggressive configured, and many companies have forced HTTP proxies in place.
- joepie91_ 11y ago
  That is a browser issue, and does not exist in native clients. If we're talking about XML-stream-over-sockets, then comparing it to browsers makes no sense (because they can't do that!), and thus long-polling and JSON-over-HTTP become irrelevant. EDIT: Hmm, did you edit your post? I don't recall the first part being there... anyhow: > XMPP is too complex, but the XML stuff is really the smallest issue, compared with other complexities in the protocol. That is one factor, but the protocol complexities often seem to derive from the question 'how do we represent this in XML?', and that's an issue. Another problem is the jargon - the MUC specification is utterly incomprehensible, because the terms it uses for things are so wildly different from how they are used everywhere else, that building a mental model is really hard. I've seen this issue in other XEPs as well.
- Flowdalic 11y ago
rakoo 11y ago
> I also don't really understand the comparison with "JSON over HTTP" One of the complainers is matrix (https://matrix.org/ https://matrix.org/), which does JSON over HTTP.
- treffer 11y ago
  Which has it's own set of problems, btw. Try to find an API for "delete". Oh, and I'm eager to see how their API survives mutual parties extending it. (There is a reason why namespaces exists)
  - Arathorn 11y ago
    http://matrix.org/docs/spec/#m-room-redaction http://matrix.org/docs/spec/#m-room-redaction is the current API for removing messages. Matrix uses namespaces for extension to survive mutual parties extending it (http://matrix.org/docs/spec/#events http://matrix.org/docs/spec/#events), and we haven't seen any problems there yet. That said, Matrix is still beta and we certainly do have a whole bunch of other problems :)
tbrownaw 11y ago
The correct technology is JSON-over-WebSockets, No, it's ASN.1 DER over TCP. :)
- wtbob 11y ago
  Or canonical S-expressions over TCP…
  - leni536 11y ago
    Or Bencode. Does anything use Bencode besides torrent files?
    - jallmann 11y ago
      I used Bencode to encode messages for a presence and messaging system. Easy to write parsers for, binary safe, worked well overall. Probably the only real issue was that the sizes of lists and dictionaries aren't known in advance. This actually is a benefit in some streaming contexts, but calls for caution when dealing with user supplied data. JSON has the same problem, for what it's worth.
  - davecridland 11y ago
    You'll be glad that someone got the IMAP reference... (Unless you're as niche as I, and were referring to ACAP).
    - wtbob 11y ago
      Actually, I was thinking of SPKI (my current nominee for 'best idea which failed'). I really do wish that canonical S-expressions had won over XML for data transfer (as opposed to publishing markup): much more readable, incredibly easy to parse, fairly lightweight. Really, nothing wrong with them except for some reason people seem to be allergic to parentheses. Why is that? Is it that one has to hit shift to type them on a standard keyboard?
      - 11y ago
davecridland 11y ago
The specific claims I was responding to when writing this (and it's still a draft) were an HTTP-based JSON protocol whose FAQ makes all these claims and more. And yeah, I think it's dumb to do things that way (both JSON/HTTP and making ill-informed and misleading comments in your FAQ). As to your comments: (1) I'm just not seeing this. Even without mobile-style push, apps like Conversations appear way down the list in Android. Note that because XMPP maintains a continuous TCP session (or WebSocket, or whatever), then you get "push" in a general sense; that is, there is no polling. Using an XMPP client on your mobile will, of course, have an impact on battery life, but I don't think a good client will drain your battery. For me, Conversations uses less power than Solitaire according to Android. (2) I'll agree that XML does make the development of XMPP libraries more complex, but we gain because it grants very simple permissionless extension capability, and that's hugely valuable. If developers working with (rather than on) XMPP are having to hit XML directly, that is absolutely a problem. Finally, resources are the way they are for all sorts of reasons, but Message Carbons really does work for having conversation sync across multiple devices. I often walk away from my desktop and continue conversations on my mobile and/or tablet, and wander back to the desktop when I feel the need to continue with a keyboard. Hope that helps.
- infie 11y ago
  Mind sharing which desktop client you use (and on which OS)?
  - davecridland 11y ago
    I use Gajim on Linux and Conversations on Android, but the technology you're looking for is "XEP-0280: Message Carbons".
- joepie91_ 11y ago
  > I'm just not seeing this. Even without mobile-style push, apps like Conversations appear way down the list in Android. Note that because XMPP maintains a continuous TCP session (or WebSocket, or whatever), then you get "push" in a general sense; that is, there is no polling. > Using an XMPP client on your mobile will, of course, have an impact on battery life, but I don't think a good client will drain your battery. For me, Conversations uses less power than Solitaire according to Android. I don't know how accurate Android's power estimates are, but I consistently have people telling me that they are switching back to proprietary messengers because every XMPP client they've tried drained their battery. Whatever causes it, there's a real problem here. > (2) I'll agree that XML does make the development of XMPP libraries more complex, but we gain because it grants very simple permissionless extension capability, and that's hugely valuable. If developers working with (rather than on) XMPP are having to hit XML directly, that is absolutely a problem. The problem is that theoretical future benefits are worthless if you never get a base implementation going. XMPP is just too hard for many developers - it's not a thing that you're going to understand well unless you work on it full-time, and that's not acceptable for a protocol. Hitting XML directly is almost always a requirement. Libraries are woefully incomplete due to the large number of XEPs, and to implement anything yourself, you must understand the XML structure. If the intention of XMPP or XMPP frameworks was to abstract away the XML, then they have failed. > Finally, resources are the way they are for all sorts of reasons, but Message Carbons really does work for having conversation sync across multiple devices. I often walk away from my desktop and continue conversations on my mobile and/or tablet, and wander back to the desktop when I feel the need to continue with a keyboard. I experience constant issues with OTR, most servers don't seem to even support message carbons (nor do major clients like Pidgin have any useful indications of it), and frankly, I have never seen a single 'regular' user get synchronization set up correctly. This is really a pervasive problem in the XMPP ecosystem: everything is too hard. Little to no thought seems to have gone into the UX, and unless you are an 'experienced XMPP user', you really have no clue what is going on. The experienced users say "meh, works for me" and don't set out to fix it. This is true for the frameworks, the clients, and most of the servers. It's just all-around user-hostile, and that needs to change.
Flowdalic 11y ago
> 1. XMPP is not battery-friendly on mobile devices (this primarily has/had to do with the absence of push notifications, it seems). I do use XMPP in a mobile environment without any noticeable impact on battery runtime. It's not the protocol which drains your battery, but how one uses it: If there is frequent activity in terms of XMPP stanzas being exchanged between your device and the device's XMPP server, then the device will be unable to e.g. put the radio to sleep. Which, in turn, is of course causing battery drain. The golden rules to prevent that: 1. Only send XMPP stanzas when necessary 2. Bundle and defer outgoing stanzas if possible 3. Tell your XMPP service when the device is inactive, so that the service is able to optimize the traffic (XEP-352: Client State Indiation [1]) 4. Consider service-side stanza blocking (e.g. XEP-16: Privacy Lists) for stanzas of "unknown" origin to prevent malicious users from draining your battery. 1: https://xmpp.org/extensions/xep-0352.html https://xmpp.org/extensions/xep-0352.html
- joepie91_ 11y ago
  What are your thoughts on http://op-co.de/blog/posts/mobile_xmpp_in_2014/ http://op-co.de/blog/posts/mobile_xmpp_in_2014/ ?
  - Flowdalic 11y ago
    tl;dr It's being worked on (or already solved), but I'm sure the XMPP community welcomes additional contributors. 1-3. Could be solved with upcoming versions of XEP-313 MAM. 4. e2e There is currently a XSF GSOC project running, bringing axolotl to Conversations (and eventually describing the protocol as an XEP). 5. There are a few standards on how to perform encrypted file transfers, but implementations are lacking. You are invited to code one. 6. XEP-357: https://xmpp.org/extensions/xep-0357.html https://xmpp.org/extensions/xep-0357.html
zamalek 11y ago
> XMPP is not battery-friendly on mobile devices (this primarily has/had to do with the absence of push notifications, it seems). There is actually an unbelievably good XEP (XMPP extension protocol) on mobile battery life.[1] I learned things from it: radio power states and so forth. It's incredibly detailed and educational and well worth the read even if you don't use XMPP. That being said, so far as actual implementations go, I have to agree with you. > XML is unnecessarily complex for this kind of usecase, and too hard and time-consuming to work with (and I agree). That is most likely what 'slow' refers to - not slow in parsing it, but in developing for it. I have an ever-evolving conformant XMPP server pet project. What I found is that the framework takes bloody ages and I have to agree with you there. However, XMPP is a protocol that is ripe for elegant architecture and you can make a great framework. If you spend the time making a good framework the protocol is an absolute pleasure to develop against: but you have to make that initial time expenditure. > The correct technology is JSON-over-WebSockets, and it doesn't really make sense to compare XMPP to a known-bad implementation and say "see? XMPP isn't that bad!" Sort of. XMPP is one of the few applications that actually really makes use of the "eXtensible" bit in XML (where it's often misused as a dumb object graph). This means that you can do something as follows: <message from='...' to='...'> <body>You have work, go to http://... to action it.</body> <form xmlns='urn:my-proprietary-stuff'>AF3BD2</form> </message> Any client that doesn't understand the <form> element would ignore it and present the message (allowing the user to click-through to the web UI). A client that does understand it would be able to do the special proprietary logic. Servers are also required (AFAIR) to pass on that unknown XML unaltered. You'd at least have a few problems in JSON due to name conflicts - which you could work around. Either way, XMPP inherits a lot of traits from XML. You either need it or you don't and if you are using it there is a good chance that you are misusing it. [1]: http://xmpp.org/extensions/xep-0286.html http://xmpp.org/extensions/xep-0286.html
- joepie91_ 11y ago
  JSON name conflicts are easily resolved by giving every extension its own key in the object. That's the equivalent of namespaces.
  - ralphm 11y ago
userbinator 11y ago
XML is unnecessarily complex but I think JSON also is. IRC and MSNP (the early versions, before they started XML-ifying everything) are what I'd consider a good format protocol for the application - a mostly line-based, textual one you can use pretty easily without any client more complex than netcat. They are also extensible enough even without XML. (Personally, I think it is absurd that a protocol needs to wrap a single text message, the most common form of message that would be encountered, in several times more bytes than the message itself.)
- joepie91_ 11y ago
  IRC has major issues, actually - notably, it's very tricky to parse correctly (and very easy to think you're parsing it correctly when you aren't), and doesn't support more than one space-containing value per 'line'. It's also implemented differently by every single server. You will need some kind of serialization format for a reliable message protocol. JSON is just the simplest possible format that can reasonably represent common data, and works in 95% of the usecases.
  - slasaus 11y ago
    I personally found BSON* (or maybe protobuf) a better format for a wire protocol. I really miss support for a date type and binary data in JSON (for my own applications). * http://bsonspec.org/ http://bsonspec.org/
    - joepie91_ 11y ago
      msgpack[1] or LEON[2] might be more useful alternatives, in that case, depending on your interoperability requirements. [1] http://msgpack.org/ http://msgpack.org/ [2] https://github.com/raypulver/LEON https://github.com/raypulver/LEON
  - amyjess 11y ago
    I'm fond of Avro. It has both a compact binary format and a human-readable plaintext format that's nearly identical to JSON (the only difference being that order of fields is significant in Avro but insignificant in JSON), with tools to convert between them.
haddr 11y ago
JSON over HTTP is not that bad concept. With the HTML5 server-sent events, this becomes a really cool stuff. And surprisingly it is working very well. For developers this is an order of magnitude easier and faster to develop rather than using low-level websockets which is reinventing the wheel.
hobarrera 11y ago
> 1. XMPP is not battery-friendly on mobile devices (this primarily has/had to do with the absence of push notifications, it seems). I never really understood this complaint. Why are the notifications sent via XMPP's TCP channel not considered "push"? How do you expect to get push notifications without an open TCP connection anyway?
- joepie91_ 11y ago
  The idea is that push notifications are handled by a single platform vendor, and you thus only have one TCP connection to maintain for all applications together, instead of a separate connection for each application. It's a form of batching.
  - Zash 11y ago
    Read: Route everything trough one vendors servers. Instead of something like a network traffic scheduler or a signal that the radio is on and now is a good time to send stuff.

ape4 11y ago

> absence of push If you are connected and your buddy sends a message you get it right away.

niklasni1 11y ago
On mobile you want messages to come through without each app maintaining a connection.
- emias 11y ago
  Because?
  - joepie91_ 11y ago
    Because TCP connections are expensive on your battery life. Especially if your network is flaky, and all those connections need to constantly repeat their handshakes.
    - ultramancool 11y ago
      Not really, TCP connections are how GCM does things even. Unless you're using keep alives or otherwise preventing sleep, I believe these devices are capable of sending an interrupt when there's activity on a network connection though I could be mistaken.
    - Flowdalic 11y ago
      Idle TCP connections do not consume any battery. And on a data connectivity change, e.g. GSM <-> WiFi switch, there is a good chance that a dozen other components start trying to re-establish their connections, which means the radio is awake anyway (for, usually, at most a few seconds).
      - rakoo 11y ago
        > Idle TCP connections do not consume any battery. An idle connection doesn't consume anything, but a useful idle connection will have some keepalive every X minutes. Multiply that by the number of connections your application will have and by the number of background application you run on your phone, and the radio will never truly sleep or enter the "low" mode. The solution wanted by Google and Apple is to maintain a single connection to their server on which every push is aggregated and sent, and that the OS itself manages.
        Flowdalic 11y ago
        I use my mobile XMPP connections without a TCP keepalive but send a server ping if there has been stanzas received in the last 30 minutes and get a useful and reliable XMPP connection without a noticeable impact on battery. If you use Android's AlarmManager to trigger the check, then Android will even take care of scheduling the "alarm" with other alarms for efficiency. > Multiply that by the number of connections your application will have and by the number of background application you run on your phone, and the radio will never truly sleep or enter the "low" mode. Why would X x Y idle connections (X: TCP connections per app, Y: applictions) prevent your radio from sleeping?
        11y ago
  - rakoo 11y ago
    Because iOS doesn't let you do that (see http://op-co.de/blog/posts/mobile_xmpp_in_2014/#index6h2 http://op-co.de/blog/posts/mobile_xmpp_in_2014/#index6h2)
    - pjc50 11y ago
      This is the real answer - long-standing connections are banned by the platform monopolist - and should be much closer to the top.
    - emias 11y ago
      Yes, but I think the claim was that a persistent TCP connection has a negative effect on battery life, which is not obvious to me. An idle connection should have no impact whatsoever.

sz4kerto 11y ago

> XML is unnecessarily complex for this kind of usecase, and too hard and time-consuming to work with (and I agree) What is 'difficult' in XML?

inversionOf 11y ago
The notion that XML is difficult comes from the most profoundly lazy evaluations possible, where the most minimal of up front understanding are effort are eschewed if you can just launch mongodb and insert some giant mystery meat bags of json. We've seen this same mistake play out over, and over, and over again, and it is incredible that in a field where projects often end up taking thousands or more of man hours, people sweat and complain about 15 minutes up front gaining a basic understanding of a structure or API, and instead pursue solutions that, for instance, doesn't even have a date type or even consistent hack standard of how to handle it. This is how we have the rapid trends in this field, where the new panacea is tomorrow's nightmare.
- stonogo 11y ago
  This is the most breathtakingly ironic defense of XML I have ever seen.
  - inversionOf 11y ago
    In what way is my post "ironic"? I would dearly love to know. Do you think my defense of XML is somehow that I am too lazy to know alternatives. You would be painfully wrong, dear friend, though such is a go to among the intellectually lazy (how many failed NoSQL projects were justified on notions of "people who criticize elements of it are just old and fear change". It's the laziest, most unprofessional retort possible, and fueled about a year of content on Hacker News, and was the meat of a million failed adventures) My post will be pummeled by the defensive, and that's okay.
    - stonogo 11y ago
      All of your invective was exactly what programmers were saying to XML proponents in the early days of XML. I still can't entirely tell if you're deliberately doing this or if you really are just wholly unaware of yourself. > The notion that binary serialization is difficult comes from the most profoundly lazy evaluations possible, where the most minimal of up front understanding are effort are eschewed if you can just open a text file and insert some giant mystery meat bags of XML. etc etc

vog 11y ago

While I agree that XMPP is not the future for app-to-app communication, I do so for a slightly different reasons. > XML is unnecessarily complex for this kind of usecase XMPP is too complex, but the XML stuff is really the smallest issue, compared with other complexities in the protocol. For example, every client that successfully connected to an XMPP server must first send a "<presence>" message. Otherwise, it won't receive any messages despite their successful connection. And there are more such surprises in XMPP. Good luck in understanding which "devices" (#xxxx) of the same Jabber ID will receive a message targeted at that ID. It's really annoying and discouraging to debug that kind of issues - especially since these aren't bugs. These work as specified, but the specifications were obviously designed with direct human chatting or direct human collaboration in mind, programs being an afterthought. > The correct technology is JSON-over-WebSockets Not really. Whenever I saw someone trying this "in the wild", I see them quickly switching to HTTP long-polling. (Not to be confused with polling! We are still talking about instant push notifications with very low network overhead.) For example, there are some libraries that used to provide wrappers around WebSockets, which switched to HTTP long-polling as fallback for old browsers. Nowadays, it seems they use long-polling by default, even though browser support becomes better and better. Don't underestimate the role of firewalls and HTTP proxies! Many firewalls are aggressive configured, and many companies have forced HTTP proxies in place.

joepie91_ 11y ago
That is a browser issue, and does not exist in native clients. If we're talking about XML-stream-over-sockets, then comparing it to browsers makes no sense (because they can't do that!), and thus long-polling and JSON-over-HTTP become irrelevant. EDIT: Hmm, did you edit your post? I don't recall the first part being there... anyhow: > XMPP is too complex, but the XML stuff is really the smallest issue, compared with other complexities in the protocol. That is one factor, but the protocol complexities often seem to derive from the question 'how do we represent this in XML?', and that's an issue. Another problem is the jargon - the MUC specification is utterly incomprehensible, because the terms it uses for things are so wildly different from how they are used everywhere else, that building a mental model is really hard. I've seen this issue in other XEPs as well.
Flowdalic 11y ago

rakoo 11y ago

> I also don't really understand the comparison with "JSON over HTTP" One of the complainers is matrix (https://matrix.org/ https://matrix.org/), which does JSON over HTTP.

treffer 11y ago
Which has it's own set of problems, btw. Try to find an API for "delete". Oh, and I'm eager to see how their API survives mutual parties extending it. (There is a reason why namespaces exists)
- Arathorn 11y ago
  http://matrix.org/docs/spec/#m-room-redaction http://matrix.org/docs/spec/#m-room-redaction is the current API for removing messages. Matrix uses namespaces for extension to survive mutual parties extending it (http://matrix.org/docs/spec/#events http://matrix.org/docs/spec/#events), and we haven't seen any problems there yet. That said, Matrix is still beta and we certainly do have a whole bunch of other problems :)

tbrownaw 11y ago

The correct technology is JSON-over-WebSockets, No, it's ASN.1 DER over TCP. :)

wtbob 11y ago
Or canonical S-expressions over TCP…
- leni536 11y ago
  Or Bencode. Does anything use Bencode besides torrent files?
  - jallmann 11y ago
    I used Bencode to encode messages for a presence and messaging system. Easy to write parsers for, binary safe, worked well overall. Probably the only real issue was that the sizes of lists and dictionaries aren't known in advance. This actually is a benefit in some streaming contexts, but calls for caution when dealing with user supplied data. JSON has the same problem, for what it's worth.
- davecridland 11y ago
  You'll be glad that someone got the IMAP reference... (Unless you're as niche as I, and were referring to ACAP).
  - wtbob 11y ago
    Actually, I was thinking of SPKI (my current nominee for 'best idea which failed'). I really do wish that canonical S-expressions had won over XML for data transfer (as opposed to publishing markup): much more readable, incredibly easy to parse, fairly lightweight. Really, nothing wrong with them except for some reason people seem to be allergic to parentheses. Why is that? Is it that one has to hit shift to type them on a standard keyboard?
    - 11y ago

davecridland 11y ago

The specific claims I was responding to when writing this (and it's still a draft) were an HTTP-based JSON protocol whose FAQ makes all these claims and more. And yeah, I think it's dumb to do things that way (both JSON/HTTP and making ill-informed and misleading comments in your FAQ). As to your comments: (1) I'm just not seeing this. Even without mobile-style push, apps like Conversations appear way down the list in Android. Note that because XMPP maintains a continuous TCP session (or WebSocket, or whatever), then you get "push" in a general sense; that is, there is no polling. Using an XMPP client on your mobile will, of course, have an impact on battery life, but I don't think a good client will drain your battery. For me, Conversations uses less power than Solitaire according to Android. (2) I'll agree that XML does make the development of XMPP libraries more complex, but we gain because it grants very simple permissionless extension capability, and that's hugely valuable. If developers working with (rather than on) XMPP are having to hit XML directly, that is absolutely a problem. Finally, resources are the way they are for all sorts of reasons, but Message Carbons really does work for having conversation sync across multiple devices. I often walk away from my desktop and continue conversations on my mobile and/or tablet, and wander back to the desktop when I feel the need to continue with a keyboard. Hope that helps.

infie 11y ago
Mind sharing which desktop client you use (and on which OS)?
- davecridland 11y ago
  I use Gajim on Linux and Conversations on Android, but the technology you're looking for is "XEP-0280: Message Carbons".
joepie91_ 11y ago
> I'm just not seeing this. Even without mobile-style push, apps like Conversations appear way down the list in Android. Note that because XMPP maintains a continuous TCP session (or WebSocket, or whatever), then you get "push" in a general sense; that is, there is no polling. > Using an XMPP client on your mobile will, of course, have an impact on battery life, but I don't think a good client will drain your battery. For me, Conversations uses less power than Solitaire according to Android. I don't know how accurate Android's power estimates are, but I consistently have people telling me that they are switching back to proprietary messengers because every XMPP client they've tried drained their battery. Whatever causes it, there's a real problem here. > (2) I'll agree that XML does make the development of XMPP libraries more complex, but we gain because it grants very simple permissionless extension capability, and that's hugely valuable. If developers working with (rather than on) XMPP are having to hit XML directly, that is absolutely a problem. The problem is that theoretical future benefits are worthless if you never get a base implementation going. XMPP is just too hard for many developers - it's not a thing that you're going to understand well unless you work on it full-time, and that's not acceptable for a protocol. Hitting XML directly is almost always a requirement. Libraries are woefully incomplete due to the large number of XEPs, and to implement anything yourself, you must understand the XML structure. If the intention of XMPP or XMPP frameworks was to abstract away the XML, then they have failed. > Finally, resources are the way they are for all sorts of reasons, but Message Carbons really does work for having conversation sync across multiple devices. I often walk away from my desktop and continue conversations on my mobile and/or tablet, and wander back to the desktop when I feel the need to continue with a keyboard. I experience constant issues with OTR, most servers don't seem to even support message carbons (nor do major clients like Pidgin have any useful indications of it), and frankly, I have never seen a single 'regular' user get synchronization set up correctly. This is really a pervasive problem in the XMPP ecosystem: everything is too hard. Little to no thought seems to have gone into the UX, and unless you are an 'experienced XMPP user', you really have no clue what is going on. The experienced users say "meh, works for me" and don't set out to fix it. This is true for the frameworks, the clients, and most of the servers. It's just all-around user-hostile, and that needs to change.

Flowdalic 11y ago

> 1. XMPP is not battery-friendly on mobile devices (this primarily has/had to do with the absence of push notifications, it seems). I do use XMPP in a mobile environment without any noticeable impact on battery runtime. It's not the protocol which drains your battery, but how one uses it: If there is frequent activity in terms of XMPP stanzas being exchanged between your device and the device's XMPP server, then the device will be unable to e.g. put the radio to sleep. Which, in turn, is of course causing battery drain. The golden rules to prevent that: 1. Only send XMPP stanzas when necessary 2. Bundle and defer outgoing stanzas if possible 3. Tell your XMPP service when the device is inactive, so that the service is able to optimize the traffic (XEP-352: Client State Indiation [1]) 4. Consider service-side stanza blocking (e.g. XEP-16: Privacy Lists) for stanzas of "unknown" origin to prevent malicious users from draining your battery. 1: https://xmpp.org/extensions/xep-0352.html https://xmpp.org/extensions/xep-0352.html

joepie91_ 11y ago
What are your thoughts on http://op-co.de/blog/posts/mobile_xmpp_in_2014/ http://op-co.de/blog/posts/mobile_xmpp_in_2014/ ?
- Flowdalic 11y ago
  tl;dr It's being worked on (or already solved), but I'm sure the XMPP community welcomes additional contributors. 1-3. Could be solved with upcoming versions of XEP-313 MAM. 4. e2e There is currently a XSF GSOC project running, bringing axolotl to Conversations (and eventually describing the protocol as an XEP). 5. There are a few standards on how to perform encrypted file transfers, but implementations are lacking. You are invited to code one. 6. XEP-357: https://xmpp.org/extensions/xep-0357.html https://xmpp.org/extensions/xep-0357.html

zamalek 11y ago

> XMPP is not battery-friendly on mobile devices (this primarily has/had to do with the absence of push notifications, it seems). There is actually an unbelievably good XEP (XMPP extension protocol) on mobile battery life.[1] I learned things from it: radio power states and so forth. It's incredibly detailed and educational and well worth the read even if you don't use XMPP. That being said, so far as actual implementations go, I have to agree with you. > XML is unnecessarily complex for this kind of usecase, and too hard and time-consuming to work with (and I agree). That is most likely what 'slow' refers to - not slow in parsing it, but in developing for it. I have an ever-evolving conformant XMPP server pet project. What I found is that the framework takes bloody ages and I have to agree with you there. However, XMPP is a protocol that is ripe for elegant architecture and you can make a great framework. If you spend the time making a good framework the protocol is an absolute pleasure to develop against: but you have to make that initial time expenditure. > The correct technology is JSON-over-WebSockets, and it doesn't really make sense to compare XMPP to a known-bad implementation and say "see? XMPP isn't that bad!" Sort of. XMPP is one of the few applications that actually really makes use of the "eXtensible" bit in XML (where it's often misused as a dumb object graph). This means that you can do something as follows: <message from='...' to='...'> <body>You have work, go to http://... to action it.</body> <form xmlns='urn:my-proprietary-stuff'>AF3BD2</form> </message> Any client that doesn't understand the <form> element would ignore it and present the message (allowing the user to click-through to the web UI). A client that does understand it would be able to do the special proprietary logic. Servers are also required (AFAIR) to pass on that unknown XML unaltered. You'd at least have a few problems in JSON due to name conflicts - which you could work around. Either way, XMPP inherits a lot of traits from XML. You either need it or you don't and if you are using it there is a good chance that you are misusing it. [1]: http://xmpp.org/extensions/xep-0286.html http://xmpp.org/extensions/xep-0286.html

joepie91_ 11y ago
JSON name conflicts are easily resolved by giving every extension its own key in the object. That's the equivalent of namespaces.
- ralphm 11y ago

userbinator 11y ago

XML is unnecessarily complex but I think JSON also is. IRC and MSNP (the early versions, before they started XML-ifying everything) are what I'd consider a good format protocol for the application - a mostly line-based, textual one you can use pretty easily without any client more complex than netcat. They are also extensible enough even without XML. (Personally, I think it is absurd that a protocol needs to wrap a single text message, the most common form of message that would be encountered, in several times more bytes than the message itself.)

joepie91_ 11y ago
IRC has major issues, actually - notably, it's very tricky to parse correctly (and very easy to think you're parsing it correctly when you aren't), and doesn't support more than one space-containing value per 'line'. It's also implemented differently by every single server. You will need some kind of serialization format for a reliable message protocol. JSON is just the simplest possible format that can reasonably represent common data, and works in 95% of the usecases.
- slasaus 11y ago
  I personally found BSON* (or maybe protobuf) a better format for a wire protocol. I really miss support for a date type and binary data in JSON (for my own applications). * http://bsonspec.org/ http://bsonspec.org/
  - joepie91_ 11y ago
    msgpack[1] or LEON[2] might be more useful alternatives, in that case, depending on your interoperability requirements. [1] http://msgpack.org/ http://msgpack.org/ [2] https://github.com/raypulver/LEON https://github.com/raypulver/LEON
- amyjess 11y ago
  I'm fond of Avro. It has both a compact binary format and a human-readable plaintext format that's nearly identical to JSON (the only difference being that order of fields is significant in Avro but insignificant in JSON), with tools to convert between them.

haddr 11y ago

JSON over HTTP is not that bad concept. With the HTML5 server-sent events, this becomes a really cool stuff. And surprisingly it is working very well. For developers this is an order of magnitude easier and faster to develop rather than using low-level websockets which is reinventing the wheel.

hobarrera 11y ago

> 1. XMPP is not battery-friendly on mobile devices (this primarily has/had to do with the absence of push notifications, it seems). I never really understood this complaint. Why are the notifications sent via XMPP's TCP channel not considered "push"? How do you expect to get push notifications without an open TCP connection anyway?

joepie91_ 11y ago
The idea is that push notifications are handled by a single platform vendor, and you thus only have one TCP connection to maintain for all applications together, instead of a separate connection for each application. It's a form of batching.
- Zash 11y ago
  Read: Route everything trough one vendors servers. Instead of something like a network traffic scheduler or a signal that the radio is on and now is a good time to send stuff.