5 ms·

XMPP For the Web with JSON

44 points by julien 13y ago

jacobolus 13y ago
I don’t understand the point of taking an existing protocol – with fairly well defined syntax/semantics, which has presumably been tested at various parts of the stack, likely has existing conformance tests, &c. – and wrapping it in a new ad-hoc protocol which has (some subset of roughly) the same semantics, just for a syntax change? Big addition of new complexity for little tangible benefit. Why not just use one of the existing Javascript implementations of an XMPP client, or if they’re unsuitable for whatever reason, make a new one?
- jkarneges 13y ago
  Lately there's been discussion within the XMPP community about how web developers tend to prefer semi-proprietary connectivity solutions like Socket.io even though the XMPP standard and surrounding solutions predate these kinds of things. We believe a large part of the problem is the encoding format. For better or worse, the web (and I mean all of it, not just browser<->webserver side) is transitioning to JSON for just about everything. Yes, it often seems like pointless wheel reinvention, but that's what is happening. May as well embrace the trend in order for our work to live on. One funny thing about all this is that when XMPP was first created (as "Jabber" in 1999), XML was selected as the format because it was the current trend. In that sense XMPP was super hip in the beginning. Hard to believe now people look at XML and think it's a relic. With that in mind, I wonder what will happen to all these trendy JSON protocols being made today? Dinosaurs of tomorrow, probably. My feeling is that it's the underlying concepts that will live on, regardless of format.
  - spin 13y ago
    I don't want to start some kind of "XML vs JSON" holy war, but I really do prefer JSON over XML because of how compact and readable it is. I really don't see JSON as a passing trend, I see it as (almost) the most compact, succint way to express hashmaps in both human-readable and machine-readable form. (XML is big, bloated, and bureaucratic.) (It's tragic that Crockford left in all the qoutes around the keys, though. We really don't need to type those quotes.) For the past couple of years, I really have been wishing for an "XMPP, but built on JSON, not XML". Really. I want this.
    - deno 13y ago
      > For the past couple of years, I really have been wishing for an "XMPP, but built on JSON, not XML". Really. I want this. The X in XMPP stands for extensible not XML. The reason XML fits the bill is because it has concept of namespaces, what JSON has? If you don’t have explicit namespace support how do you make your JSON variant of XMPP forward‐compatible? You don’t without making it look much, much, worse than any of the examples I’m about to show below. > (…) I really do prefer JSON over XML because of how compact and readable it is. > XML is big, bloated, and bureaucratic. My standard response to this is always “show me a JSON fragment that is shorter or prettier looking than an equivalent XML fragment.” XML can certainly be shorter than JSON and often is, and repeated tags are the best showcase for it: <user id="abc"> <phoneNo type="home">123456789</phoneNo> <phoneNo type="work">321654987</phoneNo> </user> This turns into this beautiful JSON: [ "users": [ { "id": "abc", "phoneNos": [ { "type": "home", "value": "123456789" }, { "type": "work", "value": "321654987" } ] } ] ] Oh, but what if order is important? Take this example: <policy id="123"> <drop someFilter="ghi" /> <accept someFilter="abc" anotherFilter="xyz"> <comment> This comment is quite long and spans several lines of text in pretty printed XML. But this is quite alright, because XML by default strips extra whitespace. </comment>  </accept> <drop someFilter="foo" anotherFilter="bar"> <reason statusCode="690">No reason</reason> </drop> </policy> Now you have to do it like that: [ "policies": [ { "id": "123", "rules": [ { "type": "drop", "someFilter": "ghi" }, { "type": "accept", "someFilter": "abc", "anotherFilter": "xyz", "comment": "Unfortunately the best you can do with JSON is to rely on your editor’s dynamic wrapping. And this is how you break the line: \nWhat a treat!", }, { "type": "drop", "someFilter": "foo", "anotherFilter": "bar", "reason": { "statusCode": 690, "message": "No reason"} } ] } ] ] For an example of how horrible JSON in wild can really be see Reddit’s JSON API. Because JSON has no syntax for metadata you get things like that: [ { "kind": "thing-foo", "items": [ # # list of comments or something that goes on for # two screens # ], "votes": 15 } ] See the keys are sorted for convenience. XML’s text representation is very minimal and consistent. You don’t need to quote Every Single Thing, and there are no special cases for not quoting just some type of values. The only redundant part of XML is repeating the name of the closing tag if the </element> has value. But this actually aids readability, as demonstrated above. Editing complex JSON by hand with a regular text editor is a nightmare, because of all those single '}', ']' and commas trailing for several lines when there’s any light nesting involved. Editing XML by hand on the other hand is no big deal.
- jakerocheleau 13y ago
  I don't have an answer but I do have a follow-up question. There are no pre-built GChat clients and I have been planning to create one using Phono, which is built on jQuery. Are there better solutions other than this, or the linked example Superfeedr? Google Chat is my main IM client and I know it's built on XMPP. I just want to pick a solid JS library to build with and make a web interface that can work for people.
  - shmerl 13y ago
    You can also use something like Strophe.js to build your own client. Or you want to add an XMPP/Jingle support as well? How exactly would it work, through routing Jingle over WebRTC? Update: I just looked at Phono - it doesn't seem like it supprots XMPP/Jingle, even though it uses WebRTC. So this kind of defeats the purpose, since you can't initiate audio/video calls to other XMPP clients. Can you advise any combinations of JavaScript client libraries and XMPP servers (i.e. without a need to use third party gateways) which can already normally support XMPP over WebSockets and Jingle over WebRTC?
    - mweibel 13y ago
      XMPP over Websockets is implemented in various servers: - Prosody (http://code.google.com/p/prosody-modules/wiki/mod_websocket) - openfire (http://code.google.com/p/openfire-websockets/) - Tigase (built-in as far as I could see) - Ejabberd in the Commercial version, the opensource version doesn't really have one yet. Regarding Jingle over WebRTC: Various people are working on it (or have been, in my case) and it's getting closer to workable. What I encountered, when I tried ~a year ago to do it, was that not all SDP messages could be easily transformed into the Jingle format. Might have been changed now though as the WebRTC Specs have changed quite a bit.
      - shmerl 13y ago
        Thanks for the pointers. I hope open source ejabberd will catch up, since I prefer it in general. Are any of the Jingle through WebRTC projects public to track their progress?
- shmerl 13y ago
  I agree. It's better to help improving current XMPP libraries for good support of WebSockets and etc. instead of bringing JSON into the picture.
shmerl 13y ago
What is so scary in the XML?? I didn't get the point of this.
- hayksaakian 13y ago
  Personally, it's just another ML I need to think about. If it's JSON, I can at least pretend that my data structure stays coherent from A to B (ex: node.js webapp to js frontend). It's also easy to look at, appearing much like a dictionary or hash + array set in many programming languages.
  - shmerl 13y ago
    XML is a "must know" kind of thing for any serious developer anyway, so I don't really see your point. XML and JSON have enough different use cases, even though they have common ones as well. For starters, XML can be automatically validated when needed using schemas, while JSON can't (even though there is a draft for JSON schema in the works). So XML has advantages over JSON is various scenarios. The price is more complexity naturally, but one should expect that. But besides the point, XMPP already uses XML. So there is no need to reinvent the wheel here.
    - Arelius 13y ago
      While, mostly working in client development, I'm a professional developer of 12 years, and while I've had my random encounter with XML, I've managed to avoid it, and do not "know" XML in any significant capacity. It's easy to forget that there is a large set of different developers. And some don't know what you may consider common knowledge. IMO, something being JSON over XML would be a significant plus in my mind.
    - hownowstephen 13y ago
      Working for a company that has started adopting JSON Schema as our primary validation tool, I can definitely agree that XML still has the upper hand when it comes to a seamless experience in data validation. Schema validation is still really primitive (though with the love JSON has been getting in the past couple years, it shouldn't be too much longer until it's in fighting shape). Something about XML just has never really sat well with me - whereas JSON just works when it comes to actually modelling objects in a way that makes sense and can be picked up quickly. I can't say I've yet to find a place where I absolutely needed a feature of XML that I couldn't find in JSON, but would love to know more situations where I can be proven wrong. I think ultimately if the goal is just to JSONify the world, then that's no good - but if we're looking to make protocols more accessible and hackable, JSON may be the way to go.
jerf 13y ago
Is it XEP-0295 compliant? http://xmpp.org/extensions/xep-0295.html http://xmpp.org/extensions/xep-0295.html
- X-Istence 13y ago
  All this does is wrap all of the XML in a {"s": "<stuff here>"}. That is useless.
- mweibel 13y ago
  XEP-0295 is humorous (1st april joke).
  - jerf 13y ago
    Really? Is that what the green text up top meant?
lloydwatkin 13y ago
Hi all, As the author of xmpp-ftw I feel I should throw some comments into this discussion. Firstly, xmpp-ftw is not about starting an XML/JSON holy war, its about lowering the barrier to entry as far as possible. So see it as more of a gateway drug :) * At present the tiny amount of effort it takes to build a proprietary (legacy) chat system with other technologies means that XMPP is somewhat being left behind (on the web) despite offering a huge number of benefits. Xmpp-ftw tries to make it as easy as possible by providing developers a format they are comfortable with and the ability to send/receive data with the minimum of fuss. This project actually came out of discussions at XSF summits I've been attending. No-one (myself included) believes that XML should be replaced with JSON in XMPP, that would be a foolish notion. Generic JSON <-> XML is awful and doesn't quite work. The idea with this project was to used named events to fill in some of the missing information for building stanzas, and telling the user what is happening. The view of JSON with XMPP has changed over the last couple of summits has changed, going from being referred to as the J word' through to open discussions about what can be done (the XSF seems like a very progressive group). No more jokes about XEP-0295 :) If you want to see a better client library trying to do something similar check out Lance Stout's stanza.io https://github.com/legastero/stanza.io https://github.com/legastero/stanza.io which came out of the same meeting. So in summary: Xmpp-ftw is attempting to drop the barrier to entry for XMPP as far as possible and make it quick and convenient for developers to implement. Its not about replacing the angle brackets :) Cheers, Lloyd FYI, yes I've used socket.io, but you can stick anything that implements an event emitter and has callbacks and have it work. Currently I'm looking at wrapping sockJS. * Drugs are bad m'kay!
ashward 13y ago
There are a couple of key points here which a lot of you are missing which are: * The JSON -> XML conversion is done on the server rather than in the client. Browsers are surprisingly bad at xml (one of the reasons buddycloud chose to move the xmpp layer out of the browser), and xml dom manipulation can make for overly complex code (we should also bear in mind one of the philosophies of xmpp which was simple clients, complex servers), and this leads me to my second point: * It's really simple. This opens it up as a no-brainer solution for web devs who would otherwise just throw up a proprietary silo system based purely on socket.io or similar. Sure, if you really want all the complexities and nuances of xmpp then use an xml based library, but if you just want to send simple messages between users then this is really simple, and hey - what would have been a silo system is suddenly federatable at zero cost. The point here is that this library is not aimed at hardcore xmpp devs, but devs who would be put off by anything other than json. Make this library as easy to use as socket.io on its own and suddenly you've got more systems running on xmpp which wouldn't otherwise have been (which is great for everyone)!

44 points by julien 13y ago

I don’t understand the point of taking an existing protocol – with fairly well defined syntax/semantics, which has presumably been tested at various parts of the stack, likely has existing conformance tests, &c. – and wrapping it in a new ad-hoc protocol which has (some subset of roughly) the same semantics, just for a syntax change? Big addition of new complexity for little tangible benefit. Why not just use one of the existing Javascript implementations of an XMPP client, or if they’re unsuitable for whatever reason, make a new one?

jkarneges 13y ago
Lately there's been discussion within the XMPP community about how web developers tend to prefer semi-proprietary connectivity solutions like Socket.io even though the XMPP standard and surrounding solutions predate these kinds of things. We believe a large part of the problem is the encoding format. For better or worse, the web (and I mean all of it, not just browser<->webserver side) is transitioning to JSON for just about everything. Yes, it often seems like pointless wheel reinvention, but that's what is happening. May as well embrace the trend in order for our work to live on. One funny thing about all this is that when XMPP was first created (as "Jabber" in 1999), XML was selected as the format because it was the current trend. In that sense XMPP was super hip in the beginning. Hard to believe now people look at XML and think it's a relic. With that in mind, I wonder what will happen to all these trendy JSON protocols being made today? Dinosaurs of tomorrow, probably. My feeling is that it's the underlying concepts that will live on, regardless of format.
- spin 13y ago
  I don't want to start some kind of "XML vs JSON" holy war, but I really do prefer JSON over XML because of how compact and readable it is. I really don't see JSON as a passing trend, I see it as (almost) the most compact, succint way to express hashmaps in both human-readable and machine-readable form. (XML is big, bloated, and bureaucratic.) (It's tragic that Crockford left in all the qoutes around the keys, though. We really don't need to type those quotes.) For the past couple of years, I really have been wishing for an "XMPP, but built on JSON, not XML". Really. I want this.
  - deno 13y ago
    > For the past couple of years, I really have been wishing for an "XMPP, but built on JSON, not XML". Really. I want this. The X in XMPP stands for extensible not XML. The reason XML fits the bill is because it has concept of namespaces, what JSON has? If you don’t have explicit namespace support how do you make your JSON variant of XMPP forward‐compatible? You don’t without making it look much, much, worse than any of the examples I’m about to show below. > (…) I really do prefer JSON over XML because of how compact and readable it is. > XML is big, bloated, and bureaucratic. My standard response to this is always “show me a JSON fragment that is shorter or prettier looking than an equivalent XML fragment.” XML can certainly be shorter than JSON and often is, and repeated tags are the best showcase for it: <user id="abc"> <phoneNo type="home">123456789</phoneNo> <phoneNo type="work">321654987</phoneNo> </user> This turns into this beautiful JSON: [ "users": [ { "id": "abc", "phoneNos": [ { "type": "home", "value": "123456789" }, { "type": "work", "value": "321654987" } ] } ] ] Oh, but what if order is important? Take this example: <policy id="123"> <drop someFilter="ghi" /> <accept someFilter="abc" anotherFilter="xyz"> <comment> This comment is quite long and spans several lines of text in pretty printed XML. But this is quite alright, because XML by default strips extra whitespace. </comment>  </accept> <drop someFilter="foo" anotherFilter="bar"> <reason statusCode="690">No reason</reason> </drop> </policy> Now you have to do it like that: [ "policies": [ { "id": "123", "rules": [ { "type": "drop", "someFilter": "ghi" }, { "type": "accept", "someFilter": "abc", "anotherFilter": "xyz", "comment": "Unfortunately the best you can do with JSON is to rely on your editor’s dynamic wrapping. And this is how you break the line: \nWhat a treat!", }, { "type": "drop", "someFilter": "foo", "anotherFilter": "bar", "reason": { "statusCode": 690, "message": "No reason"} } ] } ] ] For an example of how horrible JSON in wild can really be see Reddit’s JSON API. Because JSON has no syntax for metadata you get things like that: [ { "kind": "thing-foo", "items": [ # # list of comments or something that goes on for # two screens # ], "votes": 15 } ] See the keys are sorted for convenience. XML’s text representation is very minimal and consistent. You don’t need to quote Every Single Thing, and there are no special cases for not quoting just some type of values. The only redundant part of XML is repeating the name of the closing tag if the </element> has value. But this actually aids readability, as demonstrated above. Editing complex JSON by hand with a regular text editor is a nightmare, because of all those single '}', ']' and commas trailing for several lines when there’s any light nesting involved. Editing XML by hand on the other hand is no big deal.
jakerocheleau 13y ago
I don't have an answer but I do have a follow-up question. There are no pre-built GChat clients and I have been planning to create one using Phono, which is built on jQuery. Are there better solutions other than this, or the linked example Superfeedr? Google Chat is my main IM client and I know it's built on XMPP. I just want to pick a solid JS library to build with and make a web interface that can work for people.
- shmerl 13y ago
  You can also use something like Strophe.js to build your own client. Or you want to add an XMPP/Jingle support as well? How exactly would it work, through routing Jingle over WebRTC? Update: I just looked at Phono - it doesn't seem like it supprots XMPP/Jingle, even though it uses WebRTC. So this kind of defeats the purpose, since you can't initiate audio/video calls to other XMPP clients. Can you advise any combinations of JavaScript client libraries and XMPP servers (i.e. without a need to use third party gateways) which can already normally support XMPP over WebSockets and Jingle over WebRTC?
  - mweibel 13y ago
    XMPP over Websockets is implemented in various servers: - Prosody (http://code.google.com/p/prosody-modules/wiki/mod_websocket) - openfire (http://code.google.com/p/openfire-websockets/) - Tigase (built-in as far as I could see) - Ejabberd in the Commercial version, the opensource version doesn't really have one yet. Regarding Jingle over WebRTC: Various people are working on it (or have been, in my case) and it's getting closer to workable. What I encountered, when I tried ~a year ago to do it, was that not all SDP messages could be easily transformed into the Jingle format. Might have been changed now though as the WebRTC Specs have changed quite a bit.
    - shmerl 13y ago
      Thanks for the pointers. I hope open source ejabberd will catch up, since I prefer it in general. Are any of the Jingle through WebRTC projects public to track their progress?
shmerl 13y ago
I agree. It's better to help improving current XMPP libraries for good support of WebSockets and etc. instead of bringing JSON into the picture.

shmerl 13y ago

What is so scary in the XML?? I didn't get the point of this.

hayksaakian 13y ago
Personally, it's just another ML I need to think about. If it's JSON, I can at least pretend that my data structure stays coherent from A to B (ex: node.js webapp to js frontend). It's also easy to look at, appearing much like a dictionary or hash + array set in many programming languages.
- shmerl 13y ago
  XML is a "must know" kind of thing for any serious developer anyway, so I don't really see your point. XML and JSON have enough different use cases, even though they have common ones as well. For starters, XML can be automatically validated when needed using schemas, while JSON can't (even though there is a draft for JSON schema in the works). So XML has advantages over JSON is various scenarios. The price is more complexity naturally, but one should expect that. But besides the point, XMPP already uses XML. So there is no need to reinvent the wheel here.
  - Arelius 13y ago
    While, mostly working in client development, I'm a professional developer of 12 years, and while I've had my random encounter with XML, I've managed to avoid it, and do not "know" XML in any significant capacity. It's easy to forget that there is a large set of different developers. And some don't know what you may consider common knowledge. IMO, something being JSON over XML would be a significant plus in my mind.
  - hownowstephen 13y ago
    Working for a company that has started adopting JSON Schema as our primary validation tool, I can definitely agree that XML still has the upper hand when it comes to a seamless experience in data validation. Schema validation is still really primitive (though with the love JSON has been getting in the past couple years, it shouldn't be too much longer until it's in fighting shape). Something about XML just has never really sat well with me - whereas JSON just works when it comes to actually modelling objects in a way that makes sense and can be picked up quickly. I can't say I've yet to find a place where I absolutely needed a feature of XML that I couldn't find in JSON, but would love to know more situations where I can be proven wrong. I think ultimately if the goal is just to JSONify the world, then that's no good - but if we're looking to make protocols more accessible and hackable, JSON may be the way to go.

jerf 13y ago

Is it XEP-0295 compliant? http://xmpp.org/extensions/xep-0295.html http://xmpp.org/extensions/xep-0295.html

X-Istence 13y ago
All this does is wrap all of the XML in a {"s": "<stuff here>"}. That is useless.
mweibel 13y ago
XEP-0295 is humorous (1st april joke).
- jerf 13y ago
  Really? Is that what the green text up top meant?

lloydwatkin 13y ago

Hi all, As the author of xmpp-ftw I feel I should throw some comments into this discussion. Firstly, xmpp-ftw is not about starting an XML/JSON holy war, its about lowering the barrier to entry as far as possible. So see it as more of a gateway drug :) * At present the tiny amount of effort it takes to build a proprietary (legacy) chat system with other technologies means that XMPP is somewhat being left behind (on the web) despite offering a huge number of benefits. Xmpp-ftw tries to make it as easy as possible by providing developers a format they are comfortable with and the ability to send/receive data with the minimum of fuss. This project actually came out of discussions at XSF summits I've been attending. No-one (myself included) believes that XML should be replaced with JSON in XMPP, that would be a foolish notion. Generic JSON <-> XML is awful and doesn't quite work. The idea with this project was to used named events to fill in some of the missing information for building stanzas, and telling the user what is happening. The view of JSON with XMPP has changed over the last couple of summits has changed, going from being referred to as the J word' through to open discussions about what can be done (the XSF seems like a very progressive group). No more jokes about XEP-0295 :) If you want to see a better client library trying to do something similar check out Lance Stout's stanza.io https://github.com/legastero/stanza.io https://github.com/legastero/stanza.io which came out of the same meeting. So in summary: Xmpp-ftw is attempting to drop the barrier to entry for XMPP as far as possible and make it quick and convenient for developers to implement. Its not about replacing the angle brackets :) Cheers, Lloyd FYI, yes I've used socket.io, but you can stick anything that implements an event emitter and has callbacks and have it work. Currently I'm looking at wrapping sockJS. * Drugs are bad m'kay!

ashward 13y ago

There are a couple of key points here which a lot of you are missing which are: * The JSON -> XML conversion is done on the server rather than in the client. Browsers are surprisingly bad at xml (one of the reasons buddycloud chose to move the xmpp layer out of the browser), and xml dom manipulation can make for overly complex code (we should also bear in mind one of the philosophies of xmpp which was simple clients, complex servers), and this leads me to my second point: * It's really simple. This opens it up as a no-brainer solution for web devs who would otherwise just throw up a proprietary silo system based purely on socket.io or similar. Sure, if you really want all the complexities and nuances of xmpp then use an xml based library, but if you just want to send simple messages between users then this is really simple, and hey - what would have been a silo system is suddenly federatable at zero cost. The point here is that this library is not aimed at hardcore xmpp devs, but devs who would be put off by anything other than json. Make this library as easy to use as socket.io on its own and suddenly you've got more systems running on xmpp which wouldn't otherwise have been (which is great for everyone)!