4 ms·

Jeremie Miller is the guy behind XMPP, so this is coming from somebody who really knows what he's doing when it comes to protocols. This type of thing would be

by aston 16y ago

Jeremie Miller is the guy behind XMPP, so this is coming from somebody who really knows what he's doing when it comes to protocols. This type of thing would be cool as a federation mechanism for something like Google Wave or Diaspora. I think it could also be used as an alternative to XMPP itself...

bkudria 16y ago
I agree with you, but I can see how some might strenuously object - XMPP is not universally loved. (Luckily Telehash uses JSON)
- jerf 16y ago
  That's how you learn, though. XMPP is much better than most people realize, anyhow. Most people do not realize how much of the messiness comes from the problem and not the protocol. IM's easy, right? Just message here, message there? Yeah, sure, the first iteration of IM in the mid-1990s was. File transfers are easy, right? Yes, when both clients and the server are on the same LAN they're trivial. On the real Internet? No, actually they're quite hard. Conferences are no sweat, right? If you're hacking together a Node.js demo that simply shovels out messages to everyone in the conference with no further features, sure. If you want like features and stuff in some sort of standard way, it gets harder. And so on. I'm not saying it's perfect in every way, in particular the oldest parts of the protocol are a bit crufty (like the overloading of the presence tag to mean too many things), but there's a lot of people who rag on it for really misguided reasons, or who failed to read the core standard and got tripped up on the connection process. (In fact, there's a lot of people who don't even spend enough time on it to realize that the standard is modularized and you don't have to take every piece for every purpose.) If I were designing it from scratch today, yes it would be a JSON datagram protocol. But XMPP is many years older than JSON as a distinct thing.
  - moe 16y ago
    XMPP is much better than most people realize, anyhow. No, it is actually worse than most people realize. You only begin to realize the magnitude of the failure once you try to implement a client, and most people don't do that. XMPP is a trainwreck. It has never seen serious adoption because nobody wants to touch it - and for good reasons. XMPP is what I cite when I try to explain the "XML mindset". It leads to bad things. It leads to ridiculous overengineering through layered complexity. It leads to a client/server ecosystem where each implementation speaks a different dialect because it's nearly impossible to get the protocol right. There was a time when my roster would get screwed up in new, random, interesting ways whenever I launched a different client. Some clients would even manage to unsubscribe existing contacts for inexplicable reasons. And don't get me started on "Transports". However, instant messaging is not rocket science. Neither is semi-decentralized instant messaging. XMPP makes it seem like a much harder problem than it really is, but only because XMPP is broken beyond repair. Most people do not realize how much of the messiness comes from the problem and not the protocol. Wrong. Take a lesson from IRC, a group-chat protocol that, despite its age, works and scales amazingly well. A protocol that, despite an immense range of features, can easily be typed by a human on a telnet prompt, in real time. It wouldn't take much fix the warts on IRC and extend it to cover everything that XMPP tries to do. This is what the XMPP author should have done in first place.
    - jerf 16y ago
      I have implemented two clients, a generalized libpurple transport, and spent significant time on a server in a professional environment for sale to real customers. IRC isn't a replacement for what it does, and yeah, the problems come from the problem domain, not the protocol. I'd know the difference. None of the "professional" protocols seem to be significantly simpler, because they can't be... the domain complexity forbids it. I spent much more time working on semantic issues than I ever did working on the raw XML, orders of magnitude difference. The XML was always the easy part, even if JSON would have been even easier. The trappings are irrelevant. I could transform the entire XMPP protocol stack into JSON in a week, with implementation in ejabberd and one of my clients. XML is a sideshow. The problem is getting the standardized semantics. If XMPP taught me anything, it is that it doesn't matter how many specs you throw at a programmer, they're just going to bash on the program until it sort of works most of the time and release it. That's where your roster problems come from, it's where a not insignificant number of your transport problems come from too. (Though the transport protocol is one of the spottier bits of the protocol.) The core bits of XMPP are generally reasonably well specified and in my experience actually held up surprisingly well as I bent and spindled it a little bit. (Corporate customers don't "get" rosters, don't get that you can start with a blank roster and work your way up, so I added a module to build rosters based on grouping criteria specified by the user and driven by outside input. Well beyond the stock ejabberd shared rosters, but pretty custom to our environment. XMPP actually dealt with these semi-magical roster entries just fine, to my surprise.) Those specifications really matter and just sort of bashing some stuff out that's 90% correct most of the time isn't good enough when you're trying to communicate with so many different systems. XMPP actually managed to avoid a lot of problems that even the "professional" systems had, having learned from their experiences; AIM last I knew still had some encoding corner cases you wouldn't expect in a modern program, all of the protocols had major encoding growing pains, surprising versioning issues, all these little quirks inside them that you never noticed because you can paper over a lot when you control both the clients and the servers. Again, I know it's not perfect but in the space of "deployed IM protocols" it does not make a bad showing. IRC clients are actually just as quirky, IRC just doesn't hang on to anywhere near as much state or you'd see it mangled, spindled, and mutilated too. (Also part of the reason it's not a replacement, real users want that state.) This is fine, too, I don't have a problem with IRC for what it is, but you can't just drop it in everywhere you see an XMPP server. If you try to work IRC up to be a real, true XMPP replacement, you'll be complaining about how hard it sucks in no time. Too much suckage is in the problem space.
- jemfinch 16y ago
  "Luckily"? If the protocol were to catch on, imagine how many terabits of bandwidth would be wasted by such an unnecessarily verbose and redundant protocol. Imagine how many CPU-years will be wasted parsing and unparsing the packets. Imagine how much memory wasted because packets can't be processed as they come off the wire, but must be buffered in their entirety before they can be parsed. This is exactly the sort of thing libraries like Google's Protocol Buffers and Facebook's Thrift were invented for, both of which are open source. Optimizing for human readability using netcat just doesn't make sense for potentially core protocols such as this one.
  - bkudria 16y ago
    Really? I think they can cross that bridge when they come to it. For now, adoption is critical.
  - jerf 16y ago
    Ever gzipped yourself some JSON being used in a standardized protocol? I've got a JSON-based protocol at work for my project and my typical compression ratio on any but the smallest messages with a simple gzip is 16:1. YMMV depending on the exact contents but I don't lose much sleep over the bandwidth. As for CPU years, meh. It's human years that matter. JSON parsing is that big a challenge anyhow. Protocol buffers et al are great, certainly, and they exist for a reason, but not every app needs them and human readability turns out to be very useful in practice.
  - zachrose 16y ago
    What's "unparsing?" (Not being snarky. Genuinely curious.)
    - loup-vaillant 16y ago
      That's serializing: when you go from a data structure to a text file. The exact reverse of "parsing", actually.
  - wooster 16y ago
    There are a few pretty decent streaming JSON parsers out there. I use yajl for exactly that.

by aston 16y ago

bkudria 16y ago

I agree with you, but I can see how some might strenuously object - XMPP is not universally loved. (Luckily Telehash uses JSON)

jerf 16y ago
That's how you learn, though. XMPP is much better than most people realize, anyhow. Most people do not realize how much of the messiness comes from the problem and not the protocol. IM's easy, right? Just message here, message there? Yeah, sure, the first iteration of IM in the mid-1990s was. File transfers are easy, right? Yes, when both clients and the server are on the same LAN they're trivial. On the real Internet? No, actually they're quite hard. Conferences are no sweat, right? If you're hacking together a Node.js demo that simply shovels out messages to everyone in the conference with no further features, sure. If you want like features and stuff in some sort of standard way, it gets harder. And so on. I'm not saying it's perfect in every way, in particular the oldest parts of the protocol are a bit crufty (like the overloading of the presence tag to mean too many things), but there's a lot of people who rag on it for really misguided reasons, or who failed to read the core standard and got tripped up on the connection process. (In fact, there's a lot of people who don't even spend enough time on it to realize that the standard is modularized and you don't have to take every piece for every purpose.) If I were designing it from scratch today, yes it would be a JSON datagram protocol. But XMPP is many years older than JSON as a distinct thing.
- moe 16y ago
  XMPP is much better than most people realize, anyhow. No, it is actually worse than most people realize. You only begin to realize the magnitude of the failure once you try to implement a client, and most people don't do that. XMPP is a trainwreck. It has never seen serious adoption because nobody wants to touch it - and for good reasons. XMPP is what I cite when I try to explain the "XML mindset". It leads to bad things. It leads to ridiculous overengineering through layered complexity. It leads to a client/server ecosystem where each implementation speaks a different dialect because it's nearly impossible to get the protocol right. There was a time when my roster would get screwed up in new, random, interesting ways whenever I launched a different client. Some clients would even manage to unsubscribe existing contacts for inexplicable reasons. And don't get me started on "Transports". However, instant messaging is not rocket science. Neither is semi-decentralized instant messaging. XMPP makes it seem like a much harder problem than it really is, but only because XMPP is broken beyond repair. Most people do not realize how much of the messiness comes from the problem and not the protocol. Wrong. Take a lesson from IRC, a group-chat protocol that, despite its age, works and scales amazingly well. A protocol that, despite an immense range of features, can easily be typed by a human on a telnet prompt, in real time. It wouldn't take much fix the warts on IRC and extend it to cover everything that XMPP tries to do. This is what the XMPP author should have done in first place.
  - jerf 16y ago
    I have implemented two clients, a generalized libpurple transport, and spent significant time on a server in a professional environment for sale to real customers. IRC isn't a replacement for what it does, and yeah, the problems come from the problem domain, not the protocol. I'd know the difference. None of the "professional" protocols seem to be significantly simpler, because they can't be... the domain complexity forbids it. I spent much more time working on semantic issues than I ever did working on the raw XML, orders of magnitude difference. The XML was always the easy part, even if JSON would have been even easier. The trappings are irrelevant. I could transform the entire XMPP protocol stack into JSON in a week, with implementation in ejabberd and one of my clients. XML is a sideshow. The problem is getting the standardized semantics. If XMPP taught me anything, it is that it doesn't matter how many specs you throw at a programmer, they're just going to bash on the program until it sort of works most of the time and release it. That's where your roster problems come from, it's where a not insignificant number of your transport problems come from too. (Though the transport protocol is one of the spottier bits of the protocol.) The core bits of XMPP are generally reasonably well specified and in my experience actually held up surprisingly well as I bent and spindled it a little bit. (Corporate customers don't "get" rosters, don't get that you can start with a blank roster and work your way up, so I added a module to build rosters based on grouping criteria specified by the user and driven by outside input. Well beyond the stock ejabberd shared rosters, but pretty custom to our environment. XMPP actually dealt with these semi-magical roster entries just fine, to my surprise.) Those specifications really matter and just sort of bashing some stuff out that's 90% correct most of the time isn't good enough when you're trying to communicate with so many different systems. XMPP actually managed to avoid a lot of problems that even the "professional" systems had, having learned from their experiences; AIM last I knew still had some encoding corner cases you wouldn't expect in a modern program, all of the protocols had major encoding growing pains, surprising versioning issues, all these little quirks inside them that you never noticed because you can paper over a lot when you control both the clients and the servers. Again, I know it's not perfect but in the space of "deployed IM protocols" it does not make a bad showing. IRC clients are actually just as quirky, IRC just doesn't hang on to anywhere near as much state or you'd see it mangled, spindled, and mutilated too. (Also part of the reason it's not a replacement, real users want that state.) This is fine, too, I don't have a problem with IRC for what it is, but you can't just drop it in everywhere you see an XMPP server. If you try to work IRC up to be a real, true XMPP replacement, you'll be complaining about how hard it sucks in no time. Too much suckage is in the problem space.
jemfinch 16y ago
"Luckily"? If the protocol were to catch on, imagine how many terabits of bandwidth would be wasted by such an unnecessarily verbose and redundant protocol. Imagine how many CPU-years will be wasted parsing and unparsing the packets. Imagine how much memory wasted because packets can't be processed as they come off the wire, but must be buffered in their entirety before they can be parsed. This is exactly the sort of thing libraries like Google's Protocol Buffers and Facebook's Thrift were invented for, both of which are open source. Optimizing for human readability using netcat just doesn't make sense for potentially core protocols such as this one.
- bkudria 16y ago
  Really? I think they can cross that bridge when they come to it. For now, adoption is critical.
- jerf 16y ago
  Ever gzipped yourself some JSON being used in a standardized protocol? I've got a JSON-based protocol at work for my project and my typical compression ratio on any but the smallest messages with a simple gzip is 16:1. YMMV depending on the exact contents but I don't lose much sleep over the bandwidth. As for CPU years, meh. It's human years that matter. JSON parsing is that big a challenge anyhow. Protocol buffers et al are great, certainly, and they exist for a reason, but not every app needs them and human readability turns out to be very useful in practice.
- zachrose 16y ago
  What's "unparsing?" (Not being snarky. Genuinely curious.)
  - loup-vaillant 16y ago
    That's serializing: when you go from a data structure to a text file. The exact reverse of "parsing", actually.
- wooster 16y ago
  There are a few pretty decent streaming JSON parsers out there. I use yajl for exactly that.