5 ms·

There's no harm in having loads of options - you just get a Darwinian survival of the fittest for which app works best. Competition is healthy. The thing that

by ara4n 11y ago

There's no harm in having loads of options - you just get a Darwinian survival of the fittest for which app works best. Competition is healthy. The thing that sucks is that as end users we end up having all our chats and identity fragmented over all these different silos - be they selfhosted ones or proprietary SaaS. There's no way I'll rely on MatterMost or any of the above unless I can access my existing communities (be they on IRC, XMPP, Slack, HipChat or whatever); adding yet more fragmentation into the mix helps nobody. This is why it's vital to have an open standard for decentralising the conversations between all these different islands that kills fragmentation whenever a new one pops up. And it's actually beneficial to new contenders like MatterMost as it could help them onboard users into their UX and app without having to start new conversations and contacts. [Disclaimer: Matrix.org is such a standard, providing an open HTTP API for decentralised chatrooms, and I work on it.]

bad_user 11y ago
You mean XMPP federation? Nobody wants to implement that because companies like Slack, Atlassian, Google, Facebook, Microsoft, Apple, etc are interested in lock-in primarily.
- lez 11y ago
  "Nobody" is an exageration, since Gitter, RocketChat, Mattermost, Friends, and Let's chat DO want to implement that.
- wslh 11y ago
  I hope the open source community learned a lesson with Slack and other offerings. The trick is the user experience. Slack is a trivial piece of code to reproduce (not their business).
- heavenlyhash 11y ago
  Lots of people -- in that list, even -- started with XMPP based systems. They all move away. At first, I also thought this was an open and shut case of interest in lock-in. Now, I'm not going to say that isn't true, but it's also worth considering that XMPP... just isn't that good. Having recently started working on a chat client that was intended to speak first-and-foremost XMPP, I have to admit: XMPP is... it's decades out of date and missing a number of absolutely critical, central ideas that are essential to making stuff work. I can't honestly fault anyone who started with XMPP and then rapidly started looking to get off. Examples: - unique message IDs? Absent. XEPs kind of provide; but I can't tell you which of the three or for relevant ones are the most relevant (AMP IDs from XEP 0079? Stream Management from XEP 0198? Acking from XEP 0184? Something from Carbons or MAM in 0313 or 0280? You know, if you wanted some light reading...). - multi device? Oh. My. God. It's bananas. The spec behavior is that whenever a client sends a message, the server is supposed to consider that one the most alive, and then route all future messages exclusively to that one. So you send a message on your phone? Yeah, your desktop is just going to silently stop receiving messages. - there's a concept of "message carbons" to deal with this. This involves re-sending all your messages back to the server after you receive them, with special instructions to send them back again to your other clients. The amount of redundantly redundant XML involved is eyewatering. Combine that multidevice behavior (messages can get randomly routed anywhere at any time) with the wild-west nature of message delivery acks, and you can see how ridiculously difficult this makes the basic idea of "all clients should see the same picture". Overall, the XEP process, conceptually, is a great example of open extensibility. The trouble is, so much of this stuff is core to sane message delivery semantics that it really, practically speaking, causes huge problems when it's all considered "extensions". Stuff like message IDs fundamentally shouldn't be an extension because it's just too critical that all minimum-viable clients agree. You just can't build higher level stuff without that. XEPs are great. A community process for extensions should exist. It just needs to exist for extensions, not subsume the total set of realistic minimum viable features. I want an open chat protocol. I thought XMPP might be the one. Then I actually looked at it. Tried to write something with it. XMPP is not prepared to be the one, in any sense at all except for legacy support. I have all the respect in the world for all the folks who put effort into trying to make an open chat standard, but... honestly, we need a stronger foundation than this. (N.b. These complaints are a shameless repost from another not-very-old thread about chat where XMPP was also raised: https://news.ycombinator.com/item?id=9718784 https://news.ycombinator.com/item?id=9718784 .) I've since looked at matrix.org ... and I'm really impressed. The standard is completely reasonable. It's the first federated chat standard I've seen proposed that actually has loose timing systems that's both available during a partition and can reach consistency afterward (messages list their last-seen messages, and so act like vector clocks, an accepted good solution to problems in CAP territory). You could actually take two systems with clock skew and have them reach a single, reconciled view of the world, even after netsplits. And practically speaking... man, go look at their list of clients. Despite being a relatively young project, matrix has clients on every major platform -- as far as I can tell, this is better platform coverage than XMPP, and better feature parity within those clients since critical things like message IDs are actually baked into the spec. I really suggest giving matrix a look. There's a lot going right over there.
  - Flowdalic 11y ago
    You are absolutely right. I usually encourage people to get involved in improving the (XMPP) situation. Just last week I submitted a XEP for unique stanza IDs to the XSF [1], which will most likely be the base for IDs in MAM and other protocols. I guess that's the light reading you want. :) And yes, Carbons (XEP-280), in it's current state, also suck. I'm working on a replacement XEP called "Message Routing 2.0 " (MR2): http://geekplace.eu/xeps/xep-mr2/xep-mr2.html http://geekplace.eu/xeps/xep-mr2/xep-mr2.html It's in an early draft state, but I think it addresses most, if not all, issues with carbons, improving the multi-device situation in XMPP somewhat (I don't care if it's MR2 or an improved version of carbons that makes the race). > see how ridiculously difficult this makes the basic idea of "all clients should see the same picture" Being involved in XMPP a bit, I get quite a few requests on how to implement WhatsApp/Hangouts like groupchats. And while the basic idea is quite simple, implementing it is a non-trivial task. But it can certainly be done. But we need more motivated developers and specification designers to get it done. While the XMPP community is just great, the count of activley involved people is quite small. 1: http://mail.jabber.org/pipermail/standards/2015-June/029865.html http://mail.jabber.org/pipermail/standards/2015-June/029865....
  - davecridland 11y ago
    Matrix do have a much bigger marketing budget than the XSF, it's true. They fly their employees all over the globe, and appear to have a constant stream of FUD on tap about XMPP, presumably because it's their strongest competitor. Last I saw, though, it's got a thumping great single-point-of-control in the middle, firmly under their corporate grasp. Nothing too critical, though, only your identity. (Looking at the FAQ, the "Can I run my own Identity Server?" question is simply missing, but I can answer that one: No) XMPP's primary distinction is that all services are fully partitioned by domain. The only shared state - and I use this term very loosely - is in subscription data, which is long term. (By subscription data, I'm intending to mean the roster, pubsub, and also MUC chatroom occupancy). Matrix attempts to share considerably more state, acting much more like a single chat service scattered across multiple servers, but due to the centralized and privileged identity servers, there's a significant loss of autonomy in the system. Important to note, an XMPPservice on a single domain can be spread across multiple physical servers, fully sharing state, in which case you're well into the territory of vector clocking and CAP theorums, but that's an internal matter for your service and doesn't affect the inter-domain federation at all. In layman's terms, you can run an XMPP service without reference to any external system - although federation would need you to reference DNS. Running a Matrix server leaves you somewhat under the control of Matrix, which despite appearances is just a branding for a commercial outfit. Well, that, and XMPP is an open standard according to a definition we didn't just make up as part of our marketing. (see OpenStand.org) So first off, it may be useful to note that when looking for a universal, interoperable standard for text chat, the armed forces around the world went for XMPP. This based on its reliability and handling of low bandwidth situations (much, much, MUCH lower bandwidth than mobile). So when you say XMPP doesn't support your notion of an MVP, it's worth noting it really is just that - your notion. That doesn't make you wrong, but it may be that your focus is different from other people's. And as a second point, when considering writing something on the web, you (thankfully) don't have to implement JavaScript. Or HTML. Or CSS. Or, thank heavens, HTTP. Do you even know what happens in HTTP? Clearly not, otherwise you'd be a gibbering wreck - it's about as badly designed as a protocol can be and still work. And don't get me started on TCP; nobody gets that right. So really, if you're implementing something using XMPP, please don't leap in at the low level - just use one of the many libraries for your chosen platforms. Next: Unique message identifiers are an interesting topic, and you've referenced several XEPs that don't address the problem. One issue is that what's needed varies in different situations - generally, a short-term scoped identifier is good enough, but it's fair to say that's not always the case. The tuple of <from,to,id> is good enough for this, though clients aren't mandated to send an identifier. It's not even clear to me that we need global stable persistent single-valued identifiers, but despite there being no use-case explained to me where this is required, I'm aware it's a popular solution some undefined problem. Worth noting, too, that XEP-0198 and XEP-0184, though often confused as addressing the same problem, actually address entirely different ones. '198 addresses link reliability, tackling the two generals problem. '184 addresses end to end acks. You end up with acks in both, but 184's can be out of order, and 198 allows session resumption and retransmissions, which addresses the same cases you identify with vector clocks between domains. Your library should handle all this for you, and just note if a message is lost. Carbons you have entirely wrong, sorry. It does not involve "re-sending your messages back to the server", it just allows non-priority clients to get copies of all messages. The only oddity is that the copies are just that - they're not duplicates. This allows the client being copied to identify that the messages are going to a different device. Which device is usually based on "priority", though different servers can do things slightly differently. Despite this, it doesn't have anything to do with message traffic. And it's not random. However, clients typically respond to the device which sent the message - but if the sender has moved devices, they'll get a carbon. And yes, there is a massive "if supported" right there, but there are desktop clients that do. In fact, we don't handle the "every device has the same view of the universe" concept - you're right - we handle the "every device knows the state of every other device". It's functionally identical, but a slightly richer view. So, let's rewind. The XSF doesn't write software, though there are many libraries out there. We're working on Push, mostly to cover Apple's platform where long-running sessions are prohibited - this despite the fact that on Android, clients like Yaxim or Conversations don't excessively use power. The XSF also doesn't do the levels of social media advertising that Matrix does. This is becoming increasingly problematic for us as a community, because Matrix are explicitly targetting XMPP with things that are simply untrue. Most of the problems you identify are actually solved at the protocol layer. They really are. And it's not as bad a solution as you imply, either - in fact, the solutions to multi-device messaging and archiving are pretty good, and more based on "every device knows about the other devices" rather than "every device is treated identically", which is interesting in and of itself. Oh, and when I said low bandwidth? XMPP is routinely used over 2400bps SATCOM links with 30s latency, but it'll run lower. Try 9 bits per second, half-duplex, with 2 minutes link turnaround.

by ara4n 11y ago

bad_user 11y ago

You mean XMPP federation? Nobody wants to implement that because companies like Slack, Atlassian, Google, Facebook, Microsoft, Apple, etc are interested in lock-in primarily.

lez 11y ago
"Nobody" is an exageration, since Gitter, RocketChat, Mattermost, Friends, and Let's chat DO want to implement that.
wslh 11y ago
I hope the open source community learned a lesson with Slack and other offerings. The trick is the user experience. Slack is a trivial piece of code to reproduce (not their business).
heavenlyhash 11y ago
Lots of people -- in that list, even -- started with XMPP based systems. They all move away. At first, I also thought this was an open and shut case of interest in lock-in. Now, I'm not going to say that isn't true, but it's also worth considering that XMPP... just isn't that good. Having recently started working on a chat client that was intended to speak first-and-foremost XMPP, I have to admit: XMPP is... it's decades out of date and missing a number of absolutely critical, central ideas that are essential to making stuff work. I can't honestly fault anyone who started with XMPP and then rapidly started looking to get off. Examples: - unique message IDs? Absent. XEPs kind of provide; but I can't tell you which of the three or for relevant ones are the most relevant (AMP IDs from XEP 0079? Stream Management from XEP 0198? Acking from XEP 0184? Something from Carbons or MAM in 0313 or 0280? You know, if you wanted some light reading...). - multi device? Oh. My. God. It's bananas. The spec behavior is that whenever a client sends a message, the server is supposed to consider that one the most alive, and then route all future messages exclusively to that one. So you send a message on your phone? Yeah, your desktop is just going to silently stop receiving messages. - there's a concept of "message carbons" to deal with this. This involves re-sending all your messages back to the server after you receive them, with special instructions to send them back again to your other clients. The amount of redundantly redundant XML involved is eyewatering. Combine that multidevice behavior (messages can get randomly routed anywhere at any time) with the wild-west nature of message delivery acks, and you can see how ridiculously difficult this makes the basic idea of "all clients should see the same picture". Overall, the XEP process, conceptually, is a great example of open extensibility. The trouble is, so much of this stuff is core to sane message delivery semantics that it really, practically speaking, causes huge problems when it's all considered "extensions". Stuff like message IDs fundamentally shouldn't be an extension because it's just too critical that all minimum-viable clients agree. You just can't build higher level stuff without that. XEPs are great. A community process for extensions should exist. It just needs to exist for extensions, not subsume the total set of realistic minimum viable features. I want an open chat protocol. I thought XMPP might be the one. Then I actually looked at it. Tried to write something with it. XMPP is not prepared to be the one, in any sense at all except for legacy support. I have all the respect in the world for all the folks who put effort into trying to make an open chat standard, but... honestly, we need a stronger foundation than this. (N.b. These complaints are a shameless repost from another not-very-old thread about chat where XMPP was also raised: https://news.ycombinator.com/item?id=9718784 https://news.ycombinator.com/item?id=9718784 .) I've since looked at matrix.org ... and I'm really impressed. The standard is completely reasonable. It's the first federated chat standard I've seen proposed that actually has loose timing systems that's both available during a partition and can reach consistency afterward (messages list their last-seen messages, and so act like vector clocks, an accepted good solution to problems in CAP territory). You could actually take two systems with clock skew and have them reach a single, reconciled view of the world, even after netsplits. And practically speaking... man, go look at their list of clients. Despite being a relatively young project, matrix has clients on every major platform -- as far as I can tell, this is better platform coverage than XMPP, and better feature parity within those clients since critical things like message IDs are actually baked into the spec. I really suggest giving matrix a look. There's a lot going right over there.
- Flowdalic 11y ago
  You are absolutely right. I usually encourage people to get involved in improving the (XMPP) situation. Just last week I submitted a XEP for unique stanza IDs to the XSF [1], which will most likely be the base for IDs in MAM and other protocols. I guess that's the light reading you want. :) And yes, Carbons (XEP-280), in it's current state, also suck. I'm working on a replacement XEP called "Message Routing 2.0 " (MR2): http://geekplace.eu/xeps/xep-mr2/xep-mr2.html http://geekplace.eu/xeps/xep-mr2/xep-mr2.html It's in an early draft state, but I think it addresses most, if not all, issues with carbons, improving the multi-device situation in XMPP somewhat (I don't care if it's MR2 or an improved version of carbons that makes the race). > see how ridiculously difficult this makes the basic idea of "all clients should see the same picture" Being involved in XMPP a bit, I get quite a few requests on how to implement WhatsApp/Hangouts like groupchats. And while the basic idea is quite simple, implementing it is a non-trivial task. But it can certainly be done. But we need more motivated developers and specification designers to get it done. While the XMPP community is just great, the count of activley involved people is quite small. 1: http://mail.jabber.org/pipermail/standards/2015-June/029865.html http://mail.jabber.org/pipermail/standards/2015-June/029865....
- davecridland 11y ago
  Matrix do have a much bigger marketing budget than the XSF, it's true. They fly their employees all over the globe, and appear to have a constant stream of FUD on tap about XMPP, presumably because it's their strongest competitor. Last I saw, though, it's got a thumping great single-point-of-control in the middle, firmly under their corporate grasp. Nothing too critical, though, only your identity. (Looking at the FAQ, the "Can I run my own Identity Server?" question is simply missing, but I can answer that one: No) XMPP's primary distinction is that all services are fully partitioned by domain. The only shared state - and I use this term very loosely - is in subscription data, which is long term. (By subscription data, I'm intending to mean the roster, pubsub, and also MUC chatroom occupancy). Matrix attempts to share considerably more state, acting much more like a single chat service scattered across multiple servers, but due to the centralized and privileged identity servers, there's a significant loss of autonomy in the system. Important to note, an XMPPservice on a single domain can be spread across multiple physical servers, fully sharing state, in which case you're well into the territory of vector clocking and CAP theorums, but that's an internal matter for your service and doesn't affect the inter-domain federation at all. In layman's terms, you can run an XMPP service without reference to any external system - although federation would need you to reference DNS. Running a Matrix server leaves you somewhat under the control of Matrix, which despite appearances is just a branding for a commercial outfit. Well, that, and XMPP is an open standard according to a definition we didn't just make up as part of our marketing. (see OpenStand.org) So first off, it may be useful to note that when looking for a universal, interoperable standard for text chat, the armed forces around the world went for XMPP. This based on its reliability and handling of low bandwidth situations (much, much, MUCH lower bandwidth than mobile). So when you say XMPP doesn't support your notion of an MVP, it's worth noting it really is just that - your notion. That doesn't make you wrong, but it may be that your focus is different from other people's. And as a second point, when considering writing something on the web, you (thankfully) don't have to implement JavaScript. Or HTML. Or CSS. Or, thank heavens, HTTP. Do you even know what happens in HTTP? Clearly not, otherwise you'd be a gibbering wreck - it's about as badly designed as a protocol can be and still work. And don't get me started on TCP; nobody gets that right. So really, if you're implementing something using XMPP, please don't leap in at the low level - just use one of the many libraries for your chosen platforms. Next: Unique message identifiers are an interesting topic, and you've referenced several XEPs that don't address the problem. One issue is that what's needed varies in different situations - generally, a short-term scoped identifier is good enough, but it's fair to say that's not always the case. The tuple of <from,to,id> is good enough for this, though clients aren't mandated to send an identifier. It's not even clear to me that we need global stable persistent single-valued identifiers, but despite there being no use-case explained to me where this is required, I'm aware it's a popular solution some undefined problem. Worth noting, too, that XEP-0198 and XEP-0184, though often confused as addressing the same problem, actually address entirely different ones. '198 addresses link reliability, tackling the two generals problem. '184 addresses end to end acks. You end up with acks in both, but 184's can be out of order, and 198 allows session resumption and retransmissions, which addresses the same cases you identify with vector clocks between domains. Your library should handle all this for you, and just note if a message is lost. Carbons you have entirely wrong, sorry. It does not involve "re-sending your messages back to the server", it just allows non-priority clients to get copies of all messages. The only oddity is that the copies are just that - they're not duplicates. This allows the client being copied to identify that the messages are going to a different device. Which device is usually based on "priority", though different servers can do things slightly differently. Despite this, it doesn't have anything to do with message traffic. And it's not random. However, clients typically respond to the device which sent the message - but if the sender has moved devices, they'll get a carbon. And yes, there is a massive "if supported" right there, but there are desktop clients that do. In fact, we don't handle the "every device has the same view of the universe" concept - you're right - we handle the "every device knows the state of every other device". It's functionally identical, but a slightly richer view. So, let's rewind. The XSF doesn't write software, though there are many libraries out there. We're working on Push, mostly to cover Apple's platform where long-running sessions are prohibited - this despite the fact that on Android, clients like Yaxim or Conversations don't excessively use power. The XSF also doesn't do the levels of social media advertising that Matrix does. This is becoming increasingly problematic for us as a community, because Matrix are explicitly targetting XMPP with things that are simply untrue. Most of the problems you identify are actually solved at the protocol layer. They really are. And it's not as bad a solution as you imply, either - in fact, the solutions to multi-device messaging and archiving are pretty good, and more based on "every device knows about the other devices" rather than "every device is treated identically", which is interesting in and of itself. Oh, and when I said low bandwidth? XMPP is routinely used over 2400bps SATCOM links with 30s latency, but it'll run lower. Try 9 bits per second, half-duplex, with 2 minutes link turnaround.