9 ms·

With XMPP and federated messaging servers they would have at least working infrastructure within their country. But ... What exactly happened that XMPP lost o

by vog 10y ago

With XMPP and federated messaging servers they would have at least working infrastructure within their country. But ... What exactly happened that XMPP lost on mobile phones? What did they do wrong? XMPP was there before smartphones came, had working clients, and had (for that time) pretty decent encryption. While I understand that big players like WhatsApp want to bind all users to their own infrastructure, I don't understand why even the niche instant messangers go through the burden of creating their own infrastructure (or relying on Google's) instead of just concentrating on the client side to provide better XMPP clients. Is XMPP so bad that nobody wants to do that? If so, why?

koolba 10y ago
> Is XMPP so bad that nobody wants to do that? If so, why? No it'd be great but unfortunately the things that make something technically great don't align with the business of acquiring monetizeable users (either directly or indirectly by selling the resulting social graph). I'm convinced that the only reason we still have email is because it predates walled garden land grabs.
- sliken 10y ago
  Not to mention email isn't particularly distributed these days. I don't know the exact percentage, by yahoo, gmail, and microsoft definite host a large majority of the email accounts on the planet.
  - koolba 10y ago
    Anyone can register a domain, stand up a server, and start receiving email on it. Sending it a bit trickier as to get past spam filters there's a bit more to it (DKIM, SPF, static IP, etc) but it's still doable.
    - sliken 10y ago
      I do it, but do see regular complains that others try and fail. The main problem is getting black holed by the popular email providers.
- narrowrail 10y ago
  This discussion has been had on HN so many times, it would be difficult for me to summarize (but easy for one to search via algolia). Signal's creator runs their own email server, but always points out that 99% of the time, they are communicating with gmail on the other side. This subverts the decentralized nature of email. XMPP is not a mobile friendly protocol. Even the Matrix.org folks are doing XMPP compatibility, but not implementing XMPP directly. My point is, "don't align with the business of acquiring monetizeable users" is not entirely true, and there are technical reasons (not just business reasons). Everyone with an interest in this problem should be on the same page, and holding overly cynical views about the impediments isn't (IMO) conducive to creating solutions.
  - koolba 10y ago
    It doesn't have to be the mobile platform but it could be one interface of many. Having separate XMPP servers acting as the middle men forwarding messages to each other would have been awesome. The last leg delivery from your server to your device could be something else, possibly more optimized for the technical limitations of mobile transit. We even had a piece of this for a while as Google and I think a couple other messaging providers had XMPP support but it's gradually been killed off. The "business interests" part of this is that anybody creating a monetizeable user base via a messaging network wouldn't give the opening to others to come in via XMPP. It gives your users a way out and gives your competitors a way in.
  - deleted 10y ago
    [deleted]
tdkl 10y ago
It uses a lot of data, which even in 2016 isn't cheap for majority of mobile users.
- davecridland 10y ago
  I don't buy this. XMPP uses no more than any other IM system, and much less than most. The military use it over HF links in theatre, for heaven's sake - this is not what you'd choose to be doing if the bandwidth was so high. (HF is STANAG 5066, and runs down to a handful of bits a second - this stuff is not fast).
rahrahrah 10y ago
I'm not going to say this is the whole story, but... I recently tried XMPP on Pidgin for the first time on my laptop (Ubuntu), and I hate it. There's something very wrong and very ugly about the UI. It's feels just wrong. And not just that, I had to jump thru numerous hoops before I could get the thing working. I also recently tried to install Signal on my Xiaomi Redmi2, and it doesn't work. I don't know why but the registration always fails when "connecting to the servers", even though everything else on my phone works. (before I start getting responses about this, if your suggestion can be found on Google I already tried it) I think some things, like messaging where it's easy to create an MVP but hard to get the number of users that actually brings in the value, the entire game is polishing your one client to reduce frictions and increase experience. WhatsApp works as flawlessly on mi Redmi2 as it does on any Samsung or any iPhone. When you have zero friction to join over time your user base grows. When you have enormous friction to join, where users have to be wrestling with your broken app, it just fizzles. This isn't the case for every industry, but seems to be the case for messaging. Another example is Telegram. At one point they were ahead of WhatsApp in terms of features, now they're behind. Regardless, they keep growing their user base. My opinion is this is because their UX is at least as good as WhatsApp and has just as little friction.
- majewsky 10y ago
  > before I start getting responses about this, if your suggestion can be found on Google I already tried it Not on Google maybe, but still the page that you want: https://github.com/WhisperSystems/Signal-Android/issues/new https://github.com/WhisperSystems/Signal-Android/issues/new
- SamWhited 10y ago
  I'm sorry you had that experience; for the record, Pidgin is not representative of the XMPP protocol (Pidgin is outdated, broken, and the XMPP side of it is mostly unmaintained). If you're one of those people who likes lots of switches and knobs and everything under the sun supported I'd recommend Gajim (personally, I don't think it's stable enough for day-to-day use, but others I know just like having all the bells and whistles it provides), if you want a bit more stability, I'd try Swift, if you want a CLI I'd try Mcabber (Possibly with the Vi-mode keyboard binding patches that are floating about on GitHub somewhere)
aaron-lebo 10y ago
Federated services don't allow companies to lock you into wallen gardens. The big tech companies don't know how to make money without walled gardens (Thiel's book more or less makes this gospel without actually saying it)...hence, XMPP lost. You can go look at XMPP or any other protocol and yeah it has issues, but no more than any other protocol, and none that couldn't be fixed. But it's not obvious to find a working business model worth billions out of that, so companies that have to make billions don't want that. And of course government only cares about billion dollar companies (they don't have more reliable signaling), so...the fate of the protocols. There's usually a lot of bike-shedding after the fact, but they ignore the real reasons behind failure of adoption. There's already been multiple waves of centralization, decentralization, and so on: In the late 90s you had ICQ, MSN, AIM, etc, by the mid 2000s those had been reversed engineered to be open, new protocols were made (XMPP) and even adopted by the new giants, then Facebook, Google, etc closed back down. As Slack is currently doing with IRC. Fun stuff.
- sliken 10y ago
  Heh, "You can go look at XMPP or any other protocol and yeah it has issues, but no more than any other protocol, and none that couldn't be fixed." Please see https://xkcd.com/927/ https://xkcd.com/927/
- vog 10y ago
  > Federated services don't allow companies to lock you into wallen gardens. This does not explain why small (niche) players go that route, too. It seems you overlooked the second part of the question: While I understand that big players like WhatsApp want to bind all users to their own infrastructure, I don't understand why even the niche instant messangers go through the burden of creating their own infrastructure (or relying on Google's) instead of just concentrating on the client side to provide better XMPP clients.
  - aaron-lebo 10y ago
    You can look at this thread and see why. The argument usually is along the lines of: "it can't work because it didn't work". Niche players take their cues from the crowd, too. If a bunch of people in a forum say that tech is bad and that's the dominant mindset, it's going to limit the people who give it a shot.
sliken 10y ago
Why do you think egypt only protects their network at the border? Just because they block signal servers at the border, doesn't mean they don't implement blocks at other places on their network. XMPP seems a morass of optional pieces, so clients and servers implement random subsets and it makes life difficult for random users to use securely. Moxie makes a good point about and federated standard without control of client and server is going to evolve very slowly an fracture into incompatible pieces.
- pseudalopex 10y ago
  There's more than one way to build a federated network. XMPP follows the IETF model, with standards and code only loosely coordinated, and evolves through optional extensions. Matrix is trying something different: a versioned, more monolithic specification, with reference implementations and SDKs. Signal's approach has its share of problems, too. It took OWS two years from the start of iOS development to deliver voice and text support. (They may not have been working on it the whole time, but that just underscores the problem.) The iOS app doesn't support rotation. Desktop support was a long time coming, and the multi-device experience still kind of sucks. It's great that WhatsApp could push end-to-end encryption to a billion users at once. But none of my friends who care about end-to-end encryption trust Facebook, and Signal isn't winning over the others.
- apichat 10y ago
  > federated standard without control of client and server is going to evolve very slowly an fracture into incompatible pieces. Good answers : An Objection to ‘The ecosystem is moving’ https://gultsch.de/objection.html https://gultsch.de/objection.html Juste think about how the web and Internet evolved. Juste think about how the web and Internet evolved
  - sliken 10y ago
    "Just think about how the web and Internet evolved" What percent of DNS lookups use DNSSEC? What percent of email is e2e encrypted? What percent of NTP lookups are signed, encrypted, or trust worthy? What percent of HTTPS traffic would not die by a trivial downgrade attack? What XMPP client allows you to click on it in the apple/google play store, auto discover any contacts using it, and start chatting (with e2e) in 30 seconds? Are things getting better? Certainly, it's rather slow progress though. I don't see how signal could have gotten as far as has, as quickly as it has, without controlling both server and client, and skipping federated. Sure, by leading by example hopefully the bar is raised. All the signal compromises (controlling client and server, non federating, using GCM, requiring a sim, etc) seem to be well justified to get pretty good security to a large number of diverse people quickly. Sure federated would be great, as would be independence from the whisper systems servers which can be easily blocked (like in egypt). So sure hopefully someone comes up with something that manages the strengths of signals with federation and censor resistance. Not sure that will look much like XMPP though.
cesarb 10y ago
From what I've read, yes, XMPP is bad. But that's not the problem. Keeping a persistent connection to your server, while not keeping the phone processor and/or radio active, and while not being dropped by the phone company's NAT after a very short timeout, is not a simple problem. The native persistent connection (Google's, for Android, and Apple's for an iPhone) is the only one which is guaranteed to work good enough. Federation has the problem of being harder to prevent abuse. And for systems that use the phone number as the user's identifier, like WhatsApp or Signal, it's even harder: which server is associated to an user? What to do if the user registers to more than one server? How to prevent a server from saying it has registered a user, when the user didn't do it? And when you control the whole infrastructure, it's easier to provide a good user experience. With federation, you are depending on the quality of other's implementations.
- aaron-lebo 10y ago
  A lot of these arguments read intuitively, but: Federation has the problem of being harder to prevent abuse. But this is also an issue with centralized systems (see Facebook, Twitter right now). And for systems that use the phone number as the user's identifier, like WhatsApp or Signal, it's even harder: which server is associated to an user? What to do if the user registers to more than one server? How to prevent a server from saying it has registered a user, when the user didn't do it? This is "hard", but solved. Bitcoin and similar systems do this (prove identity) very well already. And when you control the whole infrastructure, it's easier to provide a good user experience. With federation, you are depending on the quality of other's implementations. Google doesn't control email, gmail is still a pretty good experience. Controlling the infrastructure should lead to better quality and experience, but practically often doesn't. Does anyone love iTunes?
- apichat 10y ago
  > Keeping a persistent connection to your server [...] is not a simple problem But is already solved. Explanations : "The State of Mobile XMPP in 2016" https://gultsch.de/xmpp_2016.html https://gultsch.de/xmpp_2016.html > Federation has the problem of being harder to prevent abuse. Precisely XMPP have already solutions for this problem > And when you control the whole infrastructure, it's easier to provide a good user experience. With federation, you are depending on the quality of other's implementations. Answers here : "Objection to ‘The ecosystem is moving’" https://gultsch.de/objection.html https://gultsch.de/objection.html You might also interest by GnuNameSystem wich is a federate, decentralize and censorship resistance DNS https://gnunet.org/gns https://gnunet.org/gns
lmm 10y ago
Yes, it really is that bad. It's based on XML, which in theory would be possible to do right, but all the actually existing XML libraries insist on validating everything at the drop of a hat (probably fetching schemas over the internet) and a horifically overengineered model of namespaces where you can't do anything until you've precisely defined all your namespaces. The vanilla XMPP standard doesn't permit a server to deliver the same message to more than one client for the same user. This is utterly wrongheaded and falls apart as soon as you start logging in from multiple devices (e.g. phone + computer, or even just two different computers). This was obviously terrible and has been obviously terrible for decades, but it seems impossible to change it. There is an extension to work around the problem, but extensions take forever to get approved and twice forever to get actually adopted in real clients. The groupchat UX was always awful, partly because groupchats were never properly federated (they have to live on a single server) and there was no way to do groupchat history for people who didn't keep all their devices turned on all the time (other than, you guessed it, an extension that got stuck in limbo for a decade or so) but mostly just because all the clients sucked. It really isn't hard to do groupchats right: all you have to do is alert when the user is mentioned but not alert on ordinary messages to the group. But somehow everyone managed to screw it up. XMPP was objectively terrible (or at least, all the XMPP clients were objectively terrible). Worse than AIM. But there were people who were (understandably) blind to its faults because it was open-source, and this anti-synergized with the traditional-unix luddite tradition that rejects anything other than plain text in messages, and the good old standards-first approach that brought us XHTML2.0 and CSS2.1 and a decade with no HTML standard (i.e. trying to standardize everything before any implementation, which never works - what works is allowing clients to add extensions and then standardizing what's been shown to work and become popular, a la WhatWG). The real issue is that if you try to build an XMPP client that defines something "non-standard" you'll get torn to pieces by the OSS fans. Many of the niche instant messengers are probably running XMPP under the hood, but they don't want to expose it or support federation lest they get accused of trying to "embrace, extend, extinguish".
- deleted 10y ago
  [deleted]
- aaron-lebo 10y ago
Zash 10y ago
Did you know that WhatsApp uses XMPP?
- vog 10y ago
  This is wrong. They started with XMPP, but switched to a proprietary protocol a long time ago: http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html http://highscalability.com/blog/2014/2/26/the-whatsapp-archi... | The next few years were spent re-writing and modifying quite a few parts of ejabberd, including switching from XMPP to internally developed protocol, restructuring the code base and redesigning some core components, and making lots of important modifications to Erlang VM to optimize server performance.
  - Zash 10y ago
    I wasn't aware that they had switched entirely, only that they invented their own custom compression and (really bad) authentication and encryption methods.
    - lorenzhs 10y ago
      WhatsApp uses Signal protocol for encryption nowadays: https://whispersystems.org/blog/whatsapp-complete/ https://whispersystems.org/blog/whatsapp-complete/
    - sliken 10y ago
      Well if it's not XMPP compatible it's not really XMPP. Apparently they switched to signal based protocol for transport.
upofadown 10y ago
XMPP used to have power consumption issues on mobile because of the constant TCP connection from the client to the server. There were also problems with network disconnections/changes on mobile for the same reason. XMPP uses TLS certificates for client to client and server to server connections and such things used to be a pain before Let's Encrypt for those hoping to run their own servers. The end to end encryption used to be OTR which tended to be a bit clunky to use in practice. That was then. Those issues have been more or less fixed these days. XMPP is actually starting to look interesting again. For an example of a modern XMPP client (and discussion about the issues mentioned above) see here: * https://conversations.im https://conversations.im
- frandroid 10y ago
  > A free and open source Jabber/XMPP client for Android. "Free" for CDN$2.89! Not that I care about the $3, but which is it, guys?
  - vog 10y ago
    They only want money for providing the build. The source code and build scripts are free/libre. From the website: How do I install Conversations? Conversations is entirely open source and licensed under GPLv3. So if you are a software developer, you can check out the sources from GitHub and use gradle to build your APK file. The more convenient way — which not only gives you automatic updates but also supports the further development of Conversations — is to buy the App in the Google Play Store. Buying the App from the Play Store will also give you access to our beta test.
    - ryukafalz 10y ago
      You can also install it from F-Droid: https://f-droid.org/repository/browse/?fdfilter=conversations&fdid=eu.siacs.conversations https://f-droid.org/repository/browse/?fdfilter=conversation...
  - williamscales 10y ago
    Free as in speech, not beer.
rlpb 10y ago
Comparing XMPP to WhatsApp or Signal is not a like-for-like comparison. XMPP is a just protocol. A solution also requires a hosted platform and an app. Signal and WhatsApp each provide all three: a protocol, a hosted platform and an app. XMPP is just a protocol. There's no business model in a protocol.
rascul 10y ago
I have yet to find a XMPP client on Android which supports voice and video, is free, and usable. The power and notification issues seem to be solved (at least somewhat) with some clients.
zeveb 10y ago
I think part of it is that XMPP dates back to the late 90s, when we thought XML was a good thing (not realising that it is, in fact, a kind of information technology antimatter, which annihilates anything it touches, leaving nothing but bugs and hurt feelings behind). The protocol in action feels crufty, and the massive number of optional extensions means that you're never quite certain if your client, your server, your correspondent's server and your correspondent's client will all play ball. Then you toss in the fact that all the big players have decided that walled gardens are just awesome, and the fact that no-one today would touch anything using XML with a 1,024-foot pole — and thus there's just no ecosystem for it anymore. (as an aside, I believe that in a century our children will consider the digression to XML & JavaScript to be as inexplicable as we do China's decision to scuttle its fleet of exploration)
moxie 10y ago
> With XMPP and federated messaging servers they would have at least working infrastructure within their country. Why do you think this is true? I can't see how federation is an answer to censorship. They would simply censor access to all the XMPP providers, just as they're censoring access to all the non-federated messengers. Is your idea that people would spin up new providers so quickly that the censors wouldn't be able to keep up? Every time people switched, they'd have to rediscover their entire social network all over again, and there's an asymmetry between how difficult it is to get everyone over to different hosts while rediscovering where everyone is vs how easy it is for the censors to add a single line to their block list. It'd be way easier just to have a centralized host and have people switch VPN providers as they get blocked, since then they don't have to rediscover each-other, but that isn't a great user experience either. Just like with metadata hiding, really effective censorship circumvention is going to require new protocols and new techniques, so we're going to be more likely to see those emerge in centralized rather than federated systems (that are by their nature difficult to update). I think we'll be able to respond in Signal very quickly.
- Zash 10y ago
  True, neither architecture will ultimately solve the problem of an attacker having absolute control over the lower layer. No way around it. Federation does has the nice property that it gives you some choice over who controls the infrastructure you use, but in this case it would mean moving to a different country.
- SamWhited 10y ago
  They could also sensor all email providers, or try to block all SMS providers, but historically we've seen that this is much more difficult than you make it out to be. It's not a silver bullet of course, but it's evident that federated architectures are harder to kill (see eg. Egypt attempting to censor emails and SMS's a while back, or Syria before that [and probably after too])
  - moxie 10y ago
    Can you provide specific historical examples where censorship of federated protocols was difficult or impossible for technical reasons? I can't see how there is anything technically difficult about it. To the extent that services aren't censored, it's usually either because they're not encrypted (so no reason to block them) or because they're too popular to get away with blocking.
darklajid 10y ago
XMPP _was_ bad on mobiles. The biggest issues were a) it was overly chatty (i.e. constantly sending/receiving information, draining your battery) and arguably worse b) it wasn't designed for changing addresses (wifi->data and back) or constant loss of connectivity. So if you used XMPP some years ago you'd kill your mobile, fast, and would constantly lose messages. Other features of modern IM solutions (say, an archive of your messages?) weren't present or usable either. Today? Most of these issues are resolved. Prosody is a great server to get started (again) and comes with solutions for most (all? MAM aka 'Message Archive' is still somewhat scary imo) problems I mentioned above. Conversation is a very nice and modern XMPP client. TL/DR: XMPP was 'bad' for mobile deployment for a long time and probably lost too much traction because of that. XMPP isn't bad in that respect anymore.
- apichat 10y ago
  Yes good explanations here : The State of Mobile XMPP in 2016 https://gultsch.de/xmpp_2016.html https://gultsch.de/xmpp_2016.html

by vog 10y ago

koolba 10y ago

> Is XMPP so bad that nobody wants to do that? If so, why? No it'd be great but unfortunately the things that make something technically great don't align with the business of acquiring monetizeable users (either directly or indirectly by selling the resulting social graph). I'm convinced that the only reason we still have email is because it predates walled garden land grabs.

sliken 10y ago
Not to mention email isn't particularly distributed these days. I don't know the exact percentage, by yahoo, gmail, and microsoft definite host a large majority of the email accounts on the planet.
- koolba 10y ago
  Anyone can register a domain, stand up a server, and start receiving email on it. Sending it a bit trickier as to get past spam filters there's a bit more to it (DKIM, SPF, static IP, etc) but it's still doable.
  - sliken 10y ago
    I do it, but do see regular complains that others try and fail. The main problem is getting black holed by the popular email providers.
narrowrail 10y ago
This discussion has been had on HN so many times, it would be difficult for me to summarize (but easy for one to search via algolia). Signal's creator runs their own email server, but always points out that 99% of the time, they are communicating with gmail on the other side. This subverts the decentralized nature of email. XMPP is not a mobile friendly protocol. Even the Matrix.org folks are doing XMPP compatibility, but not implementing XMPP directly. My point is, "don't align with the business of acquiring monetizeable users" is not entirely true, and there are technical reasons (not just business reasons). Everyone with an interest in this problem should be on the same page, and holding overly cynical views about the impediments isn't (IMO) conducive to creating solutions.
- koolba 10y ago
  It doesn't have to be the mobile platform but it could be one interface of many. Having separate XMPP servers acting as the middle men forwarding messages to each other would have been awesome. The last leg delivery from your server to your device could be something else, possibly more optimized for the technical limitations of mobile transit. We even had a piece of this for a while as Google and I think a couple other messaging providers had XMPP support but it's gradually been killed off. The "business interests" part of this is that anybody creating a monetizeable user base via a messaging network wouldn't give the opening to others to come in via XMPP. It gives your users a way out and gives your competitors a way in.
- deleted 10y ago
  [deleted]

tdkl 10y ago

It uses a lot of data, which even in 2016 isn't cheap for majority of mobile users.

davecridland 10y ago
I don't buy this. XMPP uses no more than any other IM system, and much less than most. The military use it over HF links in theatre, for heaven's sake - this is not what you'd choose to be doing if the bandwidth was so high. (HF is STANAG 5066, and runs down to a handful of bits a second - this stuff is not fast).

rahrahrah 10y ago

I'm not going to say this is the whole story, but... I recently tried XMPP on Pidgin for the first time on my laptop (Ubuntu), and I hate it. There's something very wrong and very ugly about the UI. It's feels just wrong. And not just that, I had to jump thru numerous hoops before I could get the thing working. I also recently tried to install Signal on my Xiaomi Redmi2, and it doesn't work. I don't know why but the registration always fails when "connecting to the servers", even though everything else on my phone works. (before I start getting responses about this, if your suggestion can be found on Google I already tried it) I think some things, like messaging where it's easy to create an MVP but hard to get the number of users that actually brings in the value, the entire game is polishing your one client to reduce frictions and increase experience. WhatsApp works as flawlessly on mi Redmi2 as it does on any Samsung or any iPhone. When you have zero friction to join over time your user base grows. When you have enormous friction to join, where users have to be wrestling with your broken app, it just fizzles. This isn't the case for every industry, but seems to be the case for messaging. Another example is Telegram. At one point they were ahead of WhatsApp in terms of features, now they're behind. Regardless, they keep growing their user base. My opinion is this is because their UX is at least as good as WhatsApp and has just as little friction.

majewsky 10y ago
> before I start getting responses about this, if your suggestion can be found on Google I already tried it Not on Google maybe, but still the page that you want: https://github.com/WhisperSystems/Signal-Android/issues/new https://github.com/WhisperSystems/Signal-Android/issues/new
SamWhited 10y ago
I'm sorry you had that experience; for the record, Pidgin is not representative of the XMPP protocol (Pidgin is outdated, broken, and the XMPP side of it is mostly unmaintained). If you're one of those people who likes lots of switches and knobs and everything under the sun supported I'd recommend Gajim (personally, I don't think it's stable enough for day-to-day use, but others I know just like having all the bells and whistles it provides), if you want a bit more stability, I'd try Swift, if you want a CLI I'd try Mcabber (Possibly with the Vi-mode keyboard binding patches that are floating about on GitHub somewhere)

aaron-lebo 10y ago

Federated services don't allow companies to lock you into wallen gardens. The big tech companies don't know how to make money without walled gardens (Thiel's book more or less makes this gospel without actually saying it)...hence, XMPP lost. You can go look at XMPP or any other protocol and yeah it has issues, but no more than any other protocol, and none that couldn't be fixed. But it's not obvious to find a working business model worth billions out of that, so companies that have to make billions don't want that. And of course government only cares about billion dollar companies (they don't have more reliable signaling), so...the fate of the protocols. There's usually a lot of bike-shedding after the fact, but they ignore the real reasons behind failure of adoption. There's already been multiple waves of centralization, decentralization, and so on: In the late 90s you had ICQ, MSN, AIM, etc, by the mid 2000s those had been reversed engineered to be open, new protocols were made (XMPP) and even adopted by the new giants, then Facebook, Google, etc closed back down. As Slack is currently doing with IRC. Fun stuff.

sliken 10y ago
Heh, "You can go look at XMPP or any other protocol and yeah it has issues, but no more than any other protocol, and none that couldn't be fixed." Please see https://xkcd.com/927/ https://xkcd.com/927/
vog 10y ago
> Federated services don't allow companies to lock you into wallen gardens. This does not explain why small (niche) players go that route, too. It seems you overlooked the second part of the question: While I understand that big players like WhatsApp want to bind all users to their own infrastructure, I don't understand why even the niche instant messangers go through the burden of creating their own infrastructure (or relying on Google's) instead of just concentrating on the client side to provide better XMPP clients.
- aaron-lebo 10y ago
  You can look at this thread and see why. The argument usually is along the lines of: "it can't work because it didn't work". Niche players take their cues from the crowd, too. If a bunch of people in a forum say that tech is bad and that's the dominant mindset, it's going to limit the people who give it a shot.

sliken 10y ago

Why do you think egypt only protects their network at the border? Just because they block signal servers at the border, doesn't mean they don't implement blocks at other places on their network. XMPP seems a morass of optional pieces, so clients and servers implement random subsets and it makes life difficult for random users to use securely. Moxie makes a good point about and federated standard without control of client and server is going to evolve very slowly an fracture into incompatible pieces.

pseudalopex 10y ago
There's more than one way to build a federated network. XMPP follows the IETF model, with standards and code only loosely coordinated, and evolves through optional extensions. Matrix is trying something different: a versioned, more monolithic specification, with reference implementations and SDKs. Signal's approach has its share of problems, too. It took OWS two years from the start of iOS development to deliver voice and text support. (They may not have been working on it the whole time, but that just underscores the problem.) The iOS app doesn't support rotation. Desktop support was a long time coming, and the multi-device experience still kind of sucks. It's great that WhatsApp could push end-to-end encryption to a billion users at once. But none of my friends who care about end-to-end encryption trust Facebook, and Signal isn't winning over the others.
apichat 10y ago
> federated standard without control of client and server is going to evolve very slowly an fracture into incompatible pieces. Good answers : An Objection to ‘The ecosystem is moving’ https://gultsch.de/objection.html https://gultsch.de/objection.html Juste think about how the web and Internet evolved. Juste think about how the web and Internet evolved
- sliken 10y ago
  "Just think about how the web and Internet evolved" What percent of DNS lookups use DNSSEC? What percent of email is e2e encrypted? What percent of NTP lookups are signed, encrypted, or trust worthy? What percent of HTTPS traffic would not die by a trivial downgrade attack? What XMPP client allows you to click on it in the apple/google play store, auto discover any contacts using it, and start chatting (with e2e) in 30 seconds? Are things getting better? Certainly, it's rather slow progress though. I don't see how signal could have gotten as far as has, as quickly as it has, without controlling both server and client, and skipping federated. Sure, by leading by example hopefully the bar is raised. All the signal compromises (controlling client and server, non federating, using GCM, requiring a sim, etc) seem to be well justified to get pretty good security to a large number of diverse people quickly. Sure federated would be great, as would be independence from the whisper systems servers which can be easily blocked (like in egypt). So sure hopefully someone comes up with something that manages the strengths of signals with federation and censor resistance. Not sure that will look much like XMPP though.

cesarb 10y ago

From what I've read, yes, XMPP is bad. But that's not the problem. Keeping a persistent connection to your server, while not keeping the phone processor and/or radio active, and while not being dropped by the phone company's NAT after a very short timeout, is not a simple problem. The native persistent connection (Google's, for Android, and Apple's for an iPhone) is the only one which is guaranteed to work good enough. Federation has the problem of being harder to prevent abuse. And for systems that use the phone number as the user's identifier, like WhatsApp or Signal, it's even harder: which server is associated to an user? What to do if the user registers to more than one server? How to prevent a server from saying it has registered a user, when the user didn't do it? And when you control the whole infrastructure, it's easier to provide a good user experience. With federation, you are depending on the quality of other's implementations.

aaron-lebo 10y ago
A lot of these arguments read intuitively, but: Federation has the problem of being harder to prevent abuse. But this is also an issue with centralized systems (see Facebook, Twitter right now). And for systems that use the phone number as the user's identifier, like WhatsApp or Signal, it's even harder: which server is associated to an user? What to do if the user registers to more than one server? How to prevent a server from saying it has registered a user, when the user didn't do it? This is "hard", but solved. Bitcoin and similar systems do this (prove identity) very well already. And when you control the whole infrastructure, it's easier to provide a good user experience. With federation, you are depending on the quality of other's implementations. Google doesn't control email, gmail is still a pretty good experience. Controlling the infrastructure should lead to better quality and experience, but practically often doesn't. Does anyone love iTunes?
apichat 10y ago
> Keeping a persistent connection to your server [...] is not a simple problem But is already solved. Explanations : "The State of Mobile XMPP in 2016" https://gultsch.de/xmpp_2016.html https://gultsch.de/xmpp_2016.html > Federation has the problem of being harder to prevent abuse. Precisely XMPP have already solutions for this problem > And when you control the whole infrastructure, it's easier to provide a good user experience. With federation, you are depending on the quality of other's implementations. Answers here : "Objection to ‘The ecosystem is moving’" https://gultsch.de/objection.html https://gultsch.de/objection.html You might also interest by GnuNameSystem wich is a federate, decentralize and censorship resistance DNS https://gnunet.org/gns https://gnunet.org/gns

lmm 10y ago

Yes, it really is that bad. It's based on XML, which in theory would be possible to do right, but all the actually existing XML libraries insist on validating everything at the drop of a hat (probably fetching schemas over the internet) and a horifically overengineered model of namespaces where you can't do anything until you've precisely defined all your namespaces. The vanilla XMPP standard doesn't permit a server to deliver the same message to more than one client for the same user. This is utterly wrongheaded and falls apart as soon as you start logging in from multiple devices (e.g. phone + computer, or even just two different computers). This was obviously terrible and has been obviously terrible for decades, but it seems impossible to change it. There is an extension to work around the problem, but extensions take forever to get approved and twice forever to get actually adopted in real clients. The groupchat UX was always awful, partly because groupchats were never properly federated (they have to live on a single server) and there was no way to do groupchat history for people who didn't keep all their devices turned on all the time (other than, you guessed it, an extension that got stuck in limbo for a decade or so) but mostly just because all the clients sucked. It really isn't hard to do groupchats right: all you have to do is alert when the user is mentioned but not alert on ordinary messages to the group. But somehow everyone managed to screw it up. XMPP was objectively terrible (or at least, all the XMPP clients were objectively terrible). Worse than AIM. But there were people who were (understandably) blind to its faults because it was open-source, and this anti-synergized with the traditional-unix luddite tradition that rejects anything other than plain text in messages, and the good old standards-first approach that brought us XHTML2.0 and CSS2.1 and a decade with no HTML standard (i.e. trying to standardize everything before any implementation, which never works - what works is allowing clients to add extensions and then standardizing what's been shown to work and become popular, a la WhatWG). The real issue is that if you try to build an XMPP client that defines something "non-standard" you'll get torn to pieces by the OSS fans. Many of the niche instant messengers are probably running XMPP under the hood, but they don't want to expose it or support federation lest they get accused of trying to "embrace, extend, extinguish".

deleted 10y ago
[deleted]
aaron-lebo 10y ago

Zash 10y ago

Did you know that WhatsApp uses XMPP?

vog 10y ago
This is wrong. They started with XMPP, but switched to a proprietary protocol a long time ago: http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html http://highscalability.com/blog/2014/2/26/the-whatsapp-archi... | The next few years were spent re-writing and modifying quite a few parts of ejabberd, including switching from XMPP to internally developed protocol, restructuring the code base and redesigning some core components, and making lots of important modifications to Erlang VM to optimize server performance.
- Zash 10y ago
  I wasn't aware that they had switched entirely, only that they invented their own custom compression and (really bad) authentication and encryption methods.
  - lorenzhs 10y ago
    WhatsApp uses Signal protocol for encryption nowadays: https://whispersystems.org/blog/whatsapp-complete/ https://whispersystems.org/blog/whatsapp-complete/
  - sliken 10y ago
    Well if it's not XMPP compatible it's not really XMPP. Apparently they switched to signal based protocol for transport.

upofadown 10y ago

XMPP used to have power consumption issues on mobile because of the constant TCP connection from the client to the server. There were also problems with network disconnections/changes on mobile for the same reason. XMPP uses TLS certificates for client to client and server to server connections and such things used to be a pain before Let's Encrypt for those hoping to run their own servers. The end to end encryption used to be OTR which tended to be a bit clunky to use in practice. That was then. Those issues have been more or less fixed these days. XMPP is actually starting to look interesting again. For an example of a modern XMPP client (and discussion about the issues mentioned above) see here: * https://conversations.im https://conversations.im

frandroid 10y ago
> A free and open source Jabber/XMPP client for Android. "Free" for CDN$2.89! Not that I care about the $3, but which is it, guys?
- vog 10y ago
  They only want money for providing the build. The source code and build scripts are free/libre. From the website: How do I install Conversations? Conversations is entirely open source and licensed under GPLv3. So if you are a software developer, you can check out the sources from GitHub and use gradle to build your APK file. The more convenient way — which not only gives you automatic updates but also supports the further development of Conversations — is to buy the App in the Google Play Store. Buying the App from the Play Store will also give you access to our beta test.
  - ryukafalz 10y ago
    You can also install it from F-Droid: https://f-droid.org/repository/browse/?fdfilter=conversations&fdid=eu.siacs.conversations https://f-droid.org/repository/browse/?fdfilter=conversation...
- williamscales 10y ago
  Free as in speech, not beer.

rlpb 10y ago

Comparing XMPP to WhatsApp or Signal is not a like-for-like comparison. XMPP is a just protocol. A solution also requires a hosted platform and an app. Signal and WhatsApp each provide all three: a protocol, a hosted platform and an app. XMPP is just a protocol. There's no business model in a protocol.

rascul 10y ago

I have yet to find a XMPP client on Android which supports voice and video, is free, and usable. The power and notification issues seem to be solved (at least somewhat) with some clients.

zeveb 10y ago

I think part of it is that XMPP dates back to the late 90s, when we thought XML was a good thing (not realising that it is, in fact, a kind of information technology antimatter, which annihilates anything it touches, leaving nothing but bugs and hurt feelings behind). The protocol in action feels crufty, and the massive number of optional extensions means that you're never quite certain if your client, your server, your correspondent's server and your correspondent's client will all play ball. Then you toss in the fact that all the big players have decided that walled gardens are just awesome, and the fact that no-one today would touch anything using XML with a 1,024-foot pole — and thus there's just no ecosystem for it anymore. (as an aside, I believe that in a century our children will consider the digression to XML & JavaScript to be as inexplicable as we do China's decision to scuttle its fleet of exploration)

moxie 10y ago

> With XMPP and federated messaging servers they would have at least working infrastructure within their country. Why do you think this is true? I can't see how federation is an answer to censorship. They would simply censor access to all the XMPP providers, just as they're censoring access to all the non-federated messengers. Is your idea that people would spin up new providers so quickly that the censors wouldn't be able to keep up? Every time people switched, they'd have to rediscover their entire social network all over again, and there's an asymmetry between how difficult it is to get everyone over to different hosts while rediscovering where everyone is vs how easy it is for the censors to add a single line to their block list. It'd be way easier just to have a centralized host and have people switch VPN providers as they get blocked, since then they don't have to rediscover each-other, but that isn't a great user experience either. Just like with metadata hiding, really effective censorship circumvention is going to require new protocols and new techniques, so we're going to be more likely to see those emerge in centralized rather than federated systems (that are by their nature difficult to update). I think we'll be able to respond in Signal very quickly.

Zash 10y ago
True, neither architecture will ultimately solve the problem of an attacker having absolute control over the lower layer. No way around it. Federation does has the nice property that it gives you some choice over who controls the infrastructure you use, but in this case it would mean moving to a different country.
SamWhited 10y ago
They could also sensor all email providers, or try to block all SMS providers, but historically we've seen that this is much more difficult than you make it out to be. It's not a silver bullet of course, but it's evident that federated architectures are harder to kill (see eg. Egypt attempting to censor emails and SMS's a while back, or Syria before that [and probably after too])
- moxie 10y ago
  Can you provide specific historical examples where censorship of federated protocols was difficult or impossible for technical reasons? I can't see how there is anything technically difficult about it. To the extent that services aren't censored, it's usually either because they're not encrypted (so no reason to block them) or because they're too popular to get away with blocking.

darklajid 10y ago

XMPP _was_ bad on mobiles. The biggest issues were a) it was overly chatty (i.e. constantly sending/receiving information, draining your battery) and arguably worse b) it wasn't designed for changing addresses (wifi->data and back) or constant loss of connectivity. So if you used XMPP some years ago you'd kill your mobile, fast, and would constantly lose messages. Other features of modern IM solutions (say, an archive of your messages?) weren't present or usable either. Today? Most of these issues are resolved. Prosody is a great server to get started (again) and comes with solutions for most (all? MAM aka 'Message Archive' is still somewhat scary imo) problems I mentioned above. Conversation is a very nice and modern XMPP client. TL/DR: XMPP was 'bad' for mobile deployment for a long time and probably lost too much traction because of that. XMPP isn't bad in that respect anymore.

apichat 10y ago
Yes good explanations here : The State of Mobile XMPP in 2016 https://gultsch.de/xmpp_2016.html https://gultsch.de/xmpp_2016.html