11 ms·

Reflections: The ecosystem is moving

226 points by tprynn 10y ago

mcepl 10y ago
Every security engineer I have met so far is to some extent (some of the to quite an extent) paranoid and control-freak. That's good for security engineers, you want them to be paranoid and control freaks. Of course, the bad part is that they are paranoid and control-freaks. So, although I don't question huge amount of work brought to the public for free by Moxie, I cannot notice that he killed all points of federation (even those which were to some extent under his control, http://www.cyanogenmod.org/blog/whisperpush-end-of-life http://www.cyanogenmod.org/blog/whisperpush-end-of-life), that he sabotages non-Google-controlled repositories (https://github.com/WhisperSystems/Signal-Android/issues/127 https://github.com/WhisperSystems/Signal-Android/issues/127) of his so called open software client. I don't care that much because for this and other reasons I won't touch Signal with a ten-feet pole, and federated alternatives (XMPP) work for me just fine, thank you. So, my suspicion is that instead of some theoretical difficulties the issue is that he got how own fiefdom under his control, and now he is not willing to let it go. All these opponents of federation and open software (I am not talking about just FLOSS here, I mean truly community-drive software), each with their fiefdoms to protect, argue that we are stuck with the open technologies from 1990s (or even earlier), and only their proprietary silo (proprietary meaning with only one implementation and under their control) is able to bring us a shiny new blink. However, they tend to ignore the other part of the story. The biggest problem of storing my life in proprietary silos is not that they are under control, I don’t live a life in delusion of importance, so I don’t expect anybody to mine my communication with my wife about what to do for dinner (and yet, even that communication is encrypted by a pretty good encryption system, which works just fine with a plain XMPP). The biggest problem is that every individual proprietary silos in the moment owner of such silo goes out of business, or for some other reasons looses interest in providing the service. So, the question is not how many technologies from 1990s we use, but how many proprietary technologies from 1990s we don't use. Yes, I have stored on my hard drive all emails since 1997, because old boring mbox files still work as they did. But all those exciting and sometimes tearful conversations I had on Yahoo! IM or ICQ are gone forever. That bigfoot.com and mail.com accounts I had for my life? Yeah. And to the second lie Moxies lies to himself (I am willing to believe he really means it). It is suddenly easy to move between proprietary silos? Yeah. It is lovely that we have to keep unique identifier (phone number) for all NSAs to trace us in some weird databases somewhere (that livejournal.com account? yes, it is now owned by the Russian Коммерса́нтъ; yes, I know that this newspaper is not that bad). And yes, it is seriously interesting if perhaps phone numbers would serve as the unique personal identifier most Americans were so afraid to introduce. A possibility to have anonymous accounts? Yeah. However, even this does not seem to make transfer between platforms that easy. If all your friends are on Telegram, then no matter how easy for whole group it would be easy to transfer, you cannot transfer unless whole group does. How many people really and truly love Twitter, and how many of those who hate it with passion are able to move to some better platform? Nobody, because that is where people are.
cm3 10y ago
Just because it's not super simple doesn't mean we should treat federation as a dead end. Sandstorm is one way for non-admins to have their own server and there's work regarding self-hosting everything. It's unsexy, it's not a money maker like a centralized messenger, but unless we want to live in a world where there's only one or two centralized messaging solutions, we as developers have to eat our own dog food and improve easy hosting and federation.
- moxie 10y ago
  How would apps running on Sandstorm avoid the problems with federation in today's world outlined in the blog post?
  - kentonv 10y ago
    It looks to me like your article is really arguing for central control of the code, rather than central control of the serving infrastructure. Having everyone running the same code means changes can be made quickly because only one implementation needs to change. Centralized serving infrastructure makes central control of the code easy. But if you look at mobile, we have central control of the code there without central control of the infrastructure: everyone has their own phone, yet runs the same apps that stay up-to-date via app stores. Sandstorm is basically taking that model to servers: within 24 hours of an app update, everyone gets a notification and can click one button to update their copy, so everyone tends to stay up-to-date. (Disclosure: I'm the lead developer of Sandstorm.)
    - pdkl95 10y ago
      > central control of the code That's just an excuse to not implement backwards compatibility in the protocol. Central control is also a security risk, as we've seen recently with Apple and the FBI. If the code is controlled centrally, the FBI (or whomever) only has to strong-arm one target. > gets a notification and can click one button to update their copy That sounds like you're trying to keep the user in control, which is great. I haven't head of Sandstorm before, but it sounds interesting. Have you considered the problem of forced updates? I'm not sure there is a good way to protect against the FBI trying something similar to the recent drama with Apple, where they order you to force a "special" update on {someone,everyone}.
marssaxman 10y ago
That's one of the most depressing things I've read in a while. If this is actually true, the internet has failed.
tprynn 10y ago
I'll add my thoughts here, since I think a number of people will be disappointed in this post. There are a number of competing goals when developing software that "the whole world can/will use". And it seems obvious to me that getting the whole world to use a secure, encrypted messenger like Signal (now, Whatsapp) is a noble goal. The obvious competing goal is federation, and the fact that we've failed at improving federated services (e-mail being the best/most important example) is a problem. Ultimately, Moxie's arguments here hold a lot of sway for me. I think it sucks that constantly-improving services are impossible to address in a federated way, and he's not really offering a solution to that. But instead he presents arguments I hadn't thought about much, which is that perhaps non-federated services can be dramatically better than federated services in the short-term. And in the long term, perhaps we'll move back to federated services; right now, the cost of changing services/ecosystems is very low. So we don't need to be as 'doom-and-gloom' about the fact that Signal does not achieve all the competing goals, because it achieves a number of extremely important goals right now, and we can work on the other goals in the future.
- marssaxman 10y ago
  It's precisely the fact that Moxie is so credible on this topic that makes his post so frustrating. I believe that he's telling the truth as he sees it, and I believe that he's as well-situated as anyone could be to pull off a white-hat secure messaging system, so if he says it can't be done with federation, it's awfully hard for me to assert that he's wrong. The problem is that I don't see the point of doing any of this without federation. If we're just using the Internet as a substrate for a bunch of proprietary silo services, then we're just back to the way things were before - walled gardens and media conglomerates and extortionate Bell System-style lock-in shenanigans. So what? Why bother? What have we gained, with all this effort, if we're back in the same old shit again? The whole point was that we no longer needed to get anyone's permission to communicate, to publish, to build new services for interacting with each other; we could just do it. If federation is gone, then those days are gone, and the most important reason to care about the Internet is gone, too.
  - moxie 10y ago
dsp1234 10y ago
No users, no walled garden. More users, more walled garden. Is this just the natural state of systems that require/acquire network effects?
- skybrian 10y ago
  I have a universal theory about this which is "spam and abuse make everything suck". This is why we can't have an open libertarian Internet, and instead have walled gardens and moderated forums - ordinary users don't want to deal with the crap. It's also why we don't have open API's, and why the API's that the big providers do give us tend to be nerfed and then discontinued - they're constantly abused, which imposes a maintenance cost that businesses have to justify. It's why an email server is so difficult to run and email delivery is so unreliable. It's why you can't (or shouldn't) rely too much on Paypal. It's why governments effectively outsource Internet policing to big providers. And so on.
- dlmetcalf 10y ago
  Not necessarily. Microsoft kept trying to make a walled garden around the web. They might have had a lot of users, but the web fought back and luckily today we have 'world wide web', not a totally isolated Microsoft Web, Google Web, Apple Web, Firefox Web etc. I hated the days when you constantly had to switch web browsers to get pages to render. I'm glad it's rare now, because we didn't give up on the hope of open systems. Mark might have lost the battle to have HTTP/2 encrypted by default, but at least when Let's Encrypt came out, it still worked in all browsers. Would we have been better to just give up and have lots of walled gardens instead? I don't think so. I'm glad we have WWW. Personally I don't think XMPP failed because being open made it slow to adapt. I think we're mostly stuck because ICQ, AOL & MSN just got there first. I don't envy being in Moxie & OWS's position of figuring out how to make this work. I'll add opinion and some of it harsh, though I'll reserve criticism until I've built something as awesomely globally deployed as Signal.
kkl 10y ago
Some relevant background information on this issue: https://github.com/LibreSignal/LibreSignal/issues/37 https://github.com/LibreSignal/LibreSignal/issues/37
- ngrilly 10y ago
  Interesting discussion.
  - cpach 10y ago
    Definitely. The background convinced me even more that the essay really makes sense.
  - mbakke 10y ago
    I don't understand how Guardian were unable to obtain a license to use Signal/Axolotl in Chatsecure. The libraries on Github are GPL3, is there some missing IP I'm not aware of? Also it saddens me that LibreSignal is effectively dead now, along with any future third-party clients. I think forking the Signal server and (later) adding federation could spark a community of free and secure messenger apps for the 21st century. Even if that means starting with an empty user base. After all, most of my contacts with Signal only ever use it to contact me, it wouldn't be difficult to convince them to change app.
    - pseudalopex 10y ago
      Apple imposes usage rules on software downloaded from the App Store. The GPL doesn't allow this, so Open Whisper Systems would have to offer different terms.
    - dlmetcalf 10y ago
      This is probably what made me double-take the most. I don't understand why Guardian were denied permission to use the code. Sure, OWS aren't under any obligation to, but it doesn't really feel like it's in the spirit of openness at all. The code might be 'open source', but it's neither free as in user freedom or beer, unless you're giving Facebook or Google your metadata now. If the code can't be reused, it's pretty hard for users to move to something else. Unless that something else is totally starting over again from scratch. It just feels weird that because of the Apple situation, in this case the GPL appears to work directly against user freedom. I think a lot of people have been taken by surprise by that, to discover the platform is not what they thought it was.
jkire 10y ago
I totally get this; federation is hard, and its so much easier and nicer ─ as a developer ─ to have a centralised service. Often centralisation is the right choice for a particular project, and I respect that. That doesn't mean we should give up trying to federate things, though. The major upside (for me) to federated systems is simple: choice. Choice over the client you use. Choice over the servers you use. Choice. For something so fundamental as how we communicate over the internet that's incredibly important. I want to be able to use the app I want without having to figure out if the other person is using the same app, and i don't want to have to have 10 different apps on my phone just to keep in touch with everyone. The reason I think that things like email and SMS are still so prevalent is how you don't need to care what service the other people are on, you don't need to worry about how other people are going to talk to you, it all just works. It's true that currently a lot of the email traffic is going through google, that sucks, but: - If google discontinues gmail then your email still works, people can still talk to you. It's annoying you need to get the new addresses for all your contacts that used to use gmail, but that will get resolved quickly enough. - A day may very well come where people decide another service is better, but then people are free to move there without having to try and convince all their friends and family to also switch. (Take browsers, for example, the browser that everyone uses to access the internet changes over time. And its great! People use what they want to use, the servers don't care, everything just works ─ well, more or less). Not everything needs to be federated, often there are situations where its advantageous for there to be a central authority. (The blog post highlights a few of those situations). That's also great! People who want those things are then free to choose to use those things as well. Using 2 apps you want to use is still better than 10 you don't want to use. Federation is hard, and it imposes constraints, but I really, really hope that we can build something that works before we give up and hand over all our comms to a particular company. This is exactly why I work at matrix.org; I just want to be able to use the clients I want without having to care what everyone else uses. I don't even have to care if matrix.org goes down (except the fact I have to fix it), I can happily talk to other people since I connect via my own server. I do actually agree with a lot of what's said, I just disagree with the conclusion. Federation iss hard, but it's just so worth it if we can get right.
- dividuum 10y ago
Arathorn 10y ago
So: I was the other side of the email conversation that Moxie alludes to in this post, so I'll respond to it here given comments are disabled on the OWS blog :) 1. Totally agreed that federation slows down evolution of the core protocol. However, higher level protocols do NOT have to be stuck in time forever in the manner of SMTP if you design them with a migration/evolution strategy from the outset. For instance, with Matrix.org the core premise is decentralising communication between different networks. This includes different versions of Matrix itself. We could build an entirely new Matrix 2.0 tomorrow, using (say) a fully p2p WebRTC Datachannel transport or IPFS or whatever rather than today's HTTP APIs... and simply bridge it into the existing Matrix network using Matrix's existing bridging architecture. In fact, one of our favourite thought experiments is the process of how to evolve Matrix into a full e2e-and-metadata-privacy-protecting protocol at some point in future - as per http://matrix.org/~matthew/2015-06-26%20Matrix%20Jardin%20Entropique.pdf http://matrix.org/~matthew/2015-06-26%20Matrix%20Jardin%20En.... How you protect metadata privacy if locked to a single centralised service like Signal is perhaps questionable. We believe that avoiding being dependent on any single communication provider is just as important to user freedom as maintaining communication privacy. If I don't trust Signal to host my communication reliably, I want to pick my own provider... whilst still being able to talk to other Signal users. One could argue that end-to-end encryption is incompatible with federation, as if a vulnerability is found in the encryption ratchet you need the power to upgrade the world 'instantly', as WhatsApp or whoever can. However, I am not convinced this is a showstopper. It's certainly convenient that WhatsApp can instantly lock out all versions of an app which are known to contain a vulnerability from their network, but you can get a similar effect by issuing a press release as per heardbleed, poodle etc; giving users a way to check if they're vulnerable; encouraging them to move to a provider or app which is not vulnerable. It's not watertight, but it's a compromise I think many would be willing to make in exchange for having the freedom to select their communication provider and communicate with who they choose. 2. If the federated protocol has well-defined layers, it's not even a problem if the lower layers are slow and hard to evolve; you just go and swap out the higher layer ones. Internet Protocol is so widely deployed it's infamously painful to evolve... but it's a relatively simple and well-focused problem domain, so a huge migration like ipv4->ipv6 happens very rarely. But it hasn't materially slowed down the adoption and uptake of the internet - in fact, ipv4 is a victim of its own success! Meanwhile, HTTP is also so widely deployed that it's a relatively slow process to do something like HTTP/1.x->HTTP/2. But, again, it turns out this isn't remotely a showstopping problem - most of the interesting stuff happens layered as applications on top of HTTP - HTTP does one thing, and does it relatively well, so it's a great building block. And it's the basis of the entirely open and interoperable architecture of the web. With Matrix, we're trying to provide a similarly simple and focused base layer: a decentralised pubsub/storage API layered on HTTP. We provide some modules that define additional semantics (messaging, VoIP etc) as layers on top of that core API (http://matrix.org/docs/spec/client_server/r0.1.0.html#modules http://matrix.org/docs/spec/client_server/r0.1.0.html#module...) which can evolve separately. We believe that having an open and interoperable protocol for securely synchronising data in realtime is as fundamental as the web itself, and this is NOT a function that should be silo'd into any provider. 3. The blog post says: "One of the controversial things we did with Signal early on was to build it as an unfederated service.". This seems a bit strange, given the TextSecure server itself exposes a basic federation API: https://github.com/WhisperSystems/TextSecure-Server/tree/master/src/main/java/org/whispersystems/textsecuregcm/federation https://github.com/WhisperSystems/TextSecure-Server/tree/mas... 4. My actual retort wasn't to tell Moxie that he's dumb - he's obviously an incredibly smart guy, even if I disagree with his position here. I actually said: "Okay. The internet wouldn't have got very far without interoperable application-layer protocols defined by 3rd parties, but if you would only ever consider using one for TextSecure that you invented yourself, that's your call :)" So, if anyone's reading this and still believes in the dream of an interoperable and federated internet, feel free to come hang out at https://matrix.to/#/#matrix:matrix.org https://matrix.to/#/#matrix:matrix.org and live the dream :)
deleted 10y ago
[deleted]
Qantourisc 10y ago
IMO you can still add federation (and you should for freedom and security), but no need to expose it to the user. For example: set the default server to signal servers. For sake of sanity, you probably want to route all servers to the default signal server. But you could easily allow user@server. If you can figure out how to use phone numbers directly between servers, congratulations, cause it will not be easy. But I think people who want to run their own server will not find that much of an issue. In terms of getting stuck in time, you could use though love, and require backwards compatibility for like a month but no-longer. But so long as you keep the phone-numbers @the signal servers, your server will be the reference, and any server not following will get the blame.
sinak 10y ago
I agree with Moxie's assertion: federated services are losing. As it stands they can't keep up with the rate of change and iteration currently offered by centralized protocols. But rather than this being a problem with federation itself, perhaps it's an issue with existing implementations. Things like XMPP give protocols so much flexibility with extensions that upgrades become costly and difficult to implement. Others, like HTTP, SMTP, and IRC are updated very slowly. It seems like it should be possible to implement federated protocols that are focussed on fast iterations. For example, a protocol could specify that each version has a fixed lifetime of just one month, and require all clients and servers update to the new release in the last week of that month. Such a fast-paced protocol would have some disadvantages over slower ones (e.g. longevity), but it seems better suited to the pace of modern software and service development. But I've never developed a protocol, let alone a federated one. I'd love to hear some potential problems or examples of failed protocols that have taken this kind of approach.
spydum 10y ago
> As a software developer, I envy writers, musicians, and filmmakers. Unlike software, when they create something, it is really done — forever. A recorded album can be just the same 20 years later, but software has to change. This is funny, a few lawyers I know, constantly complain they should have got into software.. Write once, sell many (pre-SaaS days..). They complain in their job, every day they have to go out and perform to continue to earn. Just funny how we ALL think the grass is greener on the other side (and tend to oversimplify).
kordless 10y ago
The term "federation" has been overused, misused and abused. At it's base layer, Federation can be thought of as process that unifies things and the people using the things in a way that is reproducible at a future date. Within the context of the Internet and Software (the cloud), Federation should mean "federated use of computer resources" and extend the three primitives of the cloud to be used by all, at will: 1. Compute - allows the transformation of one type of data to another. 2. Storage - allows the storage of data. 3. Network - allows the transfer of data from compute to storage, or vice versa. Federation's historic constituents have been standards, payments and identity. In order for cloud federation to be viable, the primitives above must be able to be used in a way that allows some type of "standardized trust" to be applied, some type of identity to be associated with it, and some way of paying for the resources used. One possible solution for implementing a Federated Cloud of Clouds is to use cryptocurrencies to pay for the processing of data, identify who's data it might be, and then encapsulate the data being federated into some type of standard. I have explored a generalized solution for this and have presented it to a few peers. It seems doable, if we can mobilize the development community's intent to change how software is written. BTW, I don't consider "standards" to be anything other than a light consensus mechanism applied to a set of data which is then made immutable. One might consider a blockchain a "standard" by the evidence it is unchanging, once created. Federation will only occur once we've moved infrastructure to using cryptocurrencies for deployments. It will also require a radical shift in the way we think about how software is written and paid for.
teknopaul 10y ago
Nonsense, the www is not stuck in the 90's. Browsers adapt, compete and innovate. And err http is at 2.0. The fact that gopher didn't kick off does not rubbish the idea of common protocols in the first place.
- mythz 10y ago
  HTTP is used for far more than just browsers. how broad is the current adoption of HTTP 2.0? Will there ever be a time when all HTTP clients, servers, proxies, etc will have upgraded to 2.0 - clearly that's never going to happen in the foreseeable future so we're stuck in a mishmash of supporting HTTP 1.x and HTTP 2.x clients and server software indefinitely - similar to XMPP extensions the article refers to. It's obviously much faster to innovate and update all clients when you have control over a centralized network and auto-updating clients.
  - moxie 10y ago
    What's more, HTTP's recent movement is arguably only because the web experience has centralized so much around Google. Google already had the top two most popular websites in the world, so once Google Chrome also became the most popular web client, it was easy for them to develop a new communication protocol of their own, which everyone else has more or less begrudgingly agreed to call HTTP/2 and start deploying as well.
    - bracewel 10y ago
      Google did not write the HTTP/2 spec. While Roberto Peon is one of the authors (and a core developer of the preceding SPDY protocol) saying that HTTP/2 is some Google invention is a bit of a punch in the face to the IETF and the two other authors who do not work at Google.
    - fensipens 10y ago
      So what? Many standards originated from some particular need of one or two companies popular at that time. Look at the current IETF WGs and the authors of the drafts they produce.
mempko 10y ago
Let me summarize. "We want to make sure we make money." Let's burn these star networks!
fensipens 10y ago
what Facebook did with email What did Facebook do with email?
- Tharkun 10y ago
  Bloody murder, basically. Whenever I try to communicate with someone under 30, e-mail is simply not an option. A Facebook message is often the only way (or worse).
- drjesusphd 10y ago
  Made it obsolete, didn't you hear?
Tharkun 10y ago
> An open source infrastructure for a centralized network now provides almost the same level of control as federated protocols, without giving up the ability to adapt. That's simply not true. Unless you're Google/Facebook/Apple/Amazon, you'll be unable to get anyone to use your centralized network. Which makes it worse than useless. "Build it and they will come" doesn't work in a world of walled gardens, curated content and censorship.
sly010 10y ago
I think I read this in The Mythical Man Month but writing from memory: Every software development project, no matter how well executed will produce outdated software. Said differently any software that is used is by definition outdated. This is simply, because unknown unknowns only become known unknowns during development. Bitcoin is a perfect example of this. It might stick around for a while mainly because so many people are already invested in it, but when it comes to features Ethereum runs circles around Bitcoin. On the other hand Ethereum will probably never be fully finished.
- Taek 10y ago
  Well, Ethereum features a turing complete scripting system and a ton of tools to make use of it, but it doesn't solve Bitcoins most serious problems, and is in fact worse off for most of them. Miner centralization is a huge problem for Bitcoin. The fact that 11 entities control >90% of the hashrate, and that 2 control >50% of the hashrate is a huge issue for Bitcoin. The pressures we understand to be causing this are more significant in Ethereum at scale. Ethereum is not even at scale and already is doing worse than Bitcoin in terms of hashrate centralization. Ethereum also has significant scalability problems. Bitcoin struggles to hit 4 transactions per second, and every transaction on Ethereum is substantially heavier per transaction than Bitcoin. That scripting system is not free, and already Ethereum is 20% as heavy as Bitcoin despite being 1/5th the age and having 1/10th the usage. And that's before the fancy apps like Augur have gained traction. Augur is significantly heavier than anything running today, and it's definitely not the heaviest app that people have gotten excited about. And the development is almost fully centralized. The current state of Ethereum is that what Vitalik says is what will happen, and while there are people who could shut him down... they won't. At least not in the near future. Ethereum has already performed hardforks and has several more planned. I agree that Bitcoin has taught us a lot of lessons that have paved the way for its obsolecense. But I don't think that Ethereum learned them, and I think that Ethereum will seem a lot less attractive after it has to deal with demand matching what is currently demanded from Bitcoin. I would almost go as far as saying they are orthogonal products. Bitcoin, above all else, prioritizes decentralization. Ethereum prioritizes features and flexibility, at the cost of scalability and centralization.
- Taek 10y ago
tehabe 10y ago
I think his view on the address book as your personal social graph is interesting. Because this has been mainly discussed as privacy violation because it involves uploading the address book to a server to check for in-network users. But it has also another side and sadly this side as an independent social graph is mostly ignored, at least in Germany. How is this being discussed in other places? I had this discussion when WhatsApp turned on full end to end encryption.
bad_user 10y ago
In general I agree with @moxie's arguments, however I've come to to be wary at his condescending attitude and discontent towards people disagreeing with his choices. Like there's a review on the Android Signal app complaining that it asks for too many permissions, and it does ask for an arm, a leg and your soul, with the reply being unprofessional imho. I understand that this is open-source, but if you publish it and want users, you need to act professional, otherwise it shouldn't be a surprise when people complain. To the subject at hand, I strongly disagree. Moxie says that email is frozen, being why it is unencrypted, blaming the lack of progress on it being federated. But you know, I'm willing to bet that in 10 years from now WhatsApp and Slack will be both dead, just like Yahoo Messenger and ICQ before them, while email and IRC will still be around. And I think that email is unbeatable, because it is federated, because it's governed by standards and because in spite of all constraints, it's quite adaptable, being the kind of platform supporting short term proprietary solutions because (and not in spite of) its client/server decoupling. Slack for us is just a long and one dimensional stream of greetings, jokes, warnings and meaningless blabbering. Email on the other hand is the archive of everything we do, being the Git of our interactions and the primary channel for knowledge dissemination. Is it unencrypted? Sure, but it doesn't matter though. Because we are willingly trading that for a capable search engine and a good web interface. Trade secrets aren't communicated over email anyway. I think Moxie is missing the point. He's emulating WhatsApp, but you can't beat WhatsApp at their own game. Did WhatsApp really deliver encryption to 1 billion users? Well, those are 1 billion users that probably won't use Signal or chat with Signal users. Oops.
- devishard 10y ago
  I think you're really missing the point here. > Moxie says that email is frozen, being why it is unencrypted, blaming the lack of progress on it being federated. But you know, I'm willing to bet that in 10 years from now WhatsApp and Slack will be both dead, just like Yahoo Messenger and ICQ before them, while email and IRC will still be around. Yes, and email will still be unencrypted, while the WhatsApps and Slacks of the future will probably be encrypted end-to-end. > And I think that email is unbeatable, because it is federated, because it's governed by standards and because in spite of all constraints, it's quite adaptable, being the kind of platform supporting short term proprietary solutions because (and not in spite of) its client/server decoupling. It's not adaptable enough to introduce meaningful encryption to it.
chatmasta 10y ago
Perhaps the "federation" that should evolve is not federation of the servers behind a single protocol or app, but federation between multiple apps. Applying the traditional definition of "federation" to Signal implies that any entity could host a Signal server, and any Signal client could talk to that server. Moxie mentions the main issue with this, namely that because of the "evolving ecosystem," it's impossible to keep all servers using the same protocol. Also consider that trust is another significant issue -- can you trust the operator of the Signal server you're connecting to? Can you trust not only that the server software is up to date, but also that no malicious actor has modified it? Because of these trust issues, the benefits of federation for users are unclear. Why connect to some random Signal server if you feel safer connecting to the official servers? If the answer to that question is, "because I can connect to my own Signal server, so I don't need to trust anyone," then I posit this question: Do you need permission to build a federated service? Is it possible that federation can evolve without any central planning or permission to do so? Consider the "messenger wars" of the early 2000s. There was no official "federation" of messaging services, and yet, people built clients that could interoperate with all of them. In fact, Microsoft itself wrote a client that was interoperable with AIM. [0] Why can't the same sort of "grassroots federation" evolve within the current ecosystem of chat apps? If somebody reverse engineers the whatsapp protocol and builds a "whatsapp server," then modifies the whatsapp binary to talk to it (perhaps using something like theos-jailed [1]) instead of the official servers, isn't that effectively accomplishing unsanctioned federation? Along the same lines, is there anything stopping somebody from reverse engineering Signal to talk to Whatsapp, and vice versa? The "walled gardens" have made it harder to reverse engineer binaries and protocols, but there is no fundamental impediment to achieving cross-messenger interoperability. If developers are motivated enough to build third party clients, it will always be possible, although the requisite reverse engineering effort grows more complex with every release. [0] https://nplusonemag.com/issue-19/essays/chat-wars/ https://nplusonemag.com/issue-19/essays/chat-wars/ [1] https://github.com/BishopFox/theos-jailed https://github.com/BishopFox/theos-jailed
- mcepl 10y ago
  See http://spectrum.im/ http://spectrum.im/ with https://github.com/davidgfnet/whatsapp-purple https://github.com/davidgfnet/whatsapp-purple (or use it with pidgin/adium/finch directly).
hyperion2010 10y ago
I would like to point to a very similar piece that was written about infrastructure for open science [0]. The basic argument is along similar lines, but adds another important wrinkle, which is that in federated systems centralization will eventually arise and often it will be controlled by people and organizations that the federated community do not trust and cannot control (Elsevier being the example in the scholarly world). It is often better to spend time building lasting open institutions that have principles that are in line with the values of the community and are federated at the social level rather than the technical level. Politics matters, and despite how much we might like to avoid it, sometimes we can use it to our advantage. 0. http://cameronneylon.net/blog/principles-for-open-scholarly-infrastructures/ http://cameronneylon.net/blog/principles-for-open-scholarly-...
RRRA 10y ago
This is very confusing and the argument don't hold very much. It looks sadly more like a justification against making the effort, which he has a right to not want to attempt, but doesn't make sense. The real issue is how can we have an open governance model that moves protocol forward a lot faster and it seems that with a little imagination we could tackle most of the pain points. So maybe we should fix the infrastructure instead of celebrating silos that don't talk to each other even when using the same protocols?
qb564 10y ago
I'm largely sympathetic to Moxie's core arguments, and to the extent that I disagree, it's his prerogative to do what he thinks is best with his project. I will point out that some people are using "federated" and "decentralized" as synonyms, when they are distinct concepts. The Tor network, for example, is decentralized, but not federated. You can run your own Tor node, but that node has to be accepted by the directory authorities in order to become a part of the network. You could conceivably set up your own Tor network, with your own directory authorities, but of course you can also create your own alternative network with Signal, since both the client and server code are free software. Ricochet, the messenger that uses client-side Tor hidden services to create server-less chat, is also decentralized, but not federated. Ricochet clients talk to other Ricochet clients. There is not IETF RFC out there for Tor or Ricochet, that you can use to build your own implementation and federate into the network. Moxie's claim that federation inevitably leads to moribund cruft may be over-stated; after all, http is not exactly in trouble. Nevertheless, I think that it's probably true that maintaining a federated protocol that is also an easy, seamless, user-facing piece of software is most likely a tall order.
heavenlyhash 10y ago
There's a sort of resignation about the tone here, both in the article and the comments, which is very depressing. Let's take that punch, roll with it, and redirect its energy. If centralized efforts, much like BDFLs, are looking like a good option for development cycles, okay. How can we wield that knowledge, but also build lifecycles around it that can help us buy back the powers and critical freedoms of federated systems?
kerkeslager 10y ago
Moxie: there's a third alternative to centralization and federation which nobody is talking about on this thread: P2P. I'd be interested to hear your thoughts on what pitfalls someone (me) might encounter trying to implement a peer-based messaging system. I've got a lot of learning to do before I feel qualified to actually write code for such a project, but it's something I'm actively working toward.
- Zash 10y ago
  If federation is too hard then I don't think P2P, which is even harder, will be considered.
  - kerkeslager 10y ago
    It's harder to deploy updatable encryption on a federated protocol than a centralized one, but as I see it, the difficulties are social rather than technical. Deploying updatable, usable encryption on P2P is technically more difficult than on federation or centralization, but I'm not convinced it's socially more difficult. The core problem of updating federated systems is that big players can choose not to play. If Google or Yahoo decide not to support a change to email, the change is dead in the water. This isn't just hard, it's intractable. But with P2P systems, non-reliance on peers doing anything in particular is built-in. For example, peers that don't support a change limit the size of the swarm you can peer with, but they don't inherently break the feature as long as there are some peers who support the change. Many Bittorrent clients support encryption and many don't, many bitcoin miners support alternate protocols to speed up pooled mining, and many don't. I don't know what the implications of this would be for a system like Signal. It maybe that it doesn't solve anything; I'm just not ready to conclude that for myself yet and I'd be interested to hear Moxie's take.
bikamonki 10y ago
Are we super sure about this premise: a vendor builds a centralized solution on top of (or evolved from) a federated protocol in order to be able to adapt with speed to a moving environment? For instance, I am not so sure that is the reason Whatsapp built a custom XMPP. I am more inclined to think that decentralized-anything is basically impossible to monetize so companies thinking for-profit will always chose the centralized walled garden.
- moxie 10y ago
  I'm writing from the perspective of Signal, which is not a business, so it's definitely not about monetizing users for us.
  - ge0rg 10y ago
    It would be interesting to know how you do see Signal, then. After reading your comments on the LibreSignal tracker, my impression was that you are very much writing with your business hat on, as you are framing alternative clients as "products" and trying to protect the Signal brand, as well as preventing third-party clients from using your service - all typical business moves. I can see how federation breaks the phone-number-based identity and discoverability, so I accept your position in that regard. However, opening up the service to community-developed clients would be A Good Thing. You have to maintain backward compatibility to a certain degree for the official client anyway. If community clients fail to roll out a new protocol version, it will be a valid move to lock them out together with outdated official installations. You wrote that maintaining servers is difficult, but I am sure that it is possible to find a scheme where your additional financial burden of accepting (but not providing support for) third-party clients is compensated for.
  - dlmetcalf 10y ago
    Good to hear that's remained the case @moxie.

226 points by tprynn 10y ago

mcepl 10y ago

Every security engineer I have met so far is to some extent (some of the to quite an extent) paranoid and control-freak. That's good for security engineers, you want them to be paranoid and control freaks. Of course, the bad part is that they are paranoid and control-freaks. So, although I don't question huge amount of work brought to the public for free by Moxie, I cannot notice that he killed all points of federation (even those which were to some extent under his control, http://www.cyanogenmod.org/blog/whisperpush-end-of-life http://www.cyanogenmod.org/blog/whisperpush-end-of-life), that he sabotages non-Google-controlled repositories (https://github.com/WhisperSystems/Signal-Android/issues/127 https://github.com/WhisperSystems/Signal-Android/issues/127) of his so called open software client. I don't care that much because for this and other reasons I won't touch Signal with a ten-feet pole, and federated alternatives (XMPP) work for me just fine, thank you. So, my suspicion is that instead of some theoretical difficulties the issue is that he got how own fiefdom under his control, and now he is not willing to let it go. All these opponents of federation and open software (I am not talking about just FLOSS here, I mean truly community-drive software), each with their fiefdoms to protect, argue that we are stuck with the open technologies from 1990s (or even earlier), and only their proprietary silo (proprietary meaning with only one implementation and under their control) is able to bring us a shiny new blink. However, they tend to ignore the other part of the story. The biggest problem of storing my life in proprietary silos is not that they are under control, I don’t live a life in delusion of importance, so I don’t expect anybody to mine my communication with my wife about what to do for dinner (and yet, even that communication is encrypted by a pretty good encryption system, which works just fine with a plain XMPP). The biggest problem is that every individual proprietary silos in the moment owner of such silo goes out of business, or for some other reasons looses interest in providing the service. So, the question is not how many technologies from 1990s we use, but how many proprietary technologies from 1990s we don't use. Yes, I have stored on my hard drive all emails since 1997, because old boring mbox files still work as they did. But all those exciting and sometimes tearful conversations I had on Yahoo! IM or ICQ are gone forever. That bigfoot.com and mail.com accounts I had for my life? Yeah. And to the second lie Moxies lies to himself (I am willing to believe he really means it). It is suddenly easy to move between proprietary silos? Yeah. It is lovely that we have to keep unique identifier (phone number) for all NSAs to trace us in some weird databases somewhere (that livejournal.com account? yes, it is now owned by the Russian Коммерса́нтъ; yes, I know that this newspaper is not that bad). And yes, it is seriously interesting if perhaps phone numbers would serve as the unique personal identifier most Americans were so afraid to introduce. A possibility to have anonymous accounts? Yeah. However, even this does not seem to make transfer between platforms that easy. If all your friends are on Telegram, then no matter how easy for whole group it would be easy to transfer, you cannot transfer unless whole group does. How many people really and truly love Twitter, and how many of those who hate it with passion are able to move to some better platform? Nobody, because that is where people are.

cm3 10y ago

Just because it's not super simple doesn't mean we should treat federation as a dead end. Sandstorm is one way for non-admins to have their own server and there's work regarding self-hosting everything. It's unsexy, it's not a money maker like a centralized messenger, but unless we want to live in a world where there's only one or two centralized messaging solutions, we as developers have to eat our own dog food and improve easy hosting and federation.

moxie 10y ago
How would apps running on Sandstorm avoid the problems with federation in today's world outlined in the blog post?
- kentonv 10y ago
  It looks to me like your article is really arguing for central control of the code, rather than central control of the serving infrastructure. Having everyone running the same code means changes can be made quickly because only one implementation needs to change. Centralized serving infrastructure makes central control of the code easy. But if you look at mobile, we have central control of the code there without central control of the infrastructure: everyone has their own phone, yet runs the same apps that stay up-to-date via app stores. Sandstorm is basically taking that model to servers: within 24 hours of an app update, everyone gets a notification and can click one button to update their copy, so everyone tends to stay up-to-date. (Disclosure: I'm the lead developer of Sandstorm.)
  - pdkl95 10y ago
    > central control of the code That's just an excuse to not implement backwards compatibility in the protocol. Central control is also a security risk, as we've seen recently with Apple and the FBI. If the code is controlled centrally, the FBI (or whomever) only has to strong-arm one target. > gets a notification and can click one button to update their copy That sounds like you're trying to keep the user in control, which is great. I haven't head of Sandstorm before, but it sounds interesting. Have you considered the problem of forced updates? I'm not sure there is a good way to protect against the FBI trying something similar to the recent drama with Apple, where they order you to force a "special" update on {someone,everyone}.

marssaxman 10y ago

That's one of the most depressing things I've read in a while. If this is actually true, the internet has failed.

tprynn 10y ago

I'll add my thoughts here, since I think a number of people will be disappointed in this post. There are a number of competing goals when developing software that "the whole world can/will use". And it seems obvious to me that getting the whole world to use a secure, encrypted messenger like Signal (now, Whatsapp) is a noble goal. The obvious competing goal is federation, and the fact that we've failed at improving federated services (e-mail being the best/most important example) is a problem. Ultimately, Moxie's arguments here hold a lot of sway for me. I think it sucks that constantly-improving services are impossible to address in a federated way, and he's not really offering a solution to that. But instead he presents arguments I hadn't thought about much, which is that perhaps non-federated services can be dramatically better than federated services in the short-term. And in the long term, perhaps we'll move back to federated services; right now, the cost of changing services/ecosystems is very low. So we don't need to be as 'doom-and-gloom' about the fact that Signal does not achieve all the competing goals, because it achieves a number of extremely important goals right now, and we can work on the other goals in the future.

marssaxman 10y ago
It's precisely the fact that Moxie is so credible on this topic that makes his post so frustrating. I believe that he's telling the truth as he sees it, and I believe that he's as well-situated as anyone could be to pull off a white-hat secure messaging system, so if he says it can't be done with federation, it's awfully hard for me to assert that he's wrong. The problem is that I don't see the point of doing any of this without federation. If we're just using the Internet as a substrate for a bunch of proprietary silo services, then we're just back to the way things were before - walled gardens and media conglomerates and extortionate Bell System-style lock-in shenanigans. So what? Why bother? What have we gained, with all this effort, if we're back in the same old shit again? The whole point was that we no longer needed to get anyone's permission to communicate, to publish, to build new services for interacting with each other; we could just do it. If federation is gone, then those days are gone, and the most important reason to care about the Internet is gone, too.
- moxie 10y ago

dsp1234 10y ago

No users, no walled garden. More users, more walled garden. Is this just the natural state of systems that require/acquire network effects?

skybrian 10y ago
I have a universal theory about this which is "spam and abuse make everything suck". This is why we can't have an open libertarian Internet, and instead have walled gardens and moderated forums - ordinary users don't want to deal with the crap. It's also why we don't have open API's, and why the API's that the big providers do give us tend to be nerfed and then discontinued - they're constantly abused, which imposes a maintenance cost that businesses have to justify. It's why an email server is so difficult to run and email delivery is so unreliable. It's why you can't (or shouldn't) rely too much on Paypal. It's why governments effectively outsource Internet policing to big providers. And so on.
dlmetcalf 10y ago
Not necessarily. Microsoft kept trying to make a walled garden around the web. They might have had a lot of users, but the web fought back and luckily today we have 'world wide web', not a totally isolated Microsoft Web, Google Web, Apple Web, Firefox Web etc. I hated the days when you constantly had to switch web browsers to get pages to render. I'm glad it's rare now, because we didn't give up on the hope of open systems. Mark might have lost the battle to have HTTP/2 encrypted by default, but at least when Let's Encrypt came out, it still worked in all browsers. Would we have been better to just give up and have lots of walled gardens instead? I don't think so. I'm glad we have WWW. Personally I don't think XMPP failed because being open made it slow to adapt. I think we're mostly stuck because ICQ, AOL & MSN just got there first. I don't envy being in Moxie & OWS's position of figuring out how to make this work. I'll add opinion and some of it harsh, though I'll reserve criticism until I've built something as awesomely globally deployed as Signal.

kkl 10y ago

Some relevant background information on this issue: https://github.com/LibreSignal/LibreSignal/issues/37 https://github.com/LibreSignal/LibreSignal/issues/37

ngrilly 10y ago
Interesting discussion.
- cpach 10y ago
  Definitely. The background convinced me even more that the essay really makes sense.
- mbakke 10y ago
  I don't understand how Guardian were unable to obtain a license to use Signal/Axolotl in Chatsecure. The libraries on Github are GPL3, is there some missing IP I'm not aware of? Also it saddens me that LibreSignal is effectively dead now, along with any future third-party clients. I think forking the Signal server and (later) adding federation could spark a community of free and secure messenger apps for the 21st century. Even if that means starting with an empty user base. After all, most of my contacts with Signal only ever use it to contact me, it wouldn't be difficult to convince them to change app.
  - pseudalopex 10y ago
    Apple imposes usage rules on software downloaded from the App Store. The GPL doesn't allow this, so Open Whisper Systems would have to offer different terms.
  - dlmetcalf 10y ago
    This is probably what made me double-take the most. I don't understand why Guardian were denied permission to use the code. Sure, OWS aren't under any obligation to, but it doesn't really feel like it's in the spirit of openness at all. The code might be 'open source', but it's neither free as in user freedom or beer, unless you're giving Facebook or Google your metadata now. If the code can't be reused, it's pretty hard for users to move to something else. Unless that something else is totally starting over again from scratch. It just feels weird that because of the Apple situation, in this case the GPL appears to work directly against user freedom. I think a lot of people have been taken by surprise by that, to discover the platform is not what they thought it was.

jkire 10y ago

I totally get this; federation is hard, and its so much easier and nicer ─ as a developer ─ to have a centralised service. Often centralisation is the right choice for a particular project, and I respect that. That doesn't mean we should give up trying to federate things, though. The major upside (for me) to federated systems is simple: choice. Choice over the client you use. Choice over the servers you use. Choice. For something so fundamental as how we communicate over the internet that's incredibly important. I want to be able to use the app I want without having to figure out if the other person is using the same app, and i don't want to have to have 10 different apps on my phone just to keep in touch with everyone. The reason I think that things like email and SMS are still so prevalent is how you don't need to care what service the other people are on, you don't need to worry about how other people are going to talk to you, it all just works. It's true that currently a lot of the email traffic is going through google, that sucks, but: - If google discontinues gmail then your email still works, people can still talk to you. It's annoying you need to get the new addresses for all your contacts that used to use gmail, but that will get resolved quickly enough. - A day may very well come where people decide another service is better, but then people are free to move there without having to try and convince all their friends and family to also switch. (Take browsers, for example, the browser that everyone uses to access the internet changes over time. And its great! People use what they want to use, the servers don't care, everything just works ─ well, more or less). Not everything needs to be federated, often there are situations where its advantageous for there to be a central authority. (The blog post highlights a few of those situations). That's also great! People who want those things are then free to choose to use those things as well. Using 2 apps you want to use is still better than 10 you don't want to use. Federation is hard, and it imposes constraints, but I really, really hope that we can build something that works before we give up and hand over all our comms to a particular company. This is exactly why I work at matrix.org; I just want to be able to use the clients I want without having to care what everyone else uses. I don't even have to care if matrix.org goes down (except the fact I have to fix it), I can happily talk to other people since I connect via my own server. I do actually agree with a lot of what's said, I just disagree with the conclusion. Federation iss hard, but it's just so worth it if we can get right.

dividuum 10y ago

Arathorn 10y ago

So: I was the other side of the email conversation that Moxie alludes to in this post, so I'll respond to it here given comments are disabled on the OWS blog :) 1. Totally agreed that federation slows down evolution of the core protocol. However, higher level protocols do NOT have to be stuck in time forever in the manner of SMTP if you design them with a migration/evolution strategy from the outset. For instance, with Matrix.org the core premise is decentralising communication between different networks. This includes different versions of Matrix itself. We could build an entirely new Matrix 2.0 tomorrow, using (say) a fully p2p WebRTC Datachannel transport or IPFS or whatever rather than today's HTTP APIs... and simply bridge it into the existing Matrix network using Matrix's existing bridging architecture. In fact, one of our favourite thought experiments is the process of how to evolve Matrix into a full e2e-and-metadata-privacy-protecting protocol at some point in future - as per http://matrix.org/~matthew/2015-06-26%20Matrix%20Jardin%20Entropique.pdf http://matrix.org/~matthew/2015-06-26%20Matrix%20Jardin%20En.... How you protect metadata privacy if locked to a single centralised service like Signal is perhaps questionable. We believe that avoiding being dependent on any single communication provider is just as important to user freedom as maintaining communication privacy. If I don't trust Signal to host my communication reliably, I want to pick my own provider... whilst still being able to talk to other Signal users. One could argue that end-to-end encryption is incompatible with federation, as if a vulnerability is found in the encryption ratchet you need the power to upgrade the world 'instantly', as WhatsApp or whoever can. However, I am not convinced this is a showstopper. It's certainly convenient that WhatsApp can instantly lock out all versions of an app which are known to contain a vulnerability from their network, but you can get a similar effect by issuing a press release as per heardbleed, poodle etc; giving users a way to check if they're vulnerable; encouraging them to move to a provider or app which is not vulnerable. It's not watertight, but it's a compromise I think many would be willing to make in exchange for having the freedom to select their communication provider and communicate with who they choose. 2. If the federated protocol has well-defined layers, it's not even a problem if the lower layers are slow and hard to evolve; you just go and swap out the higher layer ones. Internet Protocol is so widely deployed it's infamously painful to evolve... but it's a relatively simple and well-focused problem domain, so a huge migration like ipv4->ipv6 happens very rarely. But it hasn't materially slowed down the adoption and uptake of the internet - in fact, ipv4 is a victim of its own success! Meanwhile, HTTP is also so widely deployed that it's a relatively slow process to do something like HTTP/1.x->HTTP/2. But, again, it turns out this isn't remotely a showstopping problem - most of the interesting stuff happens layered as applications on top of HTTP - HTTP does one thing, and does it relatively well, so it's a great building block. And it's the basis of the entirely open and interoperable architecture of the web. With Matrix, we're trying to provide a similarly simple and focused base layer: a decentralised pubsub/storage API layered on HTTP. We provide some modules that define additional semantics (messaging, VoIP etc) as layers on top of that core API (http://matrix.org/docs/spec/client_server/r0.1.0.html#modules http://matrix.org/docs/spec/client_server/r0.1.0.html#module...) which can evolve separately. We believe that having an open and interoperable protocol for securely synchronising data in realtime is as fundamental as the web itself, and this is NOT a function that should be silo'd into any provider. 3. The blog post says: "One of the controversial things we did with Signal early on was to build it as an unfederated service.". This seems a bit strange, given the TextSecure server itself exposes a basic federation API: https://github.com/WhisperSystems/TextSecure-Server/tree/master/src/main/java/org/whispersystems/textsecuregcm/federation https://github.com/WhisperSystems/TextSecure-Server/tree/mas... 4. My actual retort wasn't to tell Moxie that he's dumb - he's obviously an incredibly smart guy, even if I disagree with his position here. I actually said: "Okay. The internet wouldn't have got very far without interoperable application-layer protocols defined by 3rd parties, but if you would only ever consider using one for TextSecure that you invented yourself, that's your call :)" So, if anyone's reading this and still believes in the dream of an interoperable and federated internet, feel free to come hang out at https://matrix.to/#/#matrix:matrix.org https://matrix.to/#/#matrix:matrix.org and live the dream :)

deleted 10y ago

[deleted]

Qantourisc 10y ago

IMO you can still add federation (and you should for freedom and security), but no need to expose it to the user. For example: set the default server to signal servers. For sake of sanity, you probably want to route all servers to the default signal server. But you could easily allow user@server. If you can figure out how to use phone numbers directly between servers, congratulations, cause it will not be easy. But I think people who want to run their own server will not find that much of an issue. In terms of getting stuck in time, you could use though love, and require backwards compatibility for like a month but no-longer. But so long as you keep the phone-numbers @the signal servers, your server will be the reference, and any server not following will get the blame.

sinak 10y ago

I agree with Moxie's assertion: federated services are losing. As it stands they can't keep up with the rate of change and iteration currently offered by centralized protocols. But rather than this being a problem with federation itself, perhaps it's an issue with existing implementations. Things like XMPP give protocols so much flexibility with extensions that upgrades become costly and difficult to implement. Others, like HTTP, SMTP, and IRC are updated very slowly. It seems like it should be possible to implement federated protocols that are focussed on fast iterations. For example, a protocol could specify that each version has a fixed lifetime of just one month, and require all clients and servers update to the new release in the last week of that month. Such a fast-paced protocol would have some disadvantages over slower ones (e.g. longevity), but it seems better suited to the pace of modern software and service development. But I've never developed a protocol, let alone a federated one. I'd love to hear some potential problems or examples of failed protocols that have taken this kind of approach.

spydum 10y ago

> As a software developer, I envy writers, musicians, and filmmakers. Unlike software, when they create something, it is really done — forever. A recorded album can be just the same 20 years later, but software has to change. This is funny, a few lawyers I know, constantly complain they should have got into software.. Write once, sell many (pre-SaaS days..). They complain in their job, every day they have to go out and perform to continue to earn. Just funny how we ALL think the grass is greener on the other side (and tend to oversimplify).

kordless 10y ago

The term "federation" has been overused, misused and abused. At it's base layer, Federation can be thought of as process that unifies things and the people using the things in a way that is reproducible at a future date. Within the context of the Internet and Software (the cloud), Federation should mean "federated use of computer resources" and extend the three primitives of the cloud to be used by all, at will: 1. Compute - allows the transformation of one type of data to another. 2. Storage - allows the storage of data. 3. Network - allows the transfer of data from compute to storage, or vice versa. Federation's historic constituents have been standards, payments and identity. In order for cloud federation to be viable, the primitives above must be able to be used in a way that allows some type of "standardized trust" to be applied, some type of identity to be associated with it, and some way of paying for the resources used. One possible solution for implementing a Federated Cloud of Clouds is to use cryptocurrencies to pay for the processing of data, identify who's data it might be, and then encapsulate the data being federated into some type of standard. I have explored a generalized solution for this and have presented it to a few peers. It seems doable, if we can mobilize the development community's intent to change how software is written. BTW, I don't consider "standards" to be anything other than a light consensus mechanism applied to a set of data which is then made immutable. One might consider a blockchain a "standard" by the evidence it is unchanging, once created. Federation will only occur once we've moved infrastructure to using cryptocurrencies for deployments. It will also require a radical shift in the way we think about how software is written and paid for.

teknopaul 10y ago

Nonsense, the www is not stuck in the 90's. Browsers adapt, compete and innovate. And err http is at 2.0. The fact that gopher didn't kick off does not rubbish the idea of common protocols in the first place.

mythz 10y ago
HTTP is used for far more than just browsers. how broad is the current adoption of HTTP 2.0? Will there ever be a time when all HTTP clients, servers, proxies, etc will have upgraded to 2.0 - clearly that's never going to happen in the foreseeable future so we're stuck in a mishmash of supporting HTTP 1.x and HTTP 2.x clients and server software indefinitely - similar to XMPP extensions the article refers to. It's obviously much faster to innovate and update all clients when you have control over a centralized network and auto-updating clients.
- moxie 10y ago
  What's more, HTTP's recent movement is arguably only because the web experience has centralized so much around Google. Google already had the top two most popular websites in the world, so once Google Chrome also became the most popular web client, it was easy for them to develop a new communication protocol of their own, which everyone else has more or less begrudgingly agreed to call HTTP/2 and start deploying as well.
  - bracewel 10y ago
    Google did not write the HTTP/2 spec. While Roberto Peon is one of the authors (and a core developer of the preceding SPDY protocol) saying that HTTP/2 is some Google invention is a bit of a punch in the face to the IETF and the two other authors who do not work at Google.
  - fensipens 10y ago
    So what? Many standards originated from some particular need of one or two companies popular at that time. Look at the current IETF WGs and the authors of the drafts they produce.

mempko 10y ago

Let me summarize. "We want to make sure we make money." Let's burn these star networks!

fensipens 10y ago

what Facebook did with email What did Facebook do with email?

Tharkun 10y ago
Bloody murder, basically. Whenever I try to communicate with someone under 30, e-mail is simply not an option. A Facebook message is often the only way (or worse).
drjesusphd 10y ago
Made it obsolete, didn't you hear?

Tharkun 10y ago

> An open source infrastructure for a centralized network now provides almost the same level of control as federated protocols, without giving up the ability to adapt. That's simply not true. Unless you're Google/Facebook/Apple/Amazon, you'll be unable to get anyone to use your centralized network. Which makes it worse than useless. "Build it and they will come" doesn't work in a world of walled gardens, curated content and censorship.

sly010 10y ago

I think I read this in The Mythical Man Month but writing from memory: Every software development project, no matter how well executed will produce outdated software. Said differently any software that is used is by definition outdated. This is simply, because unknown unknowns only become known unknowns during development. Bitcoin is a perfect example of this. It might stick around for a while mainly because so many people are already invested in it, but when it comes to features Ethereum runs circles around Bitcoin. On the other hand Ethereum will probably never be fully finished.

Taek 10y ago
Well, Ethereum features a turing complete scripting system and a ton of tools to make use of it, but it doesn't solve Bitcoins most serious problems, and is in fact worse off for most of them. Miner centralization is a huge problem for Bitcoin. The fact that 11 entities control >90% of the hashrate, and that 2 control >50% of the hashrate is a huge issue for Bitcoin. The pressures we understand to be causing this are more significant in Ethereum at scale. Ethereum is not even at scale and already is doing worse than Bitcoin in terms of hashrate centralization. Ethereum also has significant scalability problems. Bitcoin struggles to hit 4 transactions per second, and every transaction on Ethereum is substantially heavier per transaction than Bitcoin. That scripting system is not free, and already Ethereum is 20% as heavy as Bitcoin despite being 1/5th the age and having 1/10th the usage. And that's before the fancy apps like Augur have gained traction. Augur is significantly heavier than anything running today, and it's definitely not the heaviest app that people have gotten excited about. And the development is almost fully centralized. The current state of Ethereum is that what Vitalik says is what will happen, and while there are people who could shut him down... they won't. At least not in the near future. Ethereum has already performed hardforks and has several more planned. I agree that Bitcoin has taught us a lot of lessons that have paved the way for its obsolecense. But I don't think that Ethereum learned them, and I think that Ethereum will seem a lot less attractive after it has to deal with demand matching what is currently demanded from Bitcoin. I would almost go as far as saying they are orthogonal products. Bitcoin, above all else, prioritizes decentralization. Ethereum prioritizes features and flexibility, at the cost of scalability and centralization.
Taek 10y ago

tehabe 10y ago

I think his view on the address book as your personal social graph is interesting. Because this has been mainly discussed as privacy violation because it involves uploading the address book to a server to check for in-network users. But it has also another side and sadly this side as an independent social graph is mostly ignored, at least in Germany. How is this being discussed in other places? I had this discussion when WhatsApp turned on full end to end encryption.

bad_user 10y ago

In general I agree with @moxie's arguments, however I've come to to be wary at his condescending attitude and discontent towards people disagreeing with his choices. Like there's a review on the Android Signal app complaining that it asks for too many permissions, and it does ask for an arm, a leg and your soul, with the reply being unprofessional imho. I understand that this is open-source, but if you publish it and want users, you need to act professional, otherwise it shouldn't be a surprise when people complain. To the subject at hand, I strongly disagree. Moxie says that email is frozen, being why it is unencrypted, blaming the lack of progress on it being federated. But you know, I'm willing to bet that in 10 years from now WhatsApp and Slack will be both dead, just like Yahoo Messenger and ICQ before them, while email and IRC will still be around. And I think that email is unbeatable, because it is federated, because it's governed by standards and because in spite of all constraints, it's quite adaptable, being the kind of platform supporting short term proprietary solutions because (and not in spite of) its client/server decoupling. Slack for us is just a long and one dimensional stream of greetings, jokes, warnings and meaningless blabbering. Email on the other hand is the archive of everything we do, being the Git of our interactions and the primary channel for knowledge dissemination. Is it unencrypted? Sure, but it doesn't matter though. Because we are willingly trading that for a capable search engine and a good web interface. Trade secrets aren't communicated over email anyway. I think Moxie is missing the point. He's emulating WhatsApp, but you can't beat WhatsApp at their own game. Did WhatsApp really deliver encryption to 1 billion users? Well, those are 1 billion users that probably won't use Signal or chat with Signal users. Oops.

devishard 10y ago
I think you're really missing the point here. > Moxie says that email is frozen, being why it is unencrypted, blaming the lack of progress on it being federated. But you know, I'm willing to bet that in 10 years from now WhatsApp and Slack will be both dead, just like Yahoo Messenger and ICQ before them, while email and IRC will still be around. Yes, and email will still be unencrypted, while the WhatsApps and Slacks of the future will probably be encrypted end-to-end. > And I think that email is unbeatable, because it is federated, because it's governed by standards and because in spite of all constraints, it's quite adaptable, being the kind of platform supporting short term proprietary solutions because (and not in spite of) its client/server decoupling. It's not adaptable enough to introduce meaningful encryption to it.

chatmasta 10y ago

Perhaps the "federation" that should evolve is not federation of the servers behind a single protocol or app, but federation between multiple apps. Applying the traditional definition of "federation" to Signal implies that any entity could host a Signal server, and any Signal client could talk to that server. Moxie mentions the main issue with this, namely that because of the "evolving ecosystem," it's impossible to keep all servers using the same protocol. Also consider that trust is another significant issue -- can you trust the operator of the Signal server you're connecting to? Can you trust not only that the server software is up to date, but also that no malicious actor has modified it? Because of these trust issues, the benefits of federation for users are unclear. Why connect to some random Signal server if you feel safer connecting to the official servers? If the answer to that question is, "because I can connect to my own Signal server, so I don't need to trust anyone," then I posit this question: Do you need permission to build a federated service? Is it possible that federation can evolve without any central planning or permission to do so? Consider the "messenger wars" of the early 2000s. There was no official "federation" of messaging services, and yet, people built clients that could interoperate with all of them. In fact, Microsoft itself wrote a client that was interoperable with AIM. [0] Why can't the same sort of "grassroots federation" evolve within the current ecosystem of chat apps? If somebody reverse engineers the whatsapp protocol and builds a "whatsapp server," then modifies the whatsapp binary to talk to it (perhaps using something like theos-jailed [1]) instead of the official servers, isn't that effectively accomplishing unsanctioned federation? Along the same lines, is there anything stopping somebody from reverse engineering Signal to talk to Whatsapp, and vice versa? The "walled gardens" have made it harder to reverse engineer binaries and protocols, but there is no fundamental impediment to achieving cross-messenger interoperability. If developers are motivated enough to build third party clients, it will always be possible, although the requisite reverse engineering effort grows more complex with every release. [0] https://nplusonemag.com/issue-19/essays/chat-wars/ https://nplusonemag.com/issue-19/essays/chat-wars/ [1] https://github.com/BishopFox/theos-jailed https://github.com/BishopFox/theos-jailed

mcepl 10y ago
See http://spectrum.im/ http://spectrum.im/ with https://github.com/davidgfnet/whatsapp-purple https://github.com/davidgfnet/whatsapp-purple (or use it with pidgin/adium/finch directly).

hyperion2010 10y ago

I would like to point to a very similar piece that was written about infrastructure for open science [0]. The basic argument is along similar lines, but adds another important wrinkle, which is that in federated systems centralization will eventually arise and often it will be controlled by people and organizations that the federated community do not trust and cannot control (Elsevier being the example in the scholarly world). It is often better to spend time building lasting open institutions that have principles that are in line with the values of the community and are federated at the social level rather than the technical level. Politics matters, and despite how much we might like to avoid it, sometimes we can use it to our advantage. 0. http://cameronneylon.net/blog/principles-for-open-scholarly-infrastructures/ http://cameronneylon.net/blog/principles-for-open-scholarly-...

RRRA 10y ago

This is very confusing and the argument don't hold very much. It looks sadly more like a justification against making the effort, which he has a right to not want to attempt, but doesn't make sense. The real issue is how can we have an open governance model that moves protocol forward a lot faster and it seems that with a little imagination we could tackle most of the pain points. So maybe we should fix the infrastructure instead of celebrating silos that don't talk to each other even when using the same protocols?

qb564 10y ago

I'm largely sympathetic to Moxie's core arguments, and to the extent that I disagree, it's his prerogative to do what he thinks is best with his project. I will point out that some people are using "federated" and "decentralized" as synonyms, when they are distinct concepts. The Tor network, for example, is decentralized, but not federated. You can run your own Tor node, but that node has to be accepted by the directory authorities in order to become a part of the network. You could conceivably set up your own Tor network, with your own directory authorities, but of course you can also create your own alternative network with Signal, since both the client and server code are free software. Ricochet, the messenger that uses client-side Tor hidden services to create server-less chat, is also decentralized, but not federated. Ricochet clients talk to other Ricochet clients. There is not IETF RFC out there for Tor or Ricochet, that you can use to build your own implementation and federate into the network. Moxie's claim that federation inevitably leads to moribund cruft may be over-stated; after all, http is not exactly in trouble. Nevertheless, I think that it's probably true that maintaining a federated protocol that is also an easy, seamless, user-facing piece of software is most likely a tall order.

heavenlyhash 10y ago

There's a sort of resignation about the tone here, both in the article and the comments, which is very depressing. Let's take that punch, roll with it, and redirect its energy. If centralized efforts, much like BDFLs, are looking like a good option for development cycles, okay. How can we wield that knowledge, but also build lifecycles around it that can help us buy back the powers and critical freedoms of federated systems?

kerkeslager 10y ago

Moxie: there's a third alternative to centralization and federation which nobody is talking about on this thread: P2P. I'd be interested to hear your thoughts on what pitfalls someone (me) might encounter trying to implement a peer-based messaging system. I've got a lot of learning to do before I feel qualified to actually write code for such a project, but it's something I'm actively working toward.

Zash 10y ago
If federation is too hard then I don't think P2P, which is even harder, will be considered.
- kerkeslager 10y ago
  It's harder to deploy updatable encryption on a federated protocol than a centralized one, but as I see it, the difficulties are social rather than technical. Deploying updatable, usable encryption on P2P is technically more difficult than on federation or centralization, but I'm not convinced it's socially more difficult. The core problem of updating federated systems is that big players can choose not to play. If Google or Yahoo decide not to support a change to email, the change is dead in the water. This isn't just hard, it's intractable. But with P2P systems, non-reliance on peers doing anything in particular is built-in. For example, peers that don't support a change limit the size of the swarm you can peer with, but they don't inherently break the feature as long as there are some peers who support the change. Many Bittorrent clients support encryption and many don't, many bitcoin miners support alternate protocols to speed up pooled mining, and many don't. I don't know what the implications of this would be for a system like Signal. It maybe that it doesn't solve anything; I'm just not ready to conclude that for myself yet and I'd be interested to hear Moxie's take.

bikamonki 10y ago

Are we super sure about this premise: a vendor builds a centralized solution on top of (or evolved from) a federated protocol in order to be able to adapt with speed to a moving environment? For instance, I am not so sure that is the reason Whatsapp built a custom XMPP. I am more inclined to think that decentralized-anything is basically impossible to monetize so companies thinking for-profit will always chose the centralized walled garden.

moxie 10y ago
I'm writing from the perspective of Signal, which is not a business, so it's definitely not about monetizing users for us.
- ge0rg 10y ago
  It would be interesting to know how you do see Signal, then. After reading your comments on the LibreSignal tracker, my impression was that you are very much writing with your business hat on, as you are framing alternative clients as "products" and trying to protect the Signal brand, as well as preventing third-party clients from using your service - all typical business moves. I can see how federation breaks the phone-number-based identity and discoverability, so I accept your position in that regard. However, opening up the service to community-developed clients would be A Good Thing. You have to maintain backward compatibility to a certain degree for the official client anyway. If community clients fail to roll out a new protocol version, it will be a valid move to lock them out together with outdated official installations. You wrote that maintaining servers is difficult, but I am sure that it is possible to find a scheme where your additional financial burden of accepting (but not providing support for) third-party clients is compensated for.
- dlmetcalf 10y ago
  Good to hear that's remained the case @moxie.