8 ms·

The Mission to Decentralize the Internet

77 points by ghosh 13y ago

mark_l_watson 13y ago
Good article that hits on an important subject: decentralized privacy-enhancing systems have a tough battle against easy to use consumer friendly services from Google, Facebook, etc. I have some hope that local grid networks will catch on. Also cheap appliances that are easy to use and offer local cloud services may eventually catch on. Not to sound too political, but this is the fight between large corporations and/or financial elites firming up their control of everything. I am not even sure how much personal effort I will put into these peoples' causes, and I am a long time supporter of the FSF, EFF, and ACLU.
jokoon 13y ago
Centralized systems are almost asking to be spying on, laws or not. Even if the NSA does not have the right to do what it did, I'm sure they'd end up doing by using corruption, and there are many ways to do it. Worse, it would open the door to a lot of other shady practices, and I'm sure criminal would make a buck out of it. The fact that a huge quantity of information is put in one place increases the risk it will peeked into by a lot. Decentralized systems are a little more difficult to maintain and develop, but are so much cheaper and eliminate the risk of mass surveillance. There are still security risks depending on designs, but honestly I think decentralized has a lot of future, the advantages and features it brings counter balances the disadvantages by a lot. I wish bittorrent inc would produce more technologies and some competitor would really be cool.
- salient 13y ago
  I agree. As long as big companies become so good at tracking and making money through surveillance themselves, it's just too irresistible for spy agencies not to try and get that data one way or another. That's why companies need to at least use forward secrecy, keep data at rest encrypted, and also delete more ephemeral data a lot more quickly. Google used to delete the data they have on their users every 6 months. I don't think they plan on ever deleting anything they have anymore now. That's the wrong way to go about it. Tracking data should not exist for more than a few months at most, and private communications should not be held for more than a year. Of course, it would be so much easier for the user, if everything was end-to-end encrypted, and then it wouldn't need to be deleted.
znowi 13y ago
> On Monday, eight major tech firms, many of them competitors, united to demand an overhaul of government transparency and surveillance laws. Not long ago they united under the PRISM project to spy on their users. What a quick change of heart. And again, there's already a similar call for reform started by Mozilla months ago (in June), which none of the PRISM companies supported. They were busy releasing copycat reports denying everything back then. https://stopwatching.us/ https://stopwatching.us/
- salient 13y ago
  > Not long ago they united under the PRISM project to spy on their users. What a quick change of heart. Yes, but then they did it for your safety, and because they care about you. Now they're doing it because...well, they're starting to lose money.
  - amirmc 13y ago
    A slightly less cynical view: They were in the dark about who else was involved and didn't fancy picking a fight with USG alone (and with an unknown amount of public support). Now that things became public and they know who else is involved, they can at least co-ordinate, knowing that at least one segment of the public cares.
    - charonn0 13y ago
      It's always easier to oppress the man who believes he stands alone.
psc 13y ago
While Google, Amazon, etc. being centralized might be bad in some ways (and good in others), I'm less concerned with that kind of centralization (because there are alternatives). The most dangerous centralization is at the ISP level, because it's the first gateway to the internet. When I saw the article title I was hoping it would be about that. Unfortunately, wired internet connections make for a really good natural monopoly, so until we have super fast wireless internet everywhere, I don't expect a solution. If any wifi mesh net appears, it would be a good start, but I'm skeptical it would become widespread. The other dangerous centralization of the internet is the DNS system. I know, technically it's decentralized because there even thousands of root name servers. But politically it's centralized under ICANN. Domains are controlled by registrars and governments. That's not necessarily a bad thing, but a truly distributed system would help guarantee a neutral internet. There are some good solutions for this already, namely Namecoin (http://dot-bit.org http://dot-bit.org), which uses proof of work and a bitcoin-like blockchain as a P2P DNS. Namecoin uses the .bit TLD, but probably the largest barrier to adoption is that it's not accessible by default on devices because it's not ICANN sanctioned and therefore not in the root name servers. There are workarounds for this. ICANN could add Namecoin to the root servers but then it would still be under the control of ICANN. The true solution would be for ISPs to add it to their DNS. If the ISP decentralization is solved, this becomes trivial. In my opinion, this is the only real way to guarantee net neutrality and prevent censorship.
- lukifer 13y ago
  I absolutely agree that ISPs are the nexus point for a truly decentralized internet. I've honestly been surprised that ISPs haven't leveraged/abused their oligopoly more than they have. > If any wifi mesh net appears, it would be a good start, but I'm skeptical it would become widespread. We do have cjdns, Project Meshnet and the Hyperboria network in Seattle. Still pretty fledgling, though. http://en.wikipedia.org/wiki/Cjdns http://en.wikipedia.org/wiki/Cjdns https://projectmeshnet.org/ https://projectmeshnet.org/ http://hyperboria.net/ http://hyperboria.net/
gwu78 13y ago
The easiest path to decentralization is to either: 1 Give every user a publicly reachable IP address at their home or business, or 2 Give every user an account with some remote organization that with a publicly reachable IP address, where the user can be listed in some sort of IP address directory. (Some users now use "dynamic DNS" for this purpose.) I say 1 is better than 2. The first big hurdle in the problem of decentralization is reachability; 30 years later, users of today's internet are not "directly" connected for two way communication. By getting users past this initial hurdle of end-to-end connectivity, it opens up the problem of centralization for all developers to solve. The user can pick and choose her software to make direct connections to other users, or she can choose none of it. The www is a given. Every web developer assumes users can access the www. Direct connections to other users should also be a given. And every developer should be able to assume that users can connect to each other. It's clear that users want direct connections to each other. They want to communicate. The success of store and forward solutions like email over several decades is proof. What's not clear is that they want (or need) a middleman to broker their connections.
- amirmc 13y ago
  I agree and I'm working on a tool that helps with this [1] but based on domain names. Of course that still depends on the registrars (e.g Verisign) but we believe it's easier for folks to remember names rather than numbers and then use the infrastructure. There's a paper we wrote that goes over the architecture and examples, in case you're interested [2]. [1] http://nymote.org/software/signpost/ http://nymote.org/software/signpost/ [2] http://nymote.org/docs/2013-foci-signposts.pdf http://nymote.org/docs/2013-foci-signposts.pdf
  - gwu78 13y ago
    ".. we believe it's easier for folks to remember names rather than numbers..." Yes, but regardless of whether they use names or numbers, they still cannot get "public reachability" (a public IP address and allowance of unsolicited ingress traffic) without either getting a publicly reachable IP address (ISP's in the US call this a "business class" account) i.e. option 1 or somehow involving at least one computer that already has public reachability i.e. option 2 or 3. At least, _I_ know of no other way. If you know how to create public reachability without giving the user a publicly reachable IP address or involving a computer that already has public reachability, please develop and release the code; I think you are a networking genius and have done the impossible! In any event, my point is that it's probably better that the computer that already has public reachability be owned and controlled _by the user_, not some third party "provider". The user should only need one provider to communicate over the internet: their ISP. I say just give users a publicly reachable IP address: "upgrade" their account with the ISP to allow unsolicited ingress connections. A myriad of communications and data synchronization solutions will follow. I reckon developers would love to write code for an end-to-end network. It would certainly be more functional that one where eevrything must be done via HTTP through a "web browser". OCaml NaCl and dnscurve. Nice work! I hope your project succeeds!
peterwwillis 13y ago
Decentralized systems suck. Look at any protocol or network service designed to be fully decentralized. Inefficient routing, unstable communication, high latency and low bandwidth. You have to do a lot more work to achieve things that would otherwise be simple and straightforward. Take human communications, for example. They're largely decentralized by default, but we use centralized services to improve efficiency. If some piece of information needs to be disseminated to a large population over a period of time, we first record the message, then we post it in a centralized location viewable by lots of clients[1]. Those clients can then go back and distribute the information to its connected peers[2], but technology also allows us to distribute subscriptions of this information to a wider population in a variety of media[3]. ([1]stone tablets, [2]town crier, [3]newspapers/television) Modern people are afraid of centralization in the Internet because they are suddenly realizing they have no real control over the content or medium. But this is the case with all other forms of centralized mass communication, too. People are also afraid of their lack of control over their data, which is tantamount in these days to a form of property. Anyone would freak out if the contents of their house were suddenly only available at the discretion of some conglomerate! But you don't need "decentralization" to keep a backup of your data. Most of it originates from you anyway, and you can store it before you publish it. Some service providers even provide methods to download all your data, though obviously you can't rely on everyone providing you that option (unless someone passes a law...) On top of all this, the internet is already a system of decentralized networks and services. You can just hook in your own services into the network at any point therein and maintain them yourself. The only restriction is really by the ISP, and there's lots of those to choose from in most developed countries. Personally I think we should consider the internet a combination of utility and public highway, since at this point we all need access, but we also need competition due to the sensitivity of limits on network access.
- rthomas6 13y ago
  What about things like Bitcoin and Bittorrent? Those are decentralized protocols that both work great. I think the real problem is that decentralized protocols have no way of determining what information each user should see or not see. I also think this is a solvable problem, but that it is hard.
the_watcher 13y ago
Unfortunately, I've become sadly apathetic to online privacy concerns. There are always going to be people willing to pay those with the skills to spy. In the past, PI's are HUMINT were really the only ways to get "private" information. The internet makes it easier, and allows it to scale massively (the same characteristics that caused the internet to generate the massive economic gains of the last 20 years). I've just shifted to assume that anything that can be spied on is (semi) public. It's suboptimal, and sad, but I've made the choice of trading privacy for convenience, since I'll never stop using Google or Facebook or Twitter or Amazon. Or storing my credit cards and using online banking. Until someone designs a better option (which I expect will fight a constant battle against surveillance).
- amirmc 13y ago
  I seem to come across this view more often and it saddens me. It doesn't have to be all or nothing. For example, if someone really wants to break into my house they probably will but that doesn't mean I'm not going to bother locking my doors and windows in the future. I'm in favour of making it harder for the seemingly casual surveillance to take place using means that already exist, like encryption and decentralised systems. If more devs etc took that view and built things that were properly secure by default then maybe things would be different. However, such systems aren't really compatible with advertising-based business models and there's a real trade-off in terms of speed to build (that's why - sometimes - 'security' feels like it's slapped on at the end).
  - the_watcher 13y ago
    I think I agree with you. The locking the house doors and windows is a good analogy. I'd love for it to be more difficult for the surveillance state. It just seems like the safer way to proceed is to assume my digital life is public (especially since I don't have anything in particular to hide). Just as I lock my doors and windows despite the utopian society involving unlocked doors all the time.
    - amirmc 13y ago
pauljohncleary 13y ago
I'm really surprised http://tent.io http://tent.io has not been mentioned here. Tent is a protocol that allows users to pick and choose a datastore provider for use with any tent compatible application. The provider can be centralised, on the user's own infrastructure or anywhere in between. Applications can be installed locally (without a dependency on communicating with another server, other than the user's chosen datastore) or hosted on a web server. Users get to choose where (and control how) an application stores their data, and developers don't need to think about infrastructure, user management or authorisation when designing and building their app. The protocol is at v0.3 and is not ready for the mainstream yet, but it's coming, and (for me) has the potential to replace a bunch of web apps I regularly use (think dropbox, gmail, github etc.).

77 points by ghosh 13y ago

Good article that hits on an important subject: decentralized privacy-enhancing systems have a tough battle against easy to use consumer friendly services from Google, Facebook, etc. I have some hope that local grid networks will catch on. Also cheap appliances that are easy to use and offer local cloud services may eventually catch on. Not to sound too political, but this is the fight between large corporations and/or financial elites firming up their control of everything. I am not even sure how much personal effort I will put into these peoples' causes, and I am a long time supporter of the FSF, EFF, and ACLU.

jokoon 13y ago

Centralized systems are almost asking to be spying on, laws or not. Even if the NSA does not have the right to do what it did, I'm sure they'd end up doing by using corruption, and there are many ways to do it. Worse, it would open the door to a lot of other shady practices, and I'm sure criminal would make a buck out of it. The fact that a huge quantity of information is put in one place increases the risk it will peeked into by a lot. Decentralized systems are a little more difficult to maintain and develop, but are so much cheaper and eliminate the risk of mass surveillance. There are still security risks depending on designs, but honestly I think decentralized has a lot of future, the advantages and features it brings counter balances the disadvantages by a lot. I wish bittorrent inc would produce more technologies and some competitor would really be cool.

salient 13y ago
I agree. As long as big companies become so good at tracking and making money through surveillance themselves, it's just too irresistible for spy agencies not to try and get that data one way or another. That's why companies need to at least use forward secrecy, keep data at rest encrypted, and also delete more ephemeral data a lot more quickly. Google used to delete the data they have on their users every 6 months. I don't think they plan on ever deleting anything they have anymore now. That's the wrong way to go about it. Tracking data should not exist for more than a few months at most, and private communications should not be held for more than a year. Of course, it would be so much easier for the user, if everything was end-to-end encrypted, and then it wouldn't need to be deleted.

znowi 13y ago

> On Monday, eight major tech firms, many of them competitors, united to demand an overhaul of government transparency and surveillance laws. Not long ago they united under the PRISM project to spy on their users. What a quick change of heart. And again, there's already a similar call for reform started by Mozilla months ago (in June), which none of the PRISM companies supported. They were busy releasing copycat reports denying everything back then. https://stopwatching.us/ https://stopwatching.us/

salient 13y ago
> Not long ago they united under the PRISM project to spy on their users. What a quick change of heart. Yes, but then they did it for your safety, and because they care about you. Now they're doing it because...well, they're starting to lose money.
- amirmc 13y ago
  A slightly less cynical view: They were in the dark about who else was involved and didn't fancy picking a fight with USG alone (and with an unknown amount of public support). Now that things became public and they know who else is involved, they can at least co-ordinate, knowing that at least one segment of the public cares.
  - charonn0 13y ago
    It's always easier to oppress the man who believes he stands alone.

psc 13y ago

While Google, Amazon, etc. being centralized might be bad in some ways (and good in others), I'm less concerned with that kind of centralization (because there are alternatives). The most dangerous centralization is at the ISP level, because it's the first gateway to the internet. When I saw the article title I was hoping it would be about that. Unfortunately, wired internet connections make for a really good natural monopoly, so until we have super fast wireless internet everywhere, I don't expect a solution. If any wifi mesh net appears, it would be a good start, but I'm skeptical it would become widespread. The other dangerous centralization of the internet is the DNS system. I know, technically it's decentralized because there even thousands of root name servers. But politically it's centralized under ICANN. Domains are controlled by registrars and governments. That's not necessarily a bad thing, but a truly distributed system would help guarantee a neutral internet. There are some good solutions for this already, namely Namecoin (http://dot-bit.org http://dot-bit.org), which uses proof of work and a bitcoin-like blockchain as a P2P DNS. Namecoin uses the .bit TLD, but probably the largest barrier to adoption is that it's not accessible by default on devices because it's not ICANN sanctioned and therefore not in the root name servers. There are workarounds for this. ICANN could add Namecoin to the root servers but then it would still be under the control of ICANN. The true solution would be for ISPs to add it to their DNS. If the ISP decentralization is solved, this becomes trivial. In my opinion, this is the only real way to guarantee net neutrality and prevent censorship.

lukifer 13y ago
I absolutely agree that ISPs are the nexus point for a truly decentralized internet. I've honestly been surprised that ISPs haven't leveraged/abused their oligopoly more than they have. > If any wifi mesh net appears, it would be a good start, but I'm skeptical it would become widespread. We do have cjdns, Project Meshnet and the Hyperboria network in Seattle. Still pretty fledgling, though. http://en.wikipedia.org/wiki/Cjdns http://en.wikipedia.org/wiki/Cjdns https://projectmeshnet.org/ https://projectmeshnet.org/ http://hyperboria.net/ http://hyperboria.net/

gwu78 13y ago

The easiest path to decentralization is to either: 1 Give every user a publicly reachable IP address at their home or business, or 2 Give every user an account with some remote organization that with a publicly reachable IP address, where the user can be listed in some sort of IP address directory. (Some users now use "dynamic DNS" for this purpose.) I say 1 is better than 2. The first big hurdle in the problem of decentralization is reachability; 30 years later, users of today's internet are not "directly" connected for two way communication. By getting users past this initial hurdle of end-to-end connectivity, it opens up the problem of centralization for all developers to solve. The user can pick and choose her software to make direct connections to other users, or she can choose none of it. The www is a given. Every web developer assumes users can access the www. Direct connections to other users should also be a given. And every developer should be able to assume that users can connect to each other. It's clear that users want direct connections to each other. They want to communicate. The success of store and forward solutions like email over several decades is proof. What's not clear is that they want (or need) a middleman to broker their connections.

amirmc 13y ago
I agree and I'm working on a tool that helps with this [1] but based on domain names. Of course that still depends on the registrars (e.g Verisign) but we believe it's easier for folks to remember names rather than numbers and then use the infrastructure. There's a paper we wrote that goes over the architecture and examples, in case you're interested [2]. [1] http://nymote.org/software/signpost/ http://nymote.org/software/signpost/ [2] http://nymote.org/docs/2013-foci-signposts.pdf http://nymote.org/docs/2013-foci-signposts.pdf
- gwu78 13y ago
  ".. we believe it's easier for folks to remember names rather than numbers..." Yes, but regardless of whether they use names or numbers, they still cannot get "public reachability" (a public IP address and allowance of unsolicited ingress traffic) without either getting a publicly reachable IP address (ISP's in the US call this a "business class" account) i.e. option 1 or somehow involving at least one computer that already has public reachability i.e. option 2 or 3. At least, _I_ know of no other way. If you know how to create public reachability without giving the user a publicly reachable IP address or involving a computer that already has public reachability, please develop and release the code; I think you are a networking genius and have done the impossible! In any event, my point is that it's probably better that the computer that already has public reachability be owned and controlled _by the user_, not some third party "provider". The user should only need one provider to communicate over the internet: their ISP. I say just give users a publicly reachable IP address: "upgrade" their account with the ISP to allow unsolicited ingress connections. A myriad of communications and data synchronization solutions will follow. I reckon developers would love to write code for an end-to-end network. It would certainly be more functional that one where eevrything must be done via HTTP through a "web browser". OCaml NaCl and dnscurve. Nice work! I hope your project succeeds!

peterwwillis 13y ago

Decentralized systems suck. Look at any protocol or network service designed to be fully decentralized. Inefficient routing, unstable communication, high latency and low bandwidth. You have to do a lot more work to achieve things that would otherwise be simple and straightforward. Take human communications, for example. They're largely decentralized by default, but we use centralized services to improve efficiency. If some piece of information needs to be disseminated to a large population over a period of time, we first record the message, then we post it in a centralized location viewable by lots of clients[1]. Those clients can then go back and distribute the information to its connected peers[2], but technology also allows us to distribute subscriptions of this information to a wider population in a variety of media[3]. ([1]stone tablets, [2]town crier, [3]newspapers/television) Modern people are afraid of centralization in the Internet because they are suddenly realizing they have no real control over the content or medium. But this is the case with all other forms of centralized mass communication, too. People are also afraid of their lack of control over their data, which is tantamount in these days to a form of property. Anyone would freak out if the contents of their house were suddenly only available at the discretion of some conglomerate! But you don't need "decentralization" to keep a backup of your data. Most of it originates from you anyway, and you can store it before you publish it. Some service providers even provide methods to download all your data, though obviously you can't rely on everyone providing you that option (unless someone passes a law...) On top of all this, the internet is already a system of decentralized networks and services. You can just hook in your own services into the network at any point therein and maintain them yourself. The only restriction is really by the ISP, and there's lots of those to choose from in most developed countries. Personally I think we should consider the internet a combination of utility and public highway, since at this point we all need access, but we also need competition due to the sensitivity of limits on network access.

rthomas6 13y ago
What about things like Bitcoin and Bittorrent? Those are decentralized protocols that both work great. I think the real problem is that decentralized protocols have no way of determining what information each user should see or not see. I also think this is a solvable problem, but that it is hard.

the_watcher 13y ago

Unfortunately, I've become sadly apathetic to online privacy concerns. There are always going to be people willing to pay those with the skills to spy. In the past, PI's are HUMINT were really the only ways to get "private" information. The internet makes it easier, and allows it to scale massively (the same characteristics that caused the internet to generate the massive economic gains of the last 20 years). I've just shifted to assume that anything that can be spied on is (semi) public. It's suboptimal, and sad, but I've made the choice of trading privacy for convenience, since I'll never stop using Google or Facebook or Twitter or Amazon. Or storing my credit cards and using online banking. Until someone designs a better option (which I expect will fight a constant battle against surveillance).

amirmc 13y ago
I seem to come across this view more often and it saddens me. It doesn't have to be all or nothing. For example, if someone really wants to break into my house they probably will but that doesn't mean I'm not going to bother locking my doors and windows in the future. I'm in favour of making it harder for the seemingly casual surveillance to take place using means that already exist, like encryption and decentralised systems. If more devs etc took that view and built things that were properly secure by default then maybe things would be different. However, such systems aren't really compatible with advertising-based business models and there's a real trade-off in terms of speed to build (that's why - sometimes - 'security' feels like it's slapped on at the end).
- the_watcher 13y ago
  I think I agree with you. The locking the house doors and windows is a good analogy. I'd love for it to be more difficult for the surveillance state. It just seems like the safer way to proceed is to assume my digital life is public (especially since I don't have anything in particular to hide). Just as I lock my doors and windows despite the utopian society involving unlocked doors all the time.
  - amirmc 13y ago

pauljohncleary 13y ago

I'm really surprised http://tent.io http://tent.io has not been mentioned here. Tent is a protocol that allows users to pick and choose a datastore provider for use with any tent compatible application. The provider can be centralised, on the user's own infrastructure or anywhere in between. Applications can be installed locally (without a dependency on communicating with another server, other than the user's chosen datastore) or hosted on a web server. Users get to choose where (and control how) an application stores their data, and developers don't need to think about infrastructure, user management or authorisation when designing and building their app. The protocol is at v0.3 and is not ready for the mainstream yet, but it's coming, and (for me) has the potential to replace a bunch of web apps I regularly use (think dropbox, gmail, github etc.).