I've been hoping we return to dencentralization for awhile http://timepedia.blogspot.com/2008/05/decentralizing-web.html http://timepedia.blogspot.com/2008/05/d

I've been hoping we return to dencentralization for awhile http://timepedia.blogspot.com/2008/05/decentralizing-web.html http://timepedia.blogspot.com/2008/05/decentralizing-web.htm... but it seems the economics, bandwidth, and technology currently doesn't favor it. Consider what it would take to do decentralized search with a reasonable latency. Everyone would have to have a copy of Google's index, or at atleast a reasonable chunk of it, or proximity to some mesh node with a reasonable chunk of it. But mobile devices and mobile bandwidth are not really efficient as nodes to have complete copies, or partial copies. They're low on memory, and it's not really power efficient or network efficient for them to do the work. But what's worse is the problem is updating the index. If everyone ran a distributed crawler, there'd have to be some way to coordinate who crawls what and make sure there is not too much duplication of effort (else we waste network bandwidth and have a tragedy of the commons) You also run into problems of people trying to (and economically incentivized to) poison the index, and so you'd need an overhead of duplicate just to avoid that. It's not impossible to arrange a distributed crawl/map/reduce over millions of clients, the Folding@Home and other crowdsourced computation platforms show it can be done. But they don't have to manage replication and updating of petabyte scale databases, the problems they coordinate are easily sliced up and the results to be transmitted back aren't large. I think this tends to drive the network back towards super-nodes, as the leaf nodes aren't efficient, and the cost associated with doing the work can be pushed onto others (free rider problem). Eventually, the people running the super-nodes tend to become relied upon by everyone, and those become subject to the same kinds of problems we see today -- government attacks, hackers, privacy invasion, etc. It's not like SMTP/USENET weren't once relatively decentralized networks. But as millions and then billions of people came on line, the cost to run SMTP or USENET nodes was too high in complexity for most people to run their own. Small and medium dialup ISPs arose to handle that need. But since these were commodities, and little differentiation other than price, the prices were quickly driven to the marginal cost, and there isn't much money in running an email server for a few hundred people, you need scale. That eventually drove consolidation, and that's why today, you either get your email from Gmail, Yahoo, AOL, iCloud, Hotmail, or one of the other larger players. It may very well be that for some types of problems, there is inherent efficiency in centralization. We should look at perhaps how to change the way the services work to favor cost effectiveness of decentralization, so that rather than it being favored for ideological or aesthetic reasons, it is favored by an economic gradient.