4 ms·
It reminds me of the last wave of distributed P2P filesharing, c. 2000-2005. First we got Napster, which realized that by taking advantage of the bidirectional
by nostrademons 10y ago
It reminds me of the last wave of distributed P2P filesharing, c. 2000-2005. First we got Napster, which realized that by taking advantage of the bidirectional nature of TCP/IP, we could distribute copyright infringement so widely that it would become impossible to enforce. Then some techies at NullSoft realized that you don't actually need a central server for discovery at all, you can broadcast the existence of and metadata about files throughout the network for discovery, and put out Gnutella. Lots of excitement followed, with many people of that era believing P2P would be the Next Big Thing (among people working on this were such folks as Travis Kalanick of Red Swoosh -> Uber, Mark Zuckerburg of WireHog -> Facebook, Friis & Zenstrom of Kazaa -> Skype -> Rdio, and Robert Tappan Morris of Chord -> YCombinator).
And then folks realized that Gnutella was slow. Really slow, because folks on transient dialup connections in Estonia were critical nodes in file discovery. So they created the idea of super-peers, where the protocol was adapted to realize that some nodes are more equal than others and route requests through nodes with known-good connections. Then the next-generation P2P network (BitTorrent) decided that it was easier just to rely on HTTP for metadata exchange and went to centralized trackers, and the ecosystem that grew up around BitTorrent realized branding & centralized search engines were important for non-technical users and we got big torrent search engines like ThePirateBay and Mininova.
Eventually, the market seemed to settle on convenience & performance over distribution, and the real winners in the mass market were DropBox, Google, and Apple. There's probably a lesson in there, but as someone who's been fascinated by P2P technologies since high school and wasted a lot of time thinking about the problem in college, the lesson is pretty depressing. If hierarchy, market concentration, and single points of failure don't already exist, they will be reinvented.
- striking 10y agoThis is probably the most enlightening comment I've read all day. I've been looking for a suitable explanation for this trend, and here it is.
- Pxtl 10y agoYup. Same with Git. Nobody uses Git in a truly p2p fashion. Distributed systems are useful for a variety of reasons, but so is authority. Most successful systems seem to use a distributed p2p system because it adds useful functionality to a system that will still ultimately have a master server.
- will_hughes 10y ago> Then the next-generation P2P network (BitTorrent) decided that it was easier just to rely on HTTP for metadata exchange and went to centralized trackers, and the ecosystem that grew up around BitTorrent realized branding & centralized search engines were important for non-technical users and we got big torrent search engines like ThePirateBay and Mininova. Followed by those trackers getting taken down, we went from distributing .torrent files over HTTP, to distributing magnet links and getting the torrent metadata over DHT.
- nostrademons 10y agoYeah, Bittorrent has actually gotten more decentralized in recent years, both on the tracker front and on an increasing number of Torrent search engines being small operations. I just find it sad that apparently the only reason that things tend to get more distributed is to evade some powerful authority. I wanted distributed systems to become mainstream, because of their freedom, equality, & competition benefits, but it appears that when given the choice, the mainstream would rather play kingmaker and live under centralized overlords.
- badloginagain 10y ago> Yeah, Bittorrent has actually gotten more decentralized in recent years Only due to sustained pressure to keep it decentralized. If TPB was acquired by Google and rebranded to 'Google Bay', bittorrent would be just another Google fiefdom, and be as entrenched as Docs or YouTube. >I wanted distributed systems to become mainstream, because of their freedom, equality, & competition benefits Unfortunately you're in the vast minority of people who understand what distributed systems are, and how to connect them to anything as abstract as freedom, equality, etc. Most people just want an easy-to-remember, easier-to-use, not sketchy looking way to consume content. Profit-seeking companies are simply better equipped and more motivated to provide that service than the open-source community is, because they don't have to worry about freedom, equality, and certainly competition.
- battlebot 10y ago
- CPLX 10y agoInteresting that you wrote Spotify out of that narrative. They were a really interesting early take on P2P there for awhile too, keeping legality and centralization but leveraging the technology for increased quality of service. It's a large part of why they're the market leader now. It's not like the idea of a subscription music service was novel, their execution was unusually good as a result.
- eli_gottlieb 10y ago> If hierarchy, market concentration, and single points of failure don't already exist, they will be reinvented. From another point of view, if at least one point of responsibility does not exist, it will have to be reinvented. There have been plenty of times when I can't download something because, while I can find the torrent on a tracker, nobody is actually seeding it anymore. Sometimes nobody has been seeding some obscure file for years. Nobody in the distributed protocol is responsible for keeping up old torrents, so nobody actually does. Then it becomes easier to go as Netflix or iTunes for an unpopular old movie than to torrent it. I'm pretty sure this happened with Iron Sky when I wanted to watch it.