6 ms·
"The Decentralized Web" is the WWW system Tim Berners-Lee invented and that we're all using right now. It relies on the DNS and HTTP protocols. This new IPFS-ba
by staunch 9y ago
"The Decentralized Web" is the WWW system Tim Berners-Lee invented and that we're all using right now. It relies on the DNS and HTTP protocols. This new IPFS-based decentralized web is interesting but it's not even a measurable percentage of web traffic today. Far from being "The".
What we need more than anything is to actually realize the beautiful dream TBL had for the WWW. HN is one great example but there should be a million more.
- loup-vaillant 9y agoNo. The web as it works right now is naturally centralised, for a simple reason: the bandwidth costs on a server is proportional to the size of its audience. This is why we need YouTube to begin with, instead of millions of people posting their videos on their personal page. There are other factors of course: firewalls, asymmetric bandwidth, security issues, technical ignorance… But the client-server model does play a significant role. If you want a truly decentralised web, you need to modify a couple things: get rid of HTTP, and use a P2P protocol instead. Generalise IPv6 and get rid of NAT. Give people symmetric bandwidth. Make secure software that is simple to implement, simple to use, and hard to misuse (it's not that hard dammit, just look at qmail).
- fiatjaf 9y ago"use a P2P protocol instead". Actually, just like "the cloud" is just someone else's computer, "P2P" is just someone else's server.
- JetSpiegel 9y agoP2P is not like the cloud. It's someone else's computer, but you don't know nor care what the exact machine you are connecting to because they are mostly untrusted.
- fiatjaf 9y agoI just made an analogy, didn't say it was the same thing as the cloud. My point was that in a P2P architecture you must be a server, everyone else must be a server. And being a server is costly. There must be incentives for someone to be a server, and that's not easy. I actually don't remember a single P2P protocol or framework that has solved that problem yet. Perhaps Bitcoin -- for miners only, not for normal nodes.
- loup-vaillant 9y agoIt appears BitTorrent solved the problem well enough. There's already an incentive to share, because nodes who don't are eventually ignored by the others (if I recall correctly). As for the operating costs… they don't seem very high unless you're operating a tracker. I mean, it's just a matter of running a BitTorrent client, and not even all the time (unless you want to reliably seed something). The real problem there is the asymmetry of the bandwidth. If we had as much upload as we had download, we wouldn't have any reason not to shoot for a 1:1 ratio.
- loup-vaillant 9y agoP2P is everyone else's server. Each peer pulls its own weight. This is different from relying on a single entity.
- fiatjaf 9y agoRelying on others is fine, the problem is when someone relies on you. Please see my answer to the other comment.
- loup-vaillant 9y ago> the problem is when someone relies on you I'm not sure what you're getting at. The problem is only magnified with the web: there's only one seed. If a site is down, it's down, and you can only hope Google cache or the web archive got a copy. That problem is never worse with P2P protocols.
- WJW 9y agoThat is only valid when all nodes have an equal chance of being down. Big websites have entire teams dedicated to keeping their site online, so even though it's centralized, youtube is actually a very stable website. A torrent with one seeding peer (or an IPFS object where only one node has it pinned) can potentially disappear if a single machine has a disk failure.
- loup-vaillant 9y agoThe basic assumption is independence, so I'm not even considering centralised hosting by Amazon or Alphabet. I am also assuming that if someone cares at all about seeding something, they will make sure it's not down too often.
- jkarneges 9y ago> This is why we need YouTube to begin with, instead of millions of people posting their videos on their personal page. People can post videos to their own hosted pages already. We "need" YouTube because 1) people generally don't want to host anything, and 2) it provides connectedness (search, related videos, subscribers, etc). P2P makes the first issue worse. The second issue is a problem of data federation, not client/server systems.
- pessimizer 9y ago> We "need" YouTube because 1) people generally don't want to host anything No, because it is expensive to host things, as mentioned in the comment you are replying to.
- loup-vaillant 9y ago> People can post videos to their own hosted pages already. No they cannot. Here's an example: what if make a nice tutorial, put it on my web site and submit it here and r/programming? If the thing is well done, people will download the video, and the sheer amount of requests may be enough to render my site unresponsive, simply because I don't have the bandwidth. So If I ever make a video, there's a good chance I host it to YouTube. Despite my reluctance to feed Alphabet. Even though I already operate a server. > 1) people generally don't want to host anything People want to publish stuff —that much is obvious. They want to avoid hassles if possible. And they rarely think about what centralised hosting entails. That doesn't amount to "don't want to host anything". Granted, hosting stuff yourself is a major hassle these days. It doesn't have to be, though.
- WJW 9y agoYou can easily get more bandwidth though, it's not that it is somehow forbidden. You just don't want to pay for it. That is what youtube gives you: 'unlimited' video hosting bandwidth (and improved discoverability) in return for the opportunity to play their ads before your video. The bandwidth costs also don't magically disappear with IPFS, it is just a convenient way to make other people pay for it. In order of increasing independence you could choose to: - Host it on youtube, with the benefits and disadvantages you already mentioned. - Host it on a public S3 bucket and link to it from a HTML5 video player on your web page. You pay for only the storage and bandwidth used. - Keep the video on your own server, but pay a CDN to cache it for you. Actual bandwidth use of you server is now very low again. - Host it on a (big) server in a colocation center. You would typically buy bandwidth in advance though, so you want to be able to roughly predict how much you need. - Contract with a fiber laying company to lay a dedicated fiber line from the nearest backbone to your home. (Alternatively, a large microwave connection) Buy several 100Gbit NICS to go with it and negotiate peering agreements with the major networks. Most people don't want the hassle and cost of the latter options, but they are very possible and very legal. In your example with the programming video, if a ton of people want to see it then somewhere along the chain a lot of bandwidth is going to be used and it is _not_ free. All you do is shift who pays for it.
- jasode 9y ago>Make secure software that is simple to implement, simple to use, and hard to misuse (it's not that hard dammit, just look at qmail). I made a previous comment about this being a hard problem[1]. If competent ultra techno geeks can get hacked, the problem looks to be unsolvable for the general homeowner to safely run a p2p server. Many recurring security+usability problems that nobody has solved in the general case. Examples... Heartbleed SSL bug that's undetected for 2 years before being fixed, social engineering, homeowner misconfiguration, p2p server data not being backed up, software updates as an attack vector for decreasing security instead of hardening security, etc. If my grandmother asked me, "A friend said I can share cooking recipes if I install a p2p food wiki server, what do you think?" I would immediately say "No! Don't install it. I will find you a _centralized_ web recipes forum for you to log in to!" I ran qmail on my home server in the 1990s to personally control my SMTP needs which should theoretically make me the biggest cheerleader for a "p2p food wiki server". Instead, it informs my position to stop my family members from installing p2p software. Think about why I would do that. The "symmetric bandwidth" isn't the only underlying problem. >That problem is never worse with P2P protocols. Everything has tradeoffs -- including p2p. Otherwise, you have an incomplete picture that doesn't consider pros and cons. P2p is worse for latency, worse for analyzing a unified network landscape to identify and filter out hostile actors, worse for non-techie usability, worse for instantaneous propagation of software updates to fix zero-day exploits, worse for homeowner costs, etc. There isn't a "set & forget" p2p appliance you can build that solves all of that. BitTorrent gets around the negatives of p2p because the value of the data users transfer (pirated Star Wars movies or Adobe software etc) overrides the hassles. [1] https://news.ycombinator.com/item?id=11861683 https://news.ycombinator.com/item?id=11861683
- loup-vaillant 9y ago> I made a previous comment about this being a hard problem[1]. That comment doesn't address the same problem. I'm fully aware that current software makes it virtually impossible to conveniently and securely host stuff ourselves right now. I am also fully aware that a general purpose computer can never be idiot proof. What I was saying is, we should write special purpose software dedicated to a few chosen forms of hosting (mail, web, some P2P stuff for big files…). The software can be simple, secure, and easy to use. It just doesn't exist for the most part. Also, one problem at a time. Wikis are great, but we're talking about allowing untrusted write access to the system, which is possibly the hardest problem of all. Let's start by single author publishing and build from there.
- ad_hominem 9y agoThe traditional WWW has shown its weaknesses with websites that exist at the boundaries of legality like Sci-Hub and the Pirate Bay, or that probe the edges of free speech like Stormfront or forums that support Islamic terrorism. Fact is unless you're a Tier 1 ISP yourself, you're going to be beholden to an ISP to host your content. Your domain name will be beholden to a registrar. Your TLS certificate will be beholden to a certificate authority. You're going to be subject to every one of these organizations' terms of service, and history has shown with the examples above that they will exercise that authority to pull the plug on you if it suits them (especially if they're facing a mob of public opinion or the state puts a gun to their head).
- saurik 9y ago> It relies on the DNS and HTTP protocols. DNS is fundamentally centralized.
- 0xcde4c3db 9y agoHow so? I see how "the" DNS is centralized in ICANN's list of root servers, but from a technical standpoint isn't the use of that list just a convention?
- saurik 9y agoBy this definition of "centralized", nothing is centralized. For DNS to have any meaning--particularly in the way it is used by HTML--requires there to be a person whose job is to run a server that maps "name a human enjoys using" to "name that a computer can connect to" in a way that is stable over time. To the extent to which DNS "the protocol" (as opposed to DNS "the concept") is incapable of enforcing that everyone uses the same servers (as no protocol can force that; again: with this definition, nothing is centralized), we have been building out cryptographic mitigations into other protocols (such as TLS and X.509) that act as a form of checksum against DNS failing to be used to access the one shared reality. Yes: a user can opt to use a totally different "shadow Internet" by using different DNS root servers and different CA root servers, but that is true of any protocol.
- 0xcde4c3db 9y agoI'm not talking about just anchoring a hierarchy at an alternative root. What I mean is that as far as I know, nothing (DNSSEC notwithstanding) really prevents breaking away from the strict hierarchical model altogether and doing something like a system with web-of-trust or filtered by heuristics. From there it's possible to think about how to build a genuine consensus beyond "ICANN says these are the root servers; who am I to argue?". For TLS (DANE notwithstanding), I very well might misunderstand the situation but I thought it only mattered that the client, server, and CA agree on the server's name, not that they agree on a particular delegation of authority for assigning that name.
- 9y ago
- bogomipz 9y ago>""The Decentralized Web" is the WWW system Tim Berners-Lee invented and that we're all using right now." I believe the context of "The Decentralized Web" here is meant to contrast the increasing centralization of content from FB, Google, Medium, login walls pay-walls etc.
- hedora 9y agoThose sites are all part of the decentralized web. Port 80 on my laser printer, router and flower pot are too. Using old well-defined terms to describe new concepts just leads to confusion. It would be like writing a story called “the history of the industrial revolution” that’s really about 3d printers.
- bogomipz 9y agoSo out of the one sentence that I wrote you decided to ignore the part where I wrote that decentralization was referring to centralization of content? Another word for centralization is consolidation. You might want to look up the term "walled garden": https://www.technologyreview.com/s/512316/facebook-and-google-create-walled-gardens-for-web-newcomers-overseas/ https://www.technologyreview.com/s/512316/facebook-and-googl...
- hedora 9y agoI didn’t see the word “content” on first reading. I never have encountered Google or FB created content (other than open source code), and I also don’t use their news aggregators (or other services), so (with the exception of YouTube), I don’t think of them as being in the content game at all. Anyway, I’ve been happily ignoring their walled gardens for decades, and using the decentralized web for news, etc. Certainly, they’re in the infrastructure + surveillance business, and (especially for Google) that’s concerning. Also, “Decentralized” is a technical term. Google runs a large, decentralized infrastructure, based on web technologies. “Consolidation” is usually used as a business term, and is orthogonal to centralizing operations. For instance, many companies have outsourced payroll to one centralized company. That does not mean their industries are consolidating. When people intentionally use words wrong, it is bound to lead to confusion.
- hedora 9y agoAgreed. As cool as IPFS is, I only clicked on it because I thought it was going to explain how to scale my Tim Berners-Lee WWW server in a decentralized way (without relying on some big monolithic company). Has anyone seen a primer like that? As for IPFS, I’d love to see them succeed, but as long as you have to know about merkle trees, hashing, etc for it, it’s not going to be any more mainstream than things like PGP or IRC. Finally, they claim IPFS is good for archival. To me, that means it will be as good as putting a few hundred paper books in a few hundred different libraries around the world. With that in mind, how does censorship work with IPFS? If the government wants to rewrite history by deleting my archive, how do the mirrors avoid being tracked down, and told to selectively delete the undesirable blobs?
- pessimizer 9y ago> As for IPFS, I’d love to see them succeed, but as long as you have to know about merkle trees, hashing, etc for it You have to know about this stuff to use ipfs like you have to know about low level packet routing to use the internet.
- NoGravitas 9y agoI think we ought to be distinguishing between "decentralized" and "distributed". The original web as we used to know it was decentralized. IPFS is distributed. In social media: [Mastodon][0] is decentralized. [Patchwork][1] is distributed. [0]: https://joinmastodon.org/ https://joinmastodon.org/ [1]: https://www.scuttlebutt.nz/ https://www.scuttlebutt.nz/