11 ms·

I have a truly naive question about the distributed web: what makes the supporters of it think it will be any different from the original web. I mean, isn't it

by d--b 10y ago

I have a truly naive question about the distributed web: what makes the supporters of it think it will be any different from the original web. I mean, isn't it likely that at some point, there will be the need for a centralized search engine for it? Isn't it unavoidable that big companies like facebook runs their own non-distributed subnetwork, so that it can deliver standard functionality to all its users? The original web IS distributed already, isn't it? It's just that organically, the way people use it has become a lot more centralized, no? Or am I missing the main argument for a distributed architecture?

splintercell 10y ago
IPFS + Ethereum Smart Contracts = Distributed Web The idea is that we run that centralized search engine via Smart Contracts. In other words, using a decentralized currency such as Bitcoin and decentralized execution environment like Ethereum, we create softwares which pays people for providing CPU time and disk space (facilitated by IPFS). Few missing pieces of this puzzle are homomorphic encryption implemented on Smart Contracts level which will allow that CPU service. Once that's done then we can have the distributed web of the next level.
- Taek 10y ago
  A search engine via ethereum is not going to scale, is not easily upgraded, and will fall far short of any properly centralized search engine. If you want a competitve decentralized search engine, you're going to need a different technology.
  - traverseda 10y ago
    I think that's why they talk about homomorphic encryption.
    - Taek 10y ago
      Homomorphic encryption as best I'm aware is more than a decade away from practical use. I believe the best libraries today have blowups of 100,000,000x in resource usage, meaning it's not even practical for basic crypto operations. Multiparty computation makes some security sacrifices (breaks if M out of N people cheat, breaks undetectably iirc), but I think the blowup is less than 10x for some operations and less than 1000x for the others. Still not pretty, but at least makes certain things possible. But not competitive search engines.
- traverseda 10y ago
  It's a pity that ether seems to really want to make their own file layer, and IPFS seems to really want to make their own coin for dns.
- drivingmenuts 10y ago
  Does the issue of digital-currency volatility apply? If you're hosting content at a certain rate, then the rate drops so that it's less profitable or no longer profitable, wouldn't the proper strategy be to drop that content and then find something worth hosting?
httpagent 10y ago
I am a novice, but I'll do my best to answer. IPFS isn't a replacement for the existing web like many of its predecessors, for the purposes of your question, it really works more like a drop-in shared caching system. The site host doesn't have to participate in IPFS for this to work. As an example: You host a blog. As an IPFS user, I surf to the blog and store your content in my cache. When another IPFS user attempts to access your blog they may pull directly from my cached version, or from the original host (depending which is fastest). Merkle DAGs are used to hash content for quick locating, to ensure content is up-to-date, and to build a linked line of content over time. This gets more interesting if there's a widespread service outage. IPFS nodes will continue to serve the most up-to-date version of the web even if the web is fragmented. As new information becomes available it is integrated into the existing cache and then propagated to the rest of the fragments. I still struggle to understand how this works with databased content, but I do believe IPFS addresses this content.
- clueless404 10y ago
  > The site host doesn't have to participate in IPFS for this to work. Really? An IPFS user will scrape HTTP content, republish it over IPFS and update a directory of where non-IPFS content can be found over IPFS? I must have missed that part.
  - slang800 10y ago
    Yeah - there's a ton of immutable URLs on the web. All of CDNJS/JSDelivr/Google Hosted Libraries, most of raw.github.com, all Imgur & Instagram images, all YouTube video streams (excluding annotations & subtitles), and all torrent file caching sites (like itorrents.org). There are probably some large ones I'm forgetting about, but just mapping immutable URLs to IPFS could probably cover 1/3rd of Internet traffic. Check out https://github.com/ipfs/archives https://github.com/ipfs/archives to learn more.
    - clueless404 10y ago
      IPFS archives look like something entirely different from what the grandparent was talking about. Sure, you can manually publish archives over IPFS, but that's not something that automatically creates an IPFS cache copy of whatever you are surfing.
0xcde4c3db 10y ago
I can think of two big differences to consider: 1) The web as a whole is decentralized, but any given web site is centralized by default. A webmaster can "push" a modification or deletion from a central server and have it trickle down through caches/proxies unless someone like archive.org makes a deliberate decision to archive it. In systems like IPFS, content deletion and modification follows a model more like garbage collection, where pointers to old data can hang around and continue working even if an "authoritative" pointer is pointing to something else. Basically, nobody has the authority to break a link (law/policy-mandated filters notwithstanding). 2) The web doesn't really have any built-in mechanisms to promote redundancy for durability, availability, or performance purposes. By default, there's one daemon serving one copy. Everything else (CDNs, forward/reverse proxies, failover) is some kind of optimization tacked on after-the-fact.
daveguy 10y ago
The difference is IPFS (or similar) would distribute content. Currently communication is distributed, but content is not. IPFS is like https, but where every page is a bittorrent. There are challenges for security in distributing services and monetization. However, the key is that content would be distributed. YouTube probably would not switch to ipfs, but DNS (what was targeted recently) would make sense. A DNS host list should be more distributed. Even with DNS there is a dynamic content challenge. When a DNS entry changes, it would still need to propagate. This brings it back to a distributed communication problem. If you could distribute content more consistently and 100,000 computers could step in as dns providers with authentic data rather than concentrating dns at dyn or Google it would help. That will still require protocol updates so that the fail over uses local content. Really ipfs and distributed content could be a component to help with distribution. It definitely isn't a complete solution alone (yet -- there is a push to make it the solution).
- glibgil 10y ago
  The internet may run on content (storage) and communication, but both those things run on computation. Until computation can be safely distributed real changes can't happen
  - wyager 10y ago
    What do you mean? It seems to me that the single largest destructive force on the internet is forcible content takedowns. IPFS is immune to those.
    - clueless404 10y ago
      In which way is IPFS immune to content takedowns? IPFS provides no anonymity, so it won't take long before DMCA notices to start arriving.
      - tom_mellior 10y ago
        > In which way is IPFS immune to content takedowns? In the way that, if one node is forced to take down some information, that information may still be accessible from other nodes under the same address as before. It's not quite immunity: They can still harrass individuals. Call it "resilience", maybe. > IPFS provides no anonymity, so it won't take long before DMCA notices to start arriving. It should compose well with Tor, I'd hope.
fit2rule 10y ago
>what makes the supporters of it think it will be any different from the original web. Disclaimer: I have worked on building the Internet since before the web, and have a BOFH kind of point of view of things, 2 or 3 times over now .. I support IPFS - and things like it - as a technologist/hacker/punk-ass because, in fact, we have built an utter monstrosity of a beast of a spaghetti god of an Internet, and we can always do better. Software - and by extension, inter-networking - is like music. There will always be better ways to do it, beyond the horizon of what is current and now. Just because what we've got "basically works, even though its all broken", doesn't mean we can't 'fix whats broken'. To those of us in the group of your postulate, whats broken is the fact that its all so damned hierarchical, and requires canonical/authority issue and agency in multiple forms before the bits can flow. We can't just set up an IP address - we also have to have DNS, serve content on it on some port, etc. And, on top of this, we end up having to sort it out all the way down the OSI stack, at the hardware layer, by keeping as many hosts as possible up and running as are required to service the customers, here and now, who are using our service. All of that is what is broken about the current Internet, but of course the irony is that it is all that works about the Internet at the same time. So, IPFS/distributed/etc. means this: we can un-bork all of the above, and just have all the things route down to the real, hard, user of the system, i.e. the original content provider. IPFS, and its ilk, is all about putting the original content provider in control, while also having the network - as a function of its operating capability - contribute to helping those content providers who become, eventually, popular. So, we don't have a lot of rigid hierarchy - we have instead multi-variate spread spectra of responsibility/duty to serve up the bits of it all - packets at a time, as part of participation in the whole - to those who want something, and those who know how to find it, from those who have something, and know how to name it. In short, "its not about the network any more, its the people..."

by d--b 10y ago

splintercell 10y ago

IPFS + Ethereum Smart Contracts = Distributed Web The idea is that we run that centralized search engine via Smart Contracts. In other words, using a decentralized currency such as Bitcoin and decentralized execution environment like Ethereum, we create softwares which pays people for providing CPU time and disk space (facilitated by IPFS). Few missing pieces of this puzzle are homomorphic encryption implemented on Smart Contracts level which will allow that CPU service. Once that's done then we can have the distributed web of the next level.

Taek 10y ago
A search engine via ethereum is not going to scale, is not easily upgraded, and will fall far short of any properly centralized search engine. If you want a competitve decentralized search engine, you're going to need a different technology.
- traverseda 10y ago
  I think that's why they talk about homomorphic encryption.
  - Taek 10y ago
    Homomorphic encryption as best I'm aware is more than a decade away from practical use. I believe the best libraries today have blowups of 100,000,000x in resource usage, meaning it's not even practical for basic crypto operations. Multiparty computation makes some security sacrifices (breaks if M out of N people cheat, breaks undetectably iirc), but I think the blowup is less than 10x for some operations and less than 1000x for the others. Still not pretty, but at least makes certain things possible. But not competitive search engines.
traverseda 10y ago
It's a pity that ether seems to really want to make their own file layer, and IPFS seems to really want to make their own coin for dns.
drivingmenuts 10y ago
Does the issue of digital-currency volatility apply? If you're hosting content at a certain rate, then the rate drops so that it's less profitable or no longer profitable, wouldn't the proper strategy be to drop that content and then find something worth hosting?

httpagent 10y ago

I am a novice, but I'll do my best to answer. IPFS isn't a replacement for the existing web like many of its predecessors, for the purposes of your question, it really works more like a drop-in shared caching system. The site host doesn't have to participate in IPFS for this to work. As an example: You host a blog. As an IPFS user, I surf to the blog and store your content in my cache. When another IPFS user attempts to access your blog they may pull directly from my cached version, or from the original host (depending which is fastest). Merkle DAGs are used to hash content for quick locating, to ensure content is up-to-date, and to build a linked line of content over time. This gets more interesting if there's a widespread service outage. IPFS nodes will continue to serve the most up-to-date version of the web even if the web is fragmented. As new information becomes available it is integrated into the existing cache and then propagated to the rest of the fragments. I still struggle to understand how this works with databased content, but I do believe IPFS addresses this content.

clueless404 10y ago
> The site host doesn't have to participate in IPFS for this to work. Really? An IPFS user will scrape HTTP content, republish it over IPFS and update a directory of where non-IPFS content can be found over IPFS? I must have missed that part.
- slang800 10y ago
  Yeah - there's a ton of immutable URLs on the web. All of CDNJS/JSDelivr/Google Hosted Libraries, most of raw.github.com, all Imgur & Instagram images, all YouTube video streams (excluding annotations & subtitles), and all torrent file caching sites (like itorrents.org). There are probably some large ones I'm forgetting about, but just mapping immutable URLs to IPFS could probably cover 1/3rd of Internet traffic. Check out https://github.com/ipfs/archives https://github.com/ipfs/archives to learn more.
  - clueless404 10y ago
    IPFS archives look like something entirely different from what the grandparent was talking about. Sure, you can manually publish archives over IPFS, but that's not something that automatically creates an IPFS cache copy of whatever you are surfing.

0xcde4c3db 10y ago

I can think of two big differences to consider: 1) The web as a whole is decentralized, but any given web site is centralized by default. A webmaster can "push" a modification or deletion from a central server and have it trickle down through caches/proxies unless someone like archive.org makes a deliberate decision to archive it. In systems like IPFS, content deletion and modification follows a model more like garbage collection, where pointers to old data can hang around and continue working even if an "authoritative" pointer is pointing to something else. Basically, nobody has the authority to break a link (law/policy-mandated filters notwithstanding). 2) The web doesn't really have any built-in mechanisms to promote redundancy for durability, availability, or performance purposes. By default, there's one daemon serving one copy. Everything else (CDNs, forward/reverse proxies, failover) is some kind of optimization tacked on after-the-fact.

daveguy 10y ago

The difference is IPFS (or similar) would distribute content. Currently communication is distributed, but content is not. IPFS is like https, but where every page is a bittorrent. There are challenges for security in distributing services and monetization. However, the key is that content would be distributed. YouTube probably would not switch to ipfs, but DNS (what was targeted recently) would make sense. A DNS host list should be more distributed. Even with DNS there is a dynamic content challenge. When a DNS entry changes, it would still need to propagate. This brings it back to a distributed communication problem. If you could distribute content more consistently and 100,000 computers could step in as dns providers with authentic data rather than concentrating dns at dyn or Google it would help. That will still require protocol updates so that the fail over uses local content. Really ipfs and distributed content could be a component to help with distribution. It definitely isn't a complete solution alone (yet -- there is a push to make it the solution).

glibgil 10y ago
The internet may run on content (storage) and communication, but both those things run on computation. Until computation can be safely distributed real changes can't happen
- wyager 10y ago
  What do you mean? It seems to me that the single largest destructive force on the internet is forcible content takedowns. IPFS is immune to those.
  - clueless404 10y ago
    In which way is IPFS immune to content takedowns? IPFS provides no anonymity, so it won't take long before DMCA notices to start arriving.
    - tom_mellior 10y ago
      > In which way is IPFS immune to content takedowns? In the way that, if one node is forced to take down some information, that information may still be accessible from other nodes under the same address as before. It's not quite immunity: They can still harrass individuals. Call it "resilience", maybe. > IPFS provides no anonymity, so it won't take long before DMCA notices to start arriving. It should compose well with Tor, I'd hope.

fit2rule 10y ago

>what makes the supporters of it think it will be any different from the original web. Disclaimer: I have worked on building the Internet since before the web, and have a BOFH kind of point of view of things, 2 or 3 times over now .. I support IPFS - and things like it - as a technologist/hacker/punk-ass because, in fact, we have built an utter monstrosity of a beast of a spaghetti god of an Internet, and we can always do better. Software - and by extension, inter-networking - is like music. There will always be better ways to do it, beyond the horizon of what is current and now. Just because what we've got "basically works, even though its all broken", doesn't mean we can't 'fix whats broken'. To those of us in the group of your postulate, whats broken is the fact that its all so damned hierarchical, and requires canonical/authority issue and agency in multiple forms before the bits can flow. We can't just set up an IP address - we also have to have DNS, serve content on it on some port, etc. And, on top of this, we end up having to sort it out all the way down the OSI stack, at the hardware layer, by keeping as many hosts as possible up and running as are required to service the customers, here and now, who are using our service. All of that is what is broken about the current Internet, but of course the irony is that it is all that works about the Internet at the same time. So, IPFS/distributed/etc. means this: we can un-bork all of the above, and just have all the things route down to the real, hard, user of the system, i.e. the original content provider. IPFS, and its ilk, is all about putting the original content provider in control, while also having the network - as a function of its operating capability - contribute to helping those content providers who become, eventually, popular. So, we don't have a lot of rigid hierarchy - we have instead multi-variate spread spectra of responsibility/duty to serve up the bits of it all - packets at a time, as part of participation in the whole - to those who want something, and those who know how to find it, from those who have something, and know how to name it. In short, "its not about the network any more, its the people..."