19 ms·

HTTP is obsolete. It's time for the Distributed Web (2015)

467 points by Karrot_Kream 9y ago

CryoLogic 9y ago
Obsolete? Maybe not yet. But I do think putting forth effort to improving distributed protocols like IPFS could be very helpful in preventing internet censorship like we have seen from Comcast, YouTube, etc. recently.
- projectant 9y ago
  Exactly. It's like, Woah, hold your horses there. "Obsolete"? I don't think so. To what extent can this IPFS serve an API or a dynamic database at this point? To what extent will it ever be able to do that? I think HTTP/websockets is very good for these things. Static data is one thing, dynamic is a whole other story. It seems IPFS is just a new distributed way to archive data. So what? It doesn't help serve something like FB over a distributed network does it? And to what extent could some sort of "protocol" vulnerability stop these networks from being "uncensorable". Are they truly resistant to censorship, or could they be effectively shut down somehow? Wouldn't DDOS attacks cripple these? I mean, that's a crucial flaw, right, you just have to look up all nodes for an piece of content and constantly flood them with DDOS traffic and then, hey, you've censored the network, right?
  - carolc 9y ago
    Databases, and dynamic content in general, can be done with/on IPFS. Take look at OrbitDB (https://github.com/orbitdb/orbit-db https://github.com/orbitdb/orbit-db) - "Distributed peer-to-peer database for the decentralized web" or their blog post "Decentralized Real-Time Collaborative Documents - Conflict-free editing in the browser using js-ipfs and CRDTs" (https://blog.ipfs.io/30-js-ipfs-crdts.md https://blog.ipfs.io/30-js-ipfs-crdts.md). And all that works in the browser without running a local IPFS in the background. That's pretty amazing imo.
    - lgierth 9y ago
      Noms is also a great example of a peer-to-peer database: https://noms.io https://noms.io
    - Luker88 9y ago
      In general? No. Just because you can, it does not mean you should use a distributed db. Please remember to say that distributed, open databases have very narrow use cases. Leaving aside use cases like credit card information, there are a lot of user information that is illegal to share unless the user explicitly consents. In the EU you can't even share your access logs by default. And how do you handle authentication? Passwords? how do you avoid user enumeration, the collection of user email and info? Distributed filesystems and CDN in general are great, but let's use them for things that do not actually need a single bit of security, please.
      - carolc 9y ago
        > "Distributed filesystems and CDN in general are great, but let's use them for things that do not actually need a single bit of security, please." The notion that distributed filesystems are inherently, or can't be, secure is way off. I would argue that with these technologies, such as IPFS, they can be more secure. The use cases are not only "open databases" (by which I assume you mean open to public), private databases and data sets can be achieved just as well. Just because it's "distributed" doesn't mean it can't be private or access controlled. Agreed on the comment re. "...illegal to share unless the user explicitly consents" and I believe this will turn out better in the trustless, distributed web, eventually. Our whole current approach is based on the client-server paradigm forcing us to put every user and their data into one massive centralized database. But we can change the model here. Instead, how about you owning your data(base) and controlling who gets to access it? "Allow Facebook to read your social graph?" "Oh, no? How about another social network app?". As a user, I would want to have that choice. That bridges to your next point on authentication, which can be done on the protocol level with authenticated data structures. You can define who can read/write to a database by using public key signing/verification. It could be just you, or it could be a set of keys. One good example of this is Secure Scuttlebut (http://scuttlebot.io/ http://scuttlebot.io/). I highly recommend to take a look an understanding the data structures underneath.
gnaritas 9y ago
HTTP is not obsolete, it might be outdated, but words have meaning and HTTP one of the most used protocols there is, calling it obsolete is silly.
- PrimHelios 9y ago
  "adj. Outmoded in design, style, or construction: an obsolete locomotive. "[0] So yes, it's obsolete. [0] https://www.wordnik.com/words/obsolete https://www.wordnik.com/words/obsolete
  - tyrw 9y ago
    You are of course choosing to ignore the first definition, which says "no longer in use"
    - dredmorbius 9y ago
      If someone clarifies a meaning by saying "this is what I mean" and pointing to a dictionary definition, a more appropriate response might be "Oh, I didn't realise that's the way you meant it". That's a sophisticated response. Though you might care to decide which meaning of "sophisticated" you think I'm referencing here. (There are, in fact, words with widely varying, and even opposed, definitions. Arguing that someone means one that they clearly don't, in such a case, would be exceedingly uncharitable.)
    - PrimHelios 9y ago
      No, I'm not. Words can have multiple definitions. One definition does not apply, but a different one does. That's how language works. Hell, most adjectives mean more than one thing, that's just English.
      - tyrw 9y ago
        > One definition does not apply, but a different one does. This is you choosing to ignore the first definition. And it's NBD. I do understand how language works. You can choose to focus on one definition and ignore another, as is clear by this thread.
styfle 9y ago
Does anyone know if Dat can be used over HTTP like IPFS can?
- deleted 9y ago
  [deleted]
- frabrunelle 9y ago
  Yes, Dat sites can be rehosted over HTTPS. See for example https://github.com/beakerbrowser/dathttpd https://github.com/beakerbrowser/dathttpd Also: https://hashbase.io/ https://hashbase.io/
Semiapies 9y ago
Ugh, servers! Except, you're gonna have to have servers unless you've got the entire web backed up on everyone's computer. Otherwise, you don't and can't know how many copies of a page or other file are out there. But who's going to pay for servers to retain random peoples' and companies' web detritus? This whole project exists because that's not feasible in the long term... It's not "crazy", it's pure, thoughtless hype.
- setra 9y ago
  I can purchase a droplet on digital ocean than has gigabit and a terabyte of transfer every month for $5. A terabyte is more than enough content for me. I am sure most people could afford that to manage their interneting.
  - simplyluke 9y ago
    >I can purchase a droplet on digital ocean than has gigabit and a terabyte of transfer every month for $5 >I am sure most people could afford that to manage their interneting And before the spec has even seen real adoption, we've already seen it centralize into a few major providers. A bit tongue-in-cheek, but it's a real issue. HTTP isn't the reason things are centralizing so much as economies of scale and convenience are. I see nothing about IPFS that fundamentally changes that, and think we'd likely see similar centralization over time. Git, one of the inspirational technologies, is in theory distributed as well and in practice hyper-centralized to only a few major providers.
    - rictic 9y ago
      Distributed technology keeps centralized providers honest. If github got complacent their customers could migrate their most important data in a very short time.
      - hueving 9y ago
        Github is complacent but people haven't moved because it's difficult. The issue tracker is proprietary and losing all of that and the account references for the comments makes moving non trivial.
alexasmyths 9y ago
Wonderful articulation. But I suggest that internet is 'centralized' not due to any technological function, but due to the nature of information, particularly the economics and business of information. The 'protocol' used to pass information from A-B, were it changed to something better, would not yield what the author is suggesting. Google 'would still have all our stuff' - I believe. When 'individuals' can 'run services' from a variety of physical locales, with robustness - we will see more decentralization, I think. I believe this will happen probably by accident, gradually, as 'more tech' creeps into our homes, one day, most people will have enough 'gear' to run some kind of service from home/small office - and other places. Finally - despite the obvious failings of HTTP ... incumbency is what it is. I'll be we're stuck with it for a very, very long time. If a powerful enough entity decided to change that - like Google AND Amazon together, or the US Government, or the Chinese Government ... that could change. Funny enough I think it's China that's best positioned to do it. They have the wherewithal and the tech momentum, they could do it by 'fiat' and in 10 years implement some kind of 'new, better network' that we'd all eventually move to. Hey - America is still using 'swipe cards' and doesn't even use 'smart cards' though they've been in use for 40 years around the world :)
- caymanbruce 9y ago
  No. The Chinese government will fancy a 'centralized' internet that the government has total control. It will try as hard as it could to make sure the internet is 'centralized' and dismantle anything that is distributed or P2P. Some ISP in China doesn't even allow you to use a public IP. So I think this IPFS thing is most likely a huge threat to the GFW(Great Firewall) system of China. I hope this thing can eventually tear down the entire effort of the Internet censorship in China.
  - alexasmyths 9y ago
    I didn't mean to say that they would do something 'decentralized' - of course they wouldn't. But possibly something 'better' than HTTP. One of the things about the net, is that it's transactionally open. There's no inherent identity/or security, it was grafted on with SSL - 'kind of'. I suggest it might have been better if identity were required to even make connections, to avoid many kinds of attacks. Hopefully, it could be done in an unbureaucratic manner, also wherein 'identities' can remain de-facto anonymous. My main point was that 'http' is kind of old, but we're stuck with it unless a 'major power' does something about it.
jlrbuellv 9y ago
It's kind of funny that the example link to a "permanent" object still returns a 404 ("ipfs resolve -r /ipns/QmTodvhq9CUS9hH8rirt4YmihxJKZ5tYez8PtDmpWrVMKP: Could not resolve name.") I totally want the web to magically be distributed too, but clearly not even the author is bothering to host their IPFS content anymore...
- __s 9y ago
  > IPNS isn’t done yet, so if that link doesn’t work, don’t fret. Just know that I will be able to change what that pubkeyhash points to, but the pubkeyhash will always remain the same. When it’s done, it will solve the site updating problem.
  - IncRnd 9y ago
    That's how these things always work, by saying "it will work one day." If it would work, and the cost benefit ratio were there, people would adopt it quickly. That's what happens with just about everything else.
    - anigbrowl 9y ago
      Mo it isn't. I was trying to get small-medium businesses interested in a thing called email in 1990 and it was a hard sell.
      - IncRnd 9y ago
        Of course it was a hard sell! The cost/benefit wasn't right for that business in 1990...
    - neutronbits 9y ago
      Wow! What an ignorant view of technology adoption! Almost, every revolutionary technology you see today (right from radio and A/C current to personal computers and deep learning) did not work fine once upon a time. It is because people kept saying, "it will work one day", and continued working on them that we have these technologies making our life simpler these days.
hyperpape 9y ago
Does HTTP have to lose for IPFS to win? Except the cost argument, none of the points in this article prevent sites from serving things via HTTP with IPFS metadata for the long term (though I don't know how that would play with advertising).
- beager 9y ago
  One question that came to mind while reading this was "Why do we have to demean HTTP in order for the tenets of IPFS to succeed?" HTTP is ubiquitous, and no purism or idealistic superiority of some other protocol is going to sway everybody to the New Hot Thing. This is not a knock on IPFS but rather a recognition of reality: you're going to have to work within the current system to supplant it. And that means not only tolerating the old thing while pushing for the advantages of the new thing, but accepting that absorption of the dynamics of the new thing within the old thing represents a victory for the new thing, even if it doesn't get the named recognition. Maybe we're ready to evolve beyond the limitations of HTTP (and HTTP/2, which I see as a viable and feasible, if not short-sighted improvement to HTTP). How are you going to get Google, Facebook, Amazon, and everyone else to go along with you? If you offer the benefits as compatible add-ons to the existing norms, you will succeed. If you demand that we fully jettison HTTP to achieve something better, methinks you will have an insurmountably hard time.
- lgierth 9y ago
  I agree that HTTP doesn't have to die -- for many use cases it has serious flaws though, the most important being its coupling to location-addressing. You always need a domain name and path to construct a URL. If the host decides to change the path, or loses control over the domain, you're out of luck. Fun fact: when URLs were being defined, some people expressed the opinion that location-addressing is a huge mistake to begin with. Almost all of the links and references in the respective mailing list archives are now: broken.
gafferongames 9y ago
wipes tear from eye
cbhl 9y ago
I can't tell whether the link "promising leads" was broken on purpose when the page was written in 2015 (to make a point), or if it itself is an example of centrally-served pages that slowly bit-rot over time.
free2rhyme214 9y ago
It's time for Blockstack.
yeukhon 9y ago
So a couple questions even reading this... 1. The availability depends on the number of peers like BitTorrent? If so, and if no seed is available, how does one access the content, esp in the context of an intranet? 2. Any change to how we run infrastructure except not serving HTTP?
- tscs37 9y ago
  The usual solutions proposed to fix number 1 is to pay someone to host your content. So basically as before but with the added chattyness and overhead of a p2p protocol.
  - cookiecaper 9y ago
    IMO Filecoin is the most potentially revolutionary thing to come out of the IPFS space, and it or something like it may have impacts that extend beyond keeping content distributed. We badly need a safe, self-organizing, apparently-persistent storage mechanism. What if a "universal basic income" meant "get paid for sharing your excess disk space"? This could even be made transparent similar to OS page caches, and then everyone with a computer + internet connection would be a participant.
- Karrot_Kream 9y ago
  > The availability depends on the number of peers like BitTorrent? If so, and if no seed is available, how does one access the content, esp in the context of an intranet? There's a concept called pinning which keeps files available in a local share. You can pay other people to keep it pinned for a long amount of time, and they usually charge for the size of the file.
- Too 9y ago
  It's not like you upload something to the swarm then disconnect your own server and hope people will forever seed. The swarm is a backup and load balancer. You (or some hosting provider you pay) should still be the main seeder of your own content.
- WhiteOwlLion 9y ago
Asdfbla 9y ago
Why are distributed filesystems like IPFS so popular again these days ? Freenet has been around (and super niche) for close to 20 years soon. Is it because the Bitcoin hype has reinvigorated crypto-anarchists?
- adrianN 9y ago
  I'd run a freenet node if it wouldn't expose me to the risk of getting all my gear confiscated by some LEA.
  - krick 9y ago
    Probably I don't know something: why should it?
    - adrianN 9y ago
      Because Freenet is used for distributing CP.
    - cookiecaper 9y ago
      Freenet's model places random content on every node. Sure, it may be encrypted, but if there's a sting and your IP distributes something bad during that sting, you will be investigated and possibly arrested/falsely accused. It's the same dilemma that Tor exit node operators face. You are essentially loaning your IP out to the masses. Since this is what law enforcement uses to associate internet activity with individuals, there's a high risk that someone will eventually end up "borrowing" your IP to send something illegal.
- htns 9y ago
  I've been wondering that too. There isn't much allure to it beyond "our hosting/CDN will be cheaper if we let the customers pay for it". If anything it weakens privacy a little, since any mean-spirited gremlin gets to snoop on who visits and hosts what.
  - feanaro 9y ago
jlebrech 9y ago
something like? https://zeronet.io/ https://zeronet.io/
macawfish 9y ago
I need to comment because people are missing the point... there's nothing in this text that says the web won't need servers. Imagine that you and some friends want to launch a small local business and need to host a website. Instead of paying to host it "up in the cloud", why not plug a few raspberry pi's into the walls at each of your houses? Between that and also seeding it from your laptops, the site should have decent coverage. Maybe you could also offer some discounts for loyal customers who choose to seed it. This site will be redundantly hosted in the location where it's most likely to be accessed. If you're doing something more permanent, then you will want to make sure you have more stable hosts, and not just some peoples cell phones. Just like you do now. It's the same thing as now. If you want to host something, make sure there is at least one computer that is serving it on a decent internet connection. The difference is that with distributed web technologies, there is a smooth continuum for scaling. You don't even need to assume there is an ISP to seed! All you need is LANs. But if you want to do big things, you can harness the power of thousands of peers all streaming something that they're into. I use syncthing and resilio sync all the time, and it works great with just 3 devices.
- DaiPlusPlus 9y ago
  Distributed systems are suited to static content or "append-only" mutable data - canonical examples include Magnet links, distributed hashtables, git, and the Bitcoin blockchain - they're all reliant on content-addressable storage. Not all web-applications can support this model, for example a banking app or online shopping cart site, which depend, respectively, on secrecy instead of complete transparency and mutable ephemeral state. What is the "Raspberry Pis in the walls" solution to those problems?
  - carolc 9y ago
    See the comment below https://news.ycombinator.com/item?id=15376665 https://news.ycombinator.com/item?id=15376665. It is not true that distributed systems are only good for static content or "append-only" data. "Mutable systems" can be built on top of immutable systems.
    - 9y ago
Animats 9y ago
So why aren't these guys as big as Wix, which is in the same space with crappy technology?
Too 9y ago
While this might solve distributing content serving, i'm still stuck with transporting all my data through my single point of failure + spying ISP, when all of my neighbors live within wifi-range. For a truly distributed network we should look into ways to make internet work more like a mesh. Back in university, all the student dorms were connected to the same internal campus network, back then internet was slow but you could still share files blazingly fast with all the other students on the network (using DC++ at that time). While this wasn't exactly a pure mesh either it shows that local data works and beats internet many times, even with just a few thousand clients. Then DC++ was mostly focused on pirated content but with a more human friendly solution, like IPNS, it's not unimaginable that average Joe neighbor with one click can create a local mirror of the whole Wikipedia for you.
- lgierth 9y ago
  IPFS can discover other nodes in the same local network via mDNS -- you don't need to have an internet connection at all to share data locally. Combine this with the recent work on PubSub [1] and CRDTs [2] and you can make many applications work locally, that are otherwise annoyingly strongly coupled to internet services (thing Etherpad, Google Docs, Skype, Github, etc.) > For a truly distributed network we should look into ways to make internet work more like a mesh. Yes! We'll be putting more work into the network stack (libp2p) in the coming months. IPFS itself has done a ton to rework how content is defined and moved around, and libp2p will do the same for the network connections underneath. Think overlay networks, cryptokey routing, packet switching. [1] https://ipfs.io/blog/29-js-ipfs-pubsub/ https://ipfs.io/blog/29-js-ipfs-pubsub/ [2] https://ipfs.io/blog/30-js-ipfs-crdts.md https://ipfs.io/blog/30-js-ipfs-crdts.md
y_u_no_rust 9y ago
Personally I don't feel like the real centralization problem is on application level
z3t4 9y ago
You can add redundancy to HTTP by having more servers, just add more IP addresses to the domains A record. And files will be cached on the client (forever, if you let it).
- feanaro 9y ago
  Yes, you can, but do you think that's a comparably easy thing to do? Especially when you take into account that IPFS also facilitates coordination between unrelated parties in making the caching happen.
throw036281 9y ago
How would you deal with child pornography hosted on IPFS? What about IP infringement?
- quintushoratius 9y ago
  More importantly, how do you deal with a seeder that is (unwittingly) seeding such items?
  - stri8ed 9y ago
    Opt-in community based blacklist.
- Fnoord 9y ago
  https://steemit.com/ipfs/@dantheman/how-can-i-configure-ipfs-to-block-unwanted-content https://steemit.com/ipfs/@dantheman/how-can-i-configure-ipfs...
- Spivak 9y ago
  The same way you deal with them now. You go after the content discovery platform for which there is usually a small number of authoritative sources.
shade23 9y ago
How does ipfs deal with dynamic content? And how would you make sure that everyone uses an updated version of the website.
- brownbat 9y ago
  I was worried about that too. Scheduled republication is my best answer so far. If you promised to sign and republish the same file every day with a new timestamp, then people would know when they had the latest, and when they didn't... would just have to wonder if you fell off the earth, which we sort of do already with all those abandoned free software projects online. Republication may be cost prohibitive for large files, so instead you could republish a metadata file that pointed to the latest hash as of the metadata file's publication time. For the "hit by a bus" problem (or for a server doing this automatically, the "hit by a comet" problem?), it'd be nice to include a dead man's switch from a third party, where they can publish a "FINAL -- EXPECT NO MORE UPDATES"... But that that point you're trusting a third party. If you're willing to trust a third party this is far easier. So that might be what we'd end up with... something like DNS providers, but they're suddenly managing indexes and metadata for hosted files? I don't know... (Also, this has probably been worked out already by smarter people than me, I haven't looked at IPFS much, this was just a back of the napkin guess.)
- lgierth 9y ago
  See PubSub: https://ipfs.io/blog/29-js-ipfs-pubsub/ https://ipfs.io/blog/29-js-ipfs-pubsub/ You can use it as the base for CRDT structures: https://ipfs.io/blog/30-js-ipfs-crdts.md https://ipfs.io/blog/30-js-ipfs-crdts.md
prithvizi 9y ago
How would I use ipfs in a serverless model like AWS Lambda?
LukeB42 9y ago
Synchrony is an intended solution to centralisation written back in 2015 after being designed since around 2011: http://github.com/psybernetics/Synchrony http://github.com/psybernetics/Synchrony though this implementation also ships a peer-to-peer hyperdocument editor baked into the core of the UI so the Python PoC is a bit of a mutant. Currently aiming to release a Go implementation at some point after canning a C implementation earlier this year, where p2p is instead accessed via CONNECT proxy. Is there any interest in this project?
carlhjerpe 9y ago
What about dynamic content?
acd 9y ago
I would like distributed web methods like IPFS to be a w3c World Wide Web consortium standard. Combine IPFS and crypto currencies for payments and we have a new distribution standard. Then one could have distributed movie sites like Netflix and YouTube where you pay for royalties to legally file share the content.
teddyh 9y ago
Indeed it has been said that decentralization is the worst form of networking except all those other forms that have been tried from time to time. (With apologies to Winston Churchill.)
dgroves 9y ago
Despite the attractiveness of a distributed web - which we already have it (you do know how it works don't you?) And it is incredibly difficult to upgrade and patch; leaving it incredibly insecure. How many of those distributed web servers are still vulnerable to Heartbleed as just one example? We need better ideas...
TheRealPomax 9y ago
So, we're not commenting on "That hash is guaranteed by cryptography to always only represent the contents of that file"? Because we should be. The fundamental rule of hashing is that you are guaranteed to have collisions. The challenge is to find a good hashing function for the kind of file you're hashing, but on the internet you get EVERY TYPE OF FILE, so at the scale we're talking about for IPFS, we're dealing with a large enough number of files of every conceivable type that collisions are guaranteed.
- nearbuy 9y ago
  You're thinking of non-cryptographic hashes. A cryptographic hash that could generate collisions in any practical situation would be considered broken. You'd need more files than there are atoms in the galaxy to have even a 1 in a billion chance of collision with a secure 512 bit hash.
fingerguns 9y ago
IPFS is using Filecoin, which in itself is a questionable ICO. https://tokeneconomy.co/the-analysis-filecoin-doesnt-want-you-to-read-e60d5243f17c https://tokeneconomy.co/the-analysis-filecoin-doesnt-want-yo...
- lgierth 9y ago
  You got it the wrong way around - Filecoin will be built on top of IPFS, but IPFS itself works fine without Filecoin.
j_s 9y ago
An off-grid social network | https://news.ycombinator.com/item?id=14050049 https://news.ycombinator.com/item?id=14050049 (Apr 2017)

467 points by Karrot_Kream 9y ago

Obsolete? Maybe not yet. But I do think putting forth effort to improving distributed protocols like IPFS could be very helpful in preventing internet censorship like we have seen from Comcast, YouTube, etc. recently.

projectant 9y ago
Exactly. It's like, Woah, hold your horses there. "Obsolete"? I don't think so. To what extent can this IPFS serve an API or a dynamic database at this point? To what extent will it ever be able to do that? I think HTTP/websockets is very good for these things. Static data is one thing, dynamic is a whole other story. It seems IPFS is just a new distributed way to archive data. So what? It doesn't help serve something like FB over a distributed network does it? And to what extent could some sort of "protocol" vulnerability stop these networks from being "uncensorable". Are they truly resistant to censorship, or could they be effectively shut down somehow? Wouldn't DDOS attacks cripple these? I mean, that's a crucial flaw, right, you just have to look up all nodes for an piece of content and constantly flood them with DDOS traffic and then, hey, you've censored the network, right?
- carolc 9y ago
  Databases, and dynamic content in general, can be done with/on IPFS. Take look at OrbitDB (https://github.com/orbitdb/orbit-db https://github.com/orbitdb/orbit-db) - "Distributed peer-to-peer database for the decentralized web" or their blog post "Decentralized Real-Time Collaborative Documents - Conflict-free editing in the browser using js-ipfs and CRDTs" (https://blog.ipfs.io/30-js-ipfs-crdts.md https://blog.ipfs.io/30-js-ipfs-crdts.md). And all that works in the browser without running a local IPFS in the background. That's pretty amazing imo.
  - lgierth 9y ago
    Noms is also a great example of a peer-to-peer database: https://noms.io https://noms.io
  - Luker88 9y ago
    In general? No. Just because you can, it does not mean you should use a distributed db. Please remember to say that distributed, open databases have very narrow use cases. Leaving aside use cases like credit card information, there are a lot of user information that is illegal to share unless the user explicitly consents. In the EU you can't even share your access logs by default. And how do you handle authentication? Passwords? how do you avoid user enumeration, the collection of user email and info? Distributed filesystems and CDN in general are great, but let's use them for things that do not actually need a single bit of security, please.
    - carolc 9y ago
      > "Distributed filesystems and CDN in general are great, but let's use them for things that do not actually need a single bit of security, please." The notion that distributed filesystems are inherently, or can't be, secure is way off. I would argue that with these technologies, such as IPFS, they can be more secure. The use cases are not only "open databases" (by which I assume you mean open to public), private databases and data sets can be achieved just as well. Just because it's "distributed" doesn't mean it can't be private or access controlled. Agreed on the comment re. "...illegal to share unless the user explicitly consents" and I believe this will turn out better in the trustless, distributed web, eventually. Our whole current approach is based on the client-server paradigm forcing us to put every user and their data into one massive centralized database. But we can change the model here. Instead, how about you owning your data(base) and controlling who gets to access it? "Allow Facebook to read your social graph?" "Oh, no? How about another social network app?". As a user, I would want to have that choice. That bridges to your next point on authentication, which can be done on the protocol level with authenticated data structures. You can define who can read/write to a database by using public key signing/verification. It could be just you, or it could be a set of keys. One good example of this is Secure Scuttlebut (http://scuttlebot.io/ http://scuttlebot.io/). I highly recommend to take a look an understanding the data structures underneath.

gnaritas 9y ago

HTTP is not obsolete, it might be outdated, but words have meaning and HTTP one of the most used protocols there is, calling it obsolete is silly.

PrimHelios 9y ago
"adj. Outmoded in design, style, or construction: an obsolete locomotive. "[0] So yes, it's obsolete. [0] https://www.wordnik.com/words/obsolete https://www.wordnik.com/words/obsolete
- tyrw 9y ago
  You are of course choosing to ignore the first definition, which says "no longer in use"
  - dredmorbius 9y ago
    If someone clarifies a meaning by saying "this is what I mean" and pointing to a dictionary definition, a more appropriate response might be "Oh, I didn't realise that's the way you meant it". That's a sophisticated response. Though you might care to decide which meaning of "sophisticated" you think I'm referencing here. (There are, in fact, words with widely varying, and even opposed, definitions. Arguing that someone means one that they clearly don't, in such a case, would be exceedingly uncharitable.)
  - PrimHelios 9y ago
    No, I'm not. Words can have multiple definitions. One definition does not apply, but a different one does. That's how language works. Hell, most adjectives mean more than one thing, that's just English.
    - tyrw 9y ago
      > One definition does not apply, but a different one does. This is you choosing to ignore the first definition. And it's NBD. I do understand how language works. You can choose to focus on one definition and ignore another, as is clear by this thread.

styfle 9y ago

Does anyone know if Dat can be used over HTTP like IPFS can?

deleted 9y ago
[deleted]
frabrunelle 9y ago
Yes, Dat sites can be rehosted over HTTPS. See for example https://github.com/beakerbrowser/dathttpd https://github.com/beakerbrowser/dathttpd Also: https://hashbase.io/ https://hashbase.io/

Semiapies 9y ago

Ugh, servers! Except, you're gonna have to have servers unless you've got the entire web backed up on everyone's computer. Otherwise, you don't and can't know how many copies of a page or other file are out there. But who's going to pay for servers to retain random peoples' and companies' web detritus? This whole project exists because that's not feasible in the long term... It's not "crazy", it's pure, thoughtless hype.

setra 9y ago
I can purchase a droplet on digital ocean than has gigabit and a terabyte of transfer every month for $5. A terabyte is more than enough content for me. I am sure most people could afford that to manage their interneting.
- simplyluke 9y ago
  >I can purchase a droplet on digital ocean than has gigabit and a terabyte of transfer every month for $5 >I am sure most people could afford that to manage their interneting And before the spec has even seen real adoption, we've already seen it centralize into a few major providers. A bit tongue-in-cheek, but it's a real issue. HTTP isn't the reason things are centralizing so much as economies of scale and convenience are. I see nothing about IPFS that fundamentally changes that, and think we'd likely see similar centralization over time. Git, one of the inspirational technologies, is in theory distributed as well and in practice hyper-centralized to only a few major providers.
  - rictic 9y ago
    Distributed technology keeps centralized providers honest. If github got complacent their customers could migrate their most important data in a very short time.
    - hueving 9y ago
      Github is complacent but people haven't moved because it's difficult. The issue tracker is proprietary and losing all of that and the account references for the comments makes moving non trivial.

alexasmyths 9y ago

Wonderful articulation. But I suggest that internet is 'centralized' not due to any technological function, but due to the nature of information, particularly the economics and business of information. The 'protocol' used to pass information from A-B, were it changed to something better, would not yield what the author is suggesting. Google 'would still have all our stuff' - I believe. When 'individuals' can 'run services' from a variety of physical locales, with robustness - we will see more decentralization, I think. I believe this will happen probably by accident, gradually, as 'more tech' creeps into our homes, one day, most people will have enough 'gear' to run some kind of service from home/small office - and other places. Finally - despite the obvious failings of HTTP ... incumbency is what it is. I'll be we're stuck with it for a very, very long time. If a powerful enough entity decided to change that - like Google AND Amazon together, or the US Government, or the Chinese Government ... that could change. Funny enough I think it's China that's best positioned to do it. They have the wherewithal and the tech momentum, they could do it by 'fiat' and in 10 years implement some kind of 'new, better network' that we'd all eventually move to. Hey - America is still using 'swipe cards' and doesn't even use 'smart cards' though they've been in use for 40 years around the world :)

caymanbruce 9y ago
No. The Chinese government will fancy a 'centralized' internet that the government has total control. It will try as hard as it could to make sure the internet is 'centralized' and dismantle anything that is distributed or P2P. Some ISP in China doesn't even allow you to use a public IP. So I think this IPFS thing is most likely a huge threat to the GFW(Great Firewall) system of China. I hope this thing can eventually tear down the entire effort of the Internet censorship in China.
- alexasmyths 9y ago
  I didn't mean to say that they would do something 'decentralized' - of course they wouldn't. But possibly something 'better' than HTTP. One of the things about the net, is that it's transactionally open. There's no inherent identity/or security, it was grafted on with SSL - 'kind of'. I suggest it might have been better if identity were required to even make connections, to avoid many kinds of attacks. Hopefully, it could be done in an unbureaucratic manner, also wherein 'identities' can remain de-facto anonymous. My main point was that 'http' is kind of old, but we're stuck with it unless a 'major power' does something about it.

jlrbuellv 9y ago

It's kind of funny that the example link to a "permanent" object still returns a 404 ("ipfs resolve -r /ipns/QmTodvhq9CUS9hH8rirt4YmihxJKZ5tYez8PtDmpWrVMKP: Could not resolve name.") I totally want the web to magically be distributed too, but clearly not even the author is bothering to host their IPFS content anymore...

__s 9y ago
> IPNS isn’t done yet, so if that link doesn’t work, don’t fret. Just know that I will be able to change what that pubkeyhash points to, but the pubkeyhash will always remain the same. When it’s done, it will solve the site updating problem.
- IncRnd 9y ago
  That's how these things always work, by saying "it will work one day." If it would work, and the cost benefit ratio were there, people would adopt it quickly. That's what happens with just about everything else.
  - anigbrowl 9y ago
    Mo it isn't. I was trying to get small-medium businesses interested in a thing called email in 1990 and it was a hard sell.
    - IncRnd 9y ago
      Of course it was a hard sell! The cost/benefit wasn't right for that business in 1990...
  - neutronbits 9y ago
    Wow! What an ignorant view of technology adoption! Almost, every revolutionary technology you see today (right from radio and A/C current to personal computers and deep learning) did not work fine once upon a time. It is because people kept saying, "it will work one day", and continued working on them that we have these technologies making our life simpler these days.

hyperpape 9y ago

Does HTTP have to lose for IPFS to win? Except the cost argument, none of the points in this article prevent sites from serving things via HTTP with IPFS metadata for the long term (though I don't know how that would play with advertising).

beager 9y ago
One question that came to mind while reading this was "Why do we have to demean HTTP in order for the tenets of IPFS to succeed?" HTTP is ubiquitous, and no purism or idealistic superiority of some other protocol is going to sway everybody to the New Hot Thing. This is not a knock on IPFS but rather a recognition of reality: you're going to have to work within the current system to supplant it. And that means not only tolerating the old thing while pushing for the advantages of the new thing, but accepting that absorption of the dynamics of the new thing within the old thing represents a victory for the new thing, even if it doesn't get the named recognition. Maybe we're ready to evolve beyond the limitations of HTTP (and HTTP/2, which I see as a viable and feasible, if not short-sighted improvement to HTTP). How are you going to get Google, Facebook, Amazon, and everyone else to go along with you? If you offer the benefits as compatible add-ons to the existing norms, you will succeed. If you demand that we fully jettison HTTP to achieve something better, methinks you will have an insurmountably hard time.
lgierth 9y ago
I agree that HTTP doesn't have to die -- for many use cases it has serious flaws though, the most important being its coupling to location-addressing. You always need a domain name and path to construct a URL. If the host decides to change the path, or loses control over the domain, you're out of luck. Fun fact: when URLs were being defined, some people expressed the opinion that location-addressing is a huge mistake to begin with. Almost all of the links and references in the respective mailing list archives are now: broken.

gafferongames 9y ago

wipes tear from eye

cbhl 9y ago

I can't tell whether the link "promising leads" was broken on purpose when the page was written in 2015 (to make a point), or if it itself is an example of centrally-served pages that slowly bit-rot over time.

free2rhyme214 9y ago

It's time for Blockstack.

yeukhon 9y ago

So a couple questions even reading this... 1. The availability depends on the number of peers like BitTorrent? If so, and if no seed is available, how does one access the content, esp in the context of an intranet? 2. Any change to how we run infrastructure except not serving HTTP?

tscs37 9y ago
The usual solutions proposed to fix number 1 is to pay someone to host your content. So basically as before but with the added chattyness and overhead of a p2p protocol.
- cookiecaper 9y ago
  IMO Filecoin is the most potentially revolutionary thing to come out of the IPFS space, and it or something like it may have impacts that extend beyond keeping content distributed. We badly need a safe, self-organizing, apparently-persistent storage mechanism. What if a "universal basic income" meant "get paid for sharing your excess disk space"? This could even be made transparent similar to OS page caches, and then everyone with a computer + internet connection would be a participant.
Karrot_Kream 9y ago
> The availability depends on the number of peers like BitTorrent? If so, and if no seed is available, how does one access the content, esp in the context of an intranet? There's a concept called pinning which keeps files available in a local share. You can pay other people to keep it pinned for a long amount of time, and they usually charge for the size of the file.
Too 9y ago
It's not like you upload something to the swarm then disconnect your own server and hope people will forever seed. The swarm is a backup and load balancer. You (or some hosting provider you pay) should still be the main seeder of your own content.
WhiteOwlLion 9y ago

Asdfbla 9y ago

Why are distributed filesystems like IPFS so popular again these days ? Freenet has been around (and super niche) for close to 20 years soon. Is it because the Bitcoin hype has reinvigorated crypto-anarchists?

adrianN 9y ago
I'd run a freenet node if it wouldn't expose me to the risk of getting all my gear confiscated by some LEA.
- krick 9y ago
  Probably I don't know something: why should it?
  - adrianN 9y ago
    Because Freenet is used for distributing CP.
  - cookiecaper 9y ago
    Freenet's model places random content on every node. Sure, it may be encrypted, but if there's a sting and your IP distributes something bad during that sting, you will be investigated and possibly arrested/falsely accused. It's the same dilemma that Tor exit node operators face. You are essentially loaning your IP out to the masses. Since this is what law enforcement uses to associate internet activity with individuals, there's a high risk that someone will eventually end up "borrowing" your IP to send something illegal.
htns 9y ago
I've been wondering that too. There isn't much allure to it beyond "our hosting/CDN will be cheaper if we let the customers pay for it". If anything it weakens privacy a little, since any mean-spirited gremlin gets to snoop on who visits and hosts what.
- feanaro 9y ago

jlebrech 9y ago

something like? https://zeronet.io/ https://zeronet.io/

macawfish 9y ago

I need to comment because people are missing the point... there's nothing in this text that says the web won't need servers. Imagine that you and some friends want to launch a small local business and need to host a website. Instead of paying to host it "up in the cloud", why not plug a few raspberry pi's into the walls at each of your houses? Between that and also seeding it from your laptops, the site should have decent coverage. Maybe you could also offer some discounts for loyal customers who choose to seed it. This site will be redundantly hosted in the location where it's most likely to be accessed. If you're doing something more permanent, then you will want to make sure you have more stable hosts, and not just some peoples cell phones. Just like you do now. It's the same thing as now. If you want to host something, make sure there is at least one computer that is serving it on a decent internet connection. The difference is that with distributed web technologies, there is a smooth continuum for scaling. You don't even need to assume there is an ISP to seed! All you need is LANs. But if you want to do big things, you can harness the power of thousands of peers all streaming something that they're into. I use syncthing and resilio sync all the time, and it works great with just 3 devices.

DaiPlusPlus 9y ago
Distributed systems are suited to static content or "append-only" mutable data - canonical examples include Magnet links, distributed hashtables, git, and the Bitcoin blockchain - they're all reliant on content-addressable storage. Not all web-applications can support this model, for example a banking app or online shopping cart site, which depend, respectively, on secrecy instead of complete transparency and mutable ephemeral state. What is the "Raspberry Pis in the walls" solution to those problems?
- carolc 9y ago
  See the comment below https://news.ycombinator.com/item?id=15376665 https://news.ycombinator.com/item?id=15376665. It is not true that distributed systems are only good for static content or "append-only" data. "Mutable systems" can be built on top of immutable systems.
  - 9y ago

Animats 9y ago

So why aren't these guys as big as Wix, which is in the same space with crappy technology?

Too 9y ago

While this might solve distributing content serving, i'm still stuck with transporting all my data through my single point of failure + spying ISP, when all of my neighbors live within wifi-range. For a truly distributed network we should look into ways to make internet work more like a mesh. Back in university, all the student dorms were connected to the same internal campus network, back then internet was slow but you could still share files blazingly fast with all the other students on the network (using DC++ at that time). While this wasn't exactly a pure mesh either it shows that local data works and beats internet many times, even with just a few thousand clients. Then DC++ was mostly focused on pirated content but with a more human friendly solution, like IPNS, it's not unimaginable that average Joe neighbor with one click can create a local mirror of the whole Wikipedia for you.

lgierth 9y ago
IPFS can discover other nodes in the same local network via mDNS -- you don't need to have an internet connection at all to share data locally. Combine this with the recent work on PubSub [1] and CRDTs [2] and you can make many applications work locally, that are otherwise annoyingly strongly coupled to internet services (thing Etherpad, Google Docs, Skype, Github, etc.) > For a truly distributed network we should look into ways to make internet work more like a mesh. Yes! We'll be putting more work into the network stack (libp2p) in the coming months. IPFS itself has done a ton to rework how content is defined and moved around, and libp2p will do the same for the network connections underneath. Think overlay networks, cryptokey routing, packet switching. [1] https://ipfs.io/blog/29-js-ipfs-pubsub/ https://ipfs.io/blog/29-js-ipfs-pubsub/ [2] https://ipfs.io/blog/30-js-ipfs-crdts.md https://ipfs.io/blog/30-js-ipfs-crdts.md

y_u_no_rust 9y ago

Personally I don't feel like the real centralization problem is on application level

z3t4 9y ago

You can add redundancy to HTTP by having more servers, just add more IP addresses to the domains A record. And files will be cached on the client (forever, if you let it).

feanaro 9y ago
Yes, you can, but do you think that's a comparably easy thing to do? Especially when you take into account that IPFS also facilitates coordination between unrelated parties in making the caching happen.

throw036281 9y ago

How would you deal with child pornography hosted on IPFS? What about IP infringement?

quintushoratius 9y ago
More importantly, how do you deal with a seeder that is (unwittingly) seeding such items?
- stri8ed 9y ago
  Opt-in community based blacklist.
Fnoord 9y ago
https://steemit.com/ipfs/@dantheman/how-can-i-configure-ipfs-to-block-unwanted-content https://steemit.com/ipfs/@dantheman/how-can-i-configure-ipfs...
Spivak 9y ago
The same way you deal with them now. You go after the content discovery platform for which there is usually a small number of authoritative sources.

shade23 9y ago

How does ipfs deal with dynamic content? And how would you make sure that everyone uses an updated version of the website.

brownbat 9y ago
I was worried about that too. Scheduled republication is my best answer so far. If you promised to sign and republish the same file every day with a new timestamp, then people would know when they had the latest, and when they didn't... would just have to wonder if you fell off the earth, which we sort of do already with all those abandoned free software projects online. Republication may be cost prohibitive for large files, so instead you could republish a metadata file that pointed to the latest hash as of the metadata file's publication time. For the "hit by a bus" problem (or for a server doing this automatically, the "hit by a comet" problem?), it'd be nice to include a dead man's switch from a third party, where they can publish a "FINAL -- EXPECT NO MORE UPDATES"... But that that point you're trusting a third party. If you're willing to trust a third party this is far easier. So that might be what we'd end up with... something like DNS providers, but they're suddenly managing indexes and metadata for hosted files? I don't know... (Also, this has probably been worked out already by smarter people than me, I haven't looked at IPFS much, this was just a back of the napkin guess.)
lgierth 9y ago
See PubSub: https://ipfs.io/blog/29-js-ipfs-pubsub/ https://ipfs.io/blog/29-js-ipfs-pubsub/ You can use it as the base for CRDT structures: https://ipfs.io/blog/30-js-ipfs-crdts.md https://ipfs.io/blog/30-js-ipfs-crdts.md

prithvizi 9y ago

How would I use ipfs in a serverless model like AWS Lambda?

LukeB42 9y ago

Synchrony is an intended solution to centralisation written back in 2015 after being designed since around 2011: http://github.com/psybernetics/Synchrony http://github.com/psybernetics/Synchrony though this implementation also ships a peer-to-peer hyperdocument editor baked into the core of the UI so the Python PoC is a bit of a mutant. Currently aiming to release a Go implementation at some point after canning a C implementation earlier this year, where p2p is instead accessed via CONNECT proxy. Is there any interest in this project?

carlhjerpe 9y ago

What about dynamic content?

acd 9y ago

I would like distributed web methods like IPFS to be a w3c World Wide Web consortium standard. Combine IPFS and crypto currencies for payments and we have a new distribution standard. Then one could have distributed movie sites like Netflix and YouTube where you pay for royalties to legally file share the content.

teddyh 9y ago

Indeed it has been said that decentralization is the worst form of networking except all those other forms that have been tried from time to time. (With apologies to Winston Churchill.)

dgroves 9y ago

Despite the attractiveness of a distributed web - which we already have it (you do know how it works don't you?) And it is incredibly difficult to upgrade and patch; leaving it incredibly insecure. How many of those distributed web servers are still vulnerable to Heartbleed as just one example? We need better ideas...

TheRealPomax 9y ago

So, we're not commenting on "That hash is guaranteed by cryptography to always only represent the contents of that file"? Because we should be. The fundamental rule of hashing is that you are guaranteed to have collisions. The challenge is to find a good hashing function for the kind of file you're hashing, but on the internet you get EVERY TYPE OF FILE, so at the scale we're talking about for IPFS, we're dealing with a large enough number of files of every conceivable type that collisions are guaranteed.

nearbuy 9y ago
You're thinking of non-cryptographic hashes. A cryptographic hash that could generate collisions in any practical situation would be considered broken. You'd need more files than there are atoms in the galaxy to have even a 1 in a billion chance of collision with a secure 512 bit hash.

fingerguns 9y ago

IPFS is using Filecoin, which in itself is a questionable ICO. https://tokeneconomy.co/the-analysis-filecoin-doesnt-want-you-to-read-e60d5243f17c https://tokeneconomy.co/the-analysis-filecoin-doesnt-want-yo...

lgierth 9y ago
You got it the wrong way around - Filecoin will be built on top of IPFS, but IPFS itself works fine without Filecoin.

j_s 9y ago

An off-grid social network | https://news.ycombinator.com/item?id=14050049 https://news.ycombinator.com/item?id=14050049 (Apr 2017)