10 ms·

Why the Internet Needs IPFS

341 points by confiscate 11y ago

vezzy-fnord 11y ago
It seems like a pastime of TC is to write sensational filler articles out of the more questionable bile emanating from HN. See also "Death to C": https://news.ycombinator.com/item?id=9477878 https://news.ycombinator.com/item?id=9477878
- _prometheus 11y ago
  (IPFS dev here) "questionable bile"? we'd love your technical feedback. We know we have a ton of things to do better -- but what sort of things are you thinking about? what can we improve on?
  - larzang 11y ago
    The core of the modern web isn't static: it's transactional, it's personalized, and it's end-to-end encrypted for both security and privacy. How does a distributed caching model accommodate such things? If it can't, how can it be a replacement for http? If it's not a replacement for http, is it at least better than current CDNs? If so, how? It's hard to imagine you haven't already thought of these questions and answered them somewhere, but this non-article certainly doesn't address it or point to better resources (besides "buy my book!").
    - _prometheus 11y ago
      IPFS is not _just_ a static system. See the comments above: https://news.ycombinator.com/item?id=10329221 https://news.ycombinator.com/item?id=10329221 -- note that git ant bitcoin are "just static systems" as much as ipfs :). It turns out all content is "static", it's just "static content" that is "dynamically generated" at times-- the point being that you can still do all the dynamic stuff just fine, and use IPFS as a transport for the data.
smegger001 11y ago
IPFS is essentially web (here meaning htmls css & javascript) over torrent + a distributed hash table. Correct me if I am wrong but isn't this just freenet without the anonymity or kind of like tribler but for websites instead of pirated files?
- mburns 11y ago
  IPFS shares a lot of similarities to the Freenet datastore, yes. Both support static and dynamic keys that map to content-addressed URIs and can be accessed via a DHT. IPFS lets you selectively (opt-in) save data locally and host it, unlike Freenet which can cache and serve that data to other clients without user action. The lack of Freenet's anonymity feature means it can perform much faster and even be implemented in client-side javascript (Work In Progress) which will drastically increase user adoption.
  - _prometheus 11y ago
    (IPFS Dev) Yeah, we have many similarities with Freenet, but we also have significant differences. You mention the lack of anonymity, and yep, that's definitely one -- to speak more about this: this parts from our content model. Among explicit constraints, we have: - content must be able to move as fast as the underlying network permits. this rules out designs like freenet's and other oblivious storage platforms, as the base case. Like you said, they're just way too slow for most of IPFS use cases. But, these can be implemented trivially with the use of privacy focused transports (like Tor and I2P -- there's actually work towards this and people are getting close), content encryption, and so on. - IPFS nodes should be able to only store and/or distribute content they _explicitly_ want to store and/or distribute. This means that computers that run IPFS nodes do not have to host "other people's stuff", which is a very important thing when you consider that lots of content in the internet is -- in some for or other -- illegal under certain jurisdictions. Legitimate companies have way too much in their plate to additionally worry about potentially be storing a ton of illegal stuff. For serious companies like Google to use IPFS, we need to have a mode of operation that allows implementations to ONLY move the content THEY want to move. - Websites/Webapps must be able to operate entirely disconnected -- this means that it should be possible to build applications which create data locally, signed by the user, which can be distributed encrypted end-to-end to other users, without needing to ever touch specific backbone servers. This means users can move the data end-to-end via the closest route possible and in disconnected networks (think users on mobile phones on a plane using messaging webapp or web game and moving the bits over bluetooth or an ad-hoc wifi network. And this also means users in the broader internet do not _need to_ rely on backbone servers -- the model we're going for is that dedicated computers in the backbone CAN of course and SHOULD help, but you shouldn't HAVE TO RELY on them. Can write more about this :)
AstroJetson 11y ago
"As I explain in my upcoming book...." now the article makes sense, TC is helping her pimp her book. What the internet needs is a new financial model since the one we are have now isn't working in the long term.
- tim333 11y ago
  Yeah - stuff like "Before It’s Too Late" seems like hype to sell books. As far as I can tell the internet gets faster and cheaper each year and is not about to have it's "own internal contradictions unravel it from within".
- dfc 11y ago
  > Now this article makes sense This is all TC does, you can call it pimping or fluffing. Before this article did you think TC was an investigative journalism powerhouse? http://www.paulgraham.com/submarine.html http://www.paulgraham.com/submarine.html
sklivvz1971 11y ago
Meh, it's a distributed static website thing. Of course if one doesn't need transactions, everything is easier even today -- I mean, who doesn't use a CDN? It's a very similar concept. The problem is much tougher when the content is personalized or when there needs to be a transaction. This makes distribution a huge huge problem no one has solved yet (even a blockchain, IIRC can't really scale up in transactions...)
- _prometheus 11y ago
  (IPFS dev here) Hey! Yeah, we're focusing right now on "distributed static websites" because those are really easy to get right and make work all over the network. However, what's _really_ interesting about IPFS is making completely distributed applications which can do their process unhinged from a single origin -- websites/webapps without an origin server. Posted more about this elsewhere on this thread, take a look: https://news.ycombinator.com/item?id=10329221 https://news.ycombinator.com/item?id=10329221 Also see the Neocities + IPFS blog post: https://ipfs.io/ipfs/QmTVcD87Ecjps6wv9jMaGhvMuzZ2BgP6NyXDcnMU74RELx/ https://ipfs.io/ipfs/QmTVcD87Ecjps6wv9jMaGhvMuzZ2BgP6NyXDcnM... Take a look also at this talk, which presents different use cases for this: https://www.youtube.com/watch?v=skMTdSEaCtA https://www.youtube.com/watch?v=skMTdSEaCtA Can also see some of the random crazy things we're planning in https://github.com/ipfs/notes/issues/ https://github.com/ipfs/notes/issues/ + https://github.com/ipfs/archives/issues/ https://github.com/ipfs/archives/issues/ + https://github.com/ipfs/apps/issues/ https://github.com/ipfs/apps/issues/ (Edit: fix links)
  - supdog 11y ago
    Link to .../ipfs/notes/ appears to contain 37 bytes of nothing. Is the content available elseware? Would love to see the random crazy things you're planning.
    - lgierth 11y ago
      Yep sorry, it's in the issue tracker: https://github.com/ipfs/notes/issues https://github.com/ipfs/notes/issues
karissa 11y ago
This kinda makes me sick to my stomach -- essentially, she's saying that a peer-to-peer internet is great because companies will be able to have better uptime. She doesn't even cover the clearly obvious economic aspects of this -- why would I run an IPFS node if it just benefits the company and not me?
- _prometheus 11y ago
  (IPFS dev here) I think she does, but perhaps the discrepancy may be that she's got a different target audience: the established website owners? Also to be fair, there's not much space in these sorts of articles LOTS gets cut by editors and -- from having done something like this -- you have to hit this absuuuurdly high level and can't sink into details much at all. I'm actually pretty surprised at the level of technical detail in this article -- i would've expected much more to not make it past the "remove jargon - write for the average user" media filter. I've done some interviews / articles that ended up annoyingly waaaaay more high level and completely missed my expected mark. You might want to write to the author, too. :)
  - esfandia 11y ago
    The OP asks an interesting question though: why would I run an IPFS node if it just benefits the company and not me?
    - reitanqild 11y ago
      Maybe you won't. IIRC local machines by default only caches stuff for a very short time which makes sense for personal computers. On company networks however I'd expect sysadmins to run nodes to reduce upstream bandwidth usage. Edit: see also https://news.ycombinator.com/item?id=10329262 https://news.ycombinator.com/item?id=10329262 the specific example is the exact opposite but I see no technical reason why you cannot do this the other way around.
    - david_ar 11y ago
      There will be incentives in the future, both in terms of promoting a good seeding/leeching ratio like bittorrent ("bitswap"), as well as http://filecoin.io/ http://filecoin.io/
tptacek 11y ago
I'm a believer in this idea generally --- that we should replace applications built directly on IP/TCP with applications built on a content-addressed overlay network running on top of IP/TCP --- but I think the logic used here is faulty. For instance: I'm not clear on how IPFS protects applications from DDOS. Systems like IPFS spread the load of delivering content, but applications themselves are intrinsically centralized.
- on_ 11y ago
  I totally agree. leaving aside the granular details in the article this is the future. Bitcoin protocol, ethereum, ipfs, openbazaar, bittorrent etc are the blocks that content will be rebuilt on. The SAFEnetwork is a cool project as well. Someone needs to hash whatever DNS becomes into the blockchain so ther are no more CAs. This is all coming. Can't wait.
- _prometheus 11y ago
  (IPFS Dev here) hey again tptacek! o/ > I'm not clear on how IPFS protects applications from DDOS. Systems like IPFS spread the load of delivering content, but applications themselves are intrinsically centralized. Think about an application whose content is moving around entirely distributed by IPFS as well -- think of apps who run mostly on clientside, with signed (+ maybe encrypted) data generated on the users' browsers, with maybe a few "non-browser" nodes contributing to building indices or providing trusted oracles. What we're taking about is a model for webapps in which not just the content, but the logic + processing is decentralized too. At one extreme are bitcoin/ethereum style applications, where everyone runs the same computation to verify it, and another extreme where everyone just computes on their own data + the data they care about, and sign all their updates. How to do this well is not easy-- distributing the content is one part, another is making a really good capabilities library (Tahoe-LAFS has done an excellent job with this, for example, and e-rights has tons more great ideas). Another part still is thinking about the sync models with ephemeral nodes which create tons of small pieces of data, blast them out to content bouncers, and go offline. Building scalable real-time indices on this sort of stuff is going to be tricky :) Another interesting area is thinking about how databases look once you do this-- think both NoSQL AND SQL models on top of IPFS. yep, may sound crazy, but we have some preliminary work towards this (NoSQL is easy, SQL is less easy, but very doable! -- after all a database is just a good datastructure and good algorithms for operating on it). Happy to write more about this, it's a super interesting model we're exploring.
Animats 11y ago
It's a lot like BitTorrent - everything is identified by its hash. Many of the claimed advantages of IPFS can be achieved with subresource integrity. If you use subresource integrity, files are validated by their hash. We just need some convention by which the hash is encoded into the URL. Then any caching server in the path can safely fulfill the request.
- nicklaf 11y ago
  This isn't really a valid criticism of IPFS, unless you can point to a competing initiative which implements 'subresource integrity'. Alan Kay said that each object on a webpage should have its own URI way back in 1997, and yet here we are.
  - _prometheus 11y ago
    (IPFS dev) Yes! One of my explicit goals is to help get to Alan Kay's "one address for each object" goal.
- _prometheus 11y ago
  (IPFS dev here) I think subresource integrity is an _awesome_ thing that needs to be implemented and spread all over asap. (Actually, if you work on this, please ping us! we'd love to help!) However, this is very far from "many of the claimed advantages". I think it covers a few, but take a look at all the other stuff we can do, like making offline/disconnected webapps work! Think of IPFS like git or bitcoin, not just bittorrent.
euske 11y ago
This might mitigate some bandwidth problem, for sure, but how can this improve our privacy and the right to be forgotten? I think I'm missing something. Isn't this just a modern reimplementation of distributed hash tables that were researched circa 2005?
- _prometheus 11y ago
  It's more than that, take a look at the other comments in this thread, like https://news.ycombinator.com/item?id=10329236 https://news.ycombinator.com/item?id=10329236 and https://news.ycombinator.com/item?id=10329221 https://news.ycombinator.com/item?id=10329221 One interesting thing is you can have totally end-to-end encrypted applications, encrypt the application code and all the generated data. Also, remember that the HTTP Web itself was a reimplementation of decades-old hypertext systems. Hypertext hails all the way back to Xanadu (60-80s!).
Renaud 11y ago
Criticism of the article aside, having a decentralised, heavily redundant, safer and harder to disrupt web is an excellent idea worth pursuing. I can see how that could work for statics resources, but I don't get how you can decentralise the dynamic portion of a website without single points of failures to the backend.
- dogma1138 11y ago
  It's not really harder to disrupt by any means, at the end the internet is a network, or how congress thinks of it a series of pipes. Most internet censorship eventually works on L3, having a distributed alternative to the world wide web won't make the internet more resilient.
  - sroerick 11y ago
    >Most internet censorship eventually works on L3 Could you explain a little more about this?
    - tadfisher 11y ago
      OSI Layer 3: https://en.wikipedia.org/wiki/Network_layer https://en.wikipedia.org/wiki/Network_layer
      - sroerick 11y ago
        I meant with regards to censorship.
    - dogma1138 11y ago
      It's simple most internet blocking just sinkhole's L3 traffic. Deep packet inspection does happen but it's usually very rare on a large scale even for big operations like the "Big Firewall of China". The most common form of internet censorship is through blocking IP addresses/ranges, and ports, the next common one is using DNS redirection for additional filtering. URL and packet inspection (keywords) happen but are quite rare, selective protocol filtering (deep packet inspection/IX) can happen but it's very rare on national scales. Complete internet blockage is usually done simply through BGP, the government simply publishes new routes for all of their ISP's which usually just direct to a blackhole, since BGP is insecure in general an ISP can technically redirect traffic on it's own if it wants too and it did happen several times before (hacking team allegedly published rogue BGP routes to sniff traffic).
nicklaf 11y ago
I view the favorable performance characteristics of IPFS at scale (over the current centralized client-server architecture of the present web) more as a general symptom of the pathology resulting from the incomplete, ill-planned architecture of the web in general. The greatest advantage of an architecture like IPFS instead lies in its friendliness to more a democratic, semantic web, in which users and programs may make use of URI's at a fine-grained and peer-to-peer level. If we can decouple resources from a central server, and then build programs around these static resources, the web will not only become more permanent, but also more robust against walled-gardens robbing us of programmatic control of those resources. To paraphrase the Plan 9 authors[1], it might make sense to say that a p2p, content addressable web should try to build a web out of a lot of little systems, not a system out of a lot of little webs. (Well, that doesn't completely make sense, and I'm starting to mix my metaphors, but my point is that what we have now is significantly hindered by lack of fine-grained semantics allowing interop. Hypermedia has the potential to be so much more than simply manipulating the DOM with some javascript!) [1] http://www.plan9.bell-labs.com/sys/doc/9.html http://www.plan9.bell-labs.com/sys/doc/9.html
- _prometheus 11y ago
  (IPFS Dev) Yes yes yes yes! :) This is very much what we're going for. Glad you mention semantic web-- it tends to be a very tricky word with most hacker cultures, because everyone loves to hate on failed attempts :( :( -- but in fact SW (Linked Data!) is a super interesting model that has made Google and Facebook tons and tons of money (Knowledge Graph and Open Graph!). One interesting fallout of IPFS is that we can make Linked Data waaaaay more robust and faster, because you no longer need the _insane_ tons of little HTTP hits to query, retrieve content, retrieve other content + definitions, etc., on each request. With IPFS we can turn everything into a (merkle)dag and distribute it as a single bundle of objects. think Browserify/Webpack but for Linked Data. All with valid content addressing and signatures :) And +1 to plan9 -- plan9 (in particular Fossil/Venti and 9p) have always been great inspirations for me, and their philosophy surfaces a lot in IPFS. From the fossil/venti (git) approach to content, to making everything mountable, to just using the path namespace for everything, etc. :) (ps: woah, a message positive on BOTH Linked Data AND plan9! unexpected find!)
kwijibob 11y ago
I think the IPFS vision is fantastic. The article misses the main advantage when it tries to say that IPFS would help corporations like Google/Youtube/Netflix. Big players will always be able to expertly run powerful distributed CDN's, but newer smaller websites will always start with one server under the current model. IPFS would help to level the playing field for distributed data services.
- _prometheus 11y ago
  (IPFS Dev) Thanks! :) Yeah, we want to help sites scale without needing massive infrastructure funding.
jsprogrammer 11y ago
>our rapidly dwindling connectivity ... Sorry to be extremely obtuse, but since when has our connectivity been rapidly dwindling? It seems to have only been growing since the network launched.
- _prometheus 11y ago
  (IPFS Dev here) Sort of, actually. The gist is that in general yes, the network is growing and getting better. But, there's actually a growing divide between the hyper-developed cities and ... the rest of the world. One example is this: if your bandwidth is too low, or your latency too high, you basically cannot use the Web as it is today. Native mobile apps do muuuch better, because you download them once and can use them mostly offline or with low network usage. The reason this is "getting worse" is more of a perception problem and an artifact of this fact: storage is getting cheaper faster than bandwidth is getting cheaper. Over time, our media usage (and thus perceived requirements) grows and grows, tons of websites today are _several megabytes!!_ and all the media we use keeps growing to fit our nicer screens. Also, as more and more people come on line, and their usage increases, the networks saturate. This causes the "perceived bandwidth" to decrease, meaning that the pipes (which are getting absolutely better) can handle a smaller percentage of our individual load and thus "feel worse". Then, there's are other stupid problems, like the fact that many major websites/webapps will be totally useless if you go over certain latencies (particularly with wireless meshes, meaning lots of packet loss). This is because the servers' request timeouts are way too low. I was recently traveling through _Europe_ and in many places (trains on the countryside or small cities) the mobile (and sometimes wired even!) latency was so high that i could not browse the web. TLS handshakes wouldn't complete. HTTP servers would give up on me. It was terrible-- as a person accustomed to LTE + fiber ("Aaaaahhhhhh the data!!!") I couldn't believe just how stupid and bad of an experience we're giving our users out there. This was Europe-- now transport yourself to Bangladesh, or many places in rural India where people are beginning to be plugged into the internet. Think of places where access to Wikipedia, to Khan Academy, to the messenger services, could make huge life-changing differences for people. Having a bad web _there_ is blocking people from having the amazing powers of communication and computing that we all get to enjoy. We must fix these problems. And we're going to fix them by improving the data model and the distribution protocols, not with optimistic policies.
zobzu 11y ago
Such articles should start by: IPFS = Internet Protocol File system Just because it makes stuff clearer. Heck the project page could use some of that too ;)
- _prometheus 11y ago
  (IPFS Dev) Duly noted! will do! :)
- fiatjaf 11y ago
  It's Interplanetary.
  - zobzu 11y ago
    My point!
    - fiatjaf 11y ago
      Gah!
- david_ar 11y ago
  If you have any suggestions for improving the landing page, see https://github.com/ipfs/website/issues/54 https://github.com/ipfs/website/issues/54
skybrian 11y ago
For publishers, it's probably best thought of as a combination of a free CDN and the Internet Archive. But I don't think it's going to take off until there's a way to take down files you don't want published anymore. Even with the Internet Archive, if you add them to robots.txt, they will take it down. [1] Removing things from the Internet is always going to be imperfect since there will always be people who archive old copies of files (and that's a good thing). But the official software should honor retractions or mainstream publishers won't be interested. [1] https://archive.org/about/exclude.php https://archive.org/about/exclude.php
- erikpukinskis 11y ago
  Should be easy enough to add.
  - toomuchtodo 11y ago
    You're going to be able to force me to remove a hash/file off my own IPFS node? Fat chance.
    - erikpukinskis 11y ago
      No, just publish a list and allow people to voluntarily ignore those files.
- fiatjaf 11y ago
  This is crazy. IPFS is a protocol, not a central authority. Also, it is impossible to know who owns a file. Since it is identified only by a hash, everybody possessing the same file at any point could claim that file is theirs and prove it by producing the hash. In this way, downloading a file is owning it. Also, the Internet Archive policy seems too naïve. So anyone who just bought a domain can exclude everything saved at that domain in the IA up to that day?
  - skybrian 11y ago
    I think it's less about who owns the hash and more about who refers to it. Think about how git works. If more than one place refers to it then it's like it was copied. If you remove a hash from a directory, it's gone from that directory. If no other directory has it then it can be garbage collected. The key would be to convince publishers that removing the hash from their directory tree counts as deletion. Maybe the hashes shouldn't be that prominent? I don't see how the Internet Archive policy is naive. Yes, they're assuming whoever owns the domain owns the files and maybe sometimes that means someone deletes some files when they didn't have the right. But taking a stand against domain owners who want the files gone would probably land them in court.
nottednelson 11y ago
cf. http://named-data.net/ http://named-data.net/ and https://www.parc.com/work/focus-area/content-centric-networking/ https://www.parc.com/work/focus-area/content-centric-network...
- _prometheus 11y ago
  (IPFS dev here) +1 o/ NDN is awesome. If any NDN devs see this, please reach out. We'd love to work with NDN on things-- we think we can generate lots of demand for NDN
guessmyname 11y ago
"[...] Before it's too late"... But when will it be too late? EDIT: OP changed the title, it was originally written as "Why the Internet Needs IPFS Before It's Too Late"
wpietri 11y ago
Sigh. There's talk about how the Internet's "own internal contradictions [will] unravel it from within", but I'm not seeing it. The first time I can recall an "the Internet is doomed" prediction is 1995: https://en.wikipedia.org/wiki/Robert_Metcalfe#Incorrect_predictions https://en.wikipedia.org/wiki/Robert_Metcalfe#Incorrect_pred... At this point, there are circa 3 billion Internet users, nearly half the planet, so I think we're well past the point where "ZOMG GROWTH N STUFF" is a reasonable justification for this sort of hype. The Internet's growth rate peaked in the 90s: http://www.internetlivestats.com/internet-users/ http://www.internetlivestats.com/internet-users/ Now it's under 10% user growth, which seems entirely manageable.
- nicklaf 11y ago
  It hasn't been without significant abuse to the original vision of TCP/IP that the internet has grown to its present size. For example, where we needed IPv6, we got NAT. That said, there's no doubt that TCP/IP has been a runaway success, and I'm sure the credit largely belongs to some early visionary (who I should probably know about, but no name comes to mind). Edit: Vint Cerf and Bob Kahn.
- swsieber 11y ago
  New users is one way to measure growth. Total content or bandwoth consume give a different picture :)
  - wpietri 11y ago
    So you're saying that although user growth is slowing, bandwidth growth is accelerating? I'd like to see your figures for that; although I'm sure that bandwidth use is growing, I doubt it's growing faster than it was 15 years ago.
    - chipsy 11y ago
      We aren't streaming 4K video regularly yet, and 8K is coming up. Each time we double the dimensions, we increase the pixel data four times. We also have higher frame rates and bit depths. This is only going to stabilize after we've reached some extreme resolution with lossless encoding - we're only just now getting there with audio data.
huslage 11y ago
There are better things than this hack on top of IP. It won't work. It cannot work because the basic underlying protocols mean that the overhead of routing and other meta-traffic will rapidly overwhelm the scale required for it to work. Can we please stop talking about IPFS as if it could actually work??? Check out Content Centric Networking (ccnx.org) from Parc for something that actually has a chance at being a real solution.
- _prometheus 11y ago
  (IPFS author here) It turns out that: - (a) IPFS already works - (b) IPFS layers over CCN extremely well AND generates demand FOR CCN. (i'm a big fan of CCN actually, and want to see it in more and more systems. But CCN has huge adoption problems. Look, we don't even have IPv6 fully deployed yet! So if all IPFS does is generate enough demand for CCN to be fully deployed, we've done a good job.) - (c) all the routing problems are very real, but can be solved just fine. If you haven't looked deeper at how IPFS actually works instead of short summaries, you'll realize that the IPFS specs define the routing layer as entirely pluggable for this reason: we need to evolve to better and better schemes over time. Instead of blindly saying "this cannot work", read more first. Understand the systems _goals_, decisions, and roadmap. Ask questions if you're unsure how something could possibly work. Lots of very smart people are working on IPFS and we're doing it because we see that it can indeed (and does) work. Blind negativity does not help CCN, and does not help anyone make better things.
  - huslage 11y ago
    I see the goals and I've seen these exact problems attempting to be fixed in similar ways many times. I think the goals and roadmap are great, but overly idealized. I also think it's important to be skeptical of claims of this sort. That being said, I hope you're right.
firebones 11y ago
One milestone for IPFS: get past the point of the primary informational site about it (ipfs.io) being categorically blocked by corporate content filters as "blocked: peer-to-peer". It's hard to convey the vision within your corporation when you can't even share the primary informational site.
- _prometheus 11y ago
  (IPFS dev here) wow, that sucks. is this blocked in your corp firewall? Any chance of talking to the sysadmins? Blocking IPFS is like blocking HTTP :/ :/ :/ We may have to move our gateway back off our main domain, and have a "recommended blocker", so that people don't jump on this. Anyway, one piece of good news: despite everyone's belief about UDP transports never going through corp firewalls, QUIC now handles MOST (>60%, close to 80% i believe) of "Google Chrome<-->Google sites" traffic across the world. So, temp setbacks may be annoying, but in the end, once users install IPFS -- and once we have browser implementations, we can make all the traffic look like HTTP TLS flows, so blocking it will be very hard without also blocking regular HTTP TLS traffic.
  - dingaling 11y ago
    > so blocking it will be very hard without also blocking regular HTTP TLS traffic. Many large or data-sensitive companies do SSL MiTM at their firewall so unfortunately that's not hard at all.
    - _prometheus 11y ago
      true, but then you're not in a {censorship, attacker} free environment at all -- MITMing TLS _is_ violating the security expectations of today's websites. I wouldn't even check my non-work email. i.e. all bets are off. there will always be pockets. but over time we could win even there, as the perf improvement will matter.
    - fulafel 11y ago
      This is a minority though and the users on such nets have pretty diminished expectations on the level of net access they have. (It's also illegal BTW in civilized countries! At least on networks with employees doing web browsing, as that's personal communications of the employee and and part of fundamental rights)
MCRed 11y ago
I recently learned about MAID SAFE-- http://maidsafe.net http://maidsafe.net -- but I haven't researched it much. I would be interested if someone could compare and contrast the two.
- david_ar 11y ago
  https://github.com/ipfs/faq/issues/2 https://github.com/ipfs/faq/issues/2
grayfox 11y ago
Hey, _prometheus! What's an email I can reach you at? I'd like to pick your brain about some of this stuff and how it relates to databases. Cheers!
crablar 11y ago
Podcast about IPFS with Juan Benet: http://softwareengineeringdaily.com/2015/08/25/interplanetary-file-system-ipfs-with-juan-benet/ http://softwareengineeringdaily.com/2015/08/25/interplanetar...
friendzis 11y ago
I see two core problems with content addressable web: * Dynamic content * Private/sensitive information Dynamic content on a distributed system poses rather hard challenges on data integrity - how would it be possible to ensure that different pieces of data you have are actually from the same [temporal] set? The very same HN - how would it be possible to see at least all the children of parent you want to comment to in a distributed manner? Anything I can come up with involves some sort of central dependency repository. Two child "commits" create branches in content tree. If these children are duplicates - we pick the better one and cast the other into abyss, the bigger challenge is how do we merge them together. This is a wide topic, so please excuse my brevity. Private information is yet another rake to step on. I would really love my bank account information to remain accessible to me and the bank only. I don't even trust myself not to lose decryption keys, I don't trust long-term security of encryption algorithms (flaws could be found, brute forcing might become viable), therefore the information better stays in as little places as possible. Another end of this stick is authentication/authorization. Encryption does not work, because access rights change. What I'm authorized to view/do today might not be the same tomorrow. The only solution is not to serve the content in the first place. As for authentication I don't see a solution at all. Although content addressable web is awesome solution for [more or less] static content - wikipedia, blogs, etc.
- david_ar 11y ago
  Re your first question, see https://github.com/ipfs/notes/issues/40 https://github.com/ipfs/notes/issues/40 In terms of revoking access rights, that's not really a problem unique to IPFS (cf. DRM)
- einrealist 11y ago
  I see more core problems: 1) This only scales if enough people are willing to give up a portion of their storage devices. I would think twice about that for my SSDs. 2) Private customer internet connections often have a lesser upload bandwidth and that is part of the overall traffic calculation of ISPs. So the networks have to change a lot to face changing traffic requirements in this regard. That is something that will not scale with the demand of a successful protocol (see BitTorrent). Also this IPFS traffic may harmful to other traffic like gaming sessions. 3) Illegal content will be the killer. Don't expect laws to change for IPFS. No one will permanently use VPNs / anonymizers permanently for using IPFS, because some law firms will specialise on gathering endpoint information like they already do with BitTorrent.
jerguismi 11y ago
“We use content-addressing so content can be decoupled from origin servers, and instead, can be stored permanently. " Why should I believe that claim? What are the incentives for storing my data in the network? As far as I understand, no incentivization is done for running the network. That's why I wouldn't trust it to store anything but a trivial amount of data.
- david_ar 11y ago
  Not yet, but this is planned: http://filecoin.io/ http://filecoin.io/
  - repomies691 11y ago
    OK, why would I believe that IPFS is able to handle this issue better than the numerous other projects trying to implement incentivization? IPFS focus has been quite clearly elsewhere.
    - david_ar 11y ago
      You're right, this isn't the only possible incentive system, and ideally IPFS will be compatible with multiple ones. Could you link to the other projects you're talking about?
      - DanWaterworth 11y ago
        There are a few things that I see as less than ideal with IPFS, so I recently set myself the challenge of designing a better system. I really like the way IPFS breaks down the problem, so the components remain the same. Regarding incentivization, looking at the literature, you can split existing incentive schemes into two camps, local and global. Tit-for-tat would be an example of a local scheme, currency based systems would be global. Neither of these types of scheme really work, with local schemes, you have no way to use accrued reputation with new nodes. Global schemes require too much coordination and overhead to be suitable for small 'transactions'. As I understand it, IPFS deals with the problem by having a local scheme, but with a get out clause. If you want something you may have to do work on behalf of the node you want something from, but that work should be easier if you have a good reputation with other nodes in the network. Under my proposed system, nodes keep track of how much they are in debt or in credit with the nodes that they are in contact with. The amount that you can go into debt with another node is limited to 1% of the amount of useful data that you have transferred to that node. (This way, if a node terminates a connection whilst it is in debt to another node, that loss, amortized over the useful data that you have received is very small). Nodes are incentivized keep connections open, since short lived connections have very tight reins on how much debt/credit can be accrued. As this stands, this doesn't solve the problem. It's entirely local and there's no way to initiate connections since nodes, by default, don't trust each other. The solution is two-fold: credit transfer and proof of work. I have designed a protocol that allows nodes to transfer credit to cancel out 'cycles of debt'. Nodes can also gain credit with other nodes by solving pointless problems. The innovation here is that when confronted with a puzzle, a node is able to delegate the work to its debtors. Proof of work is useful since, given the choice, a node would probably prefer to use its upstream bandwidth than burn up CPU time. There are more details to this, like: the specifics of how to discover and execute a credit transfer along a long chain using only local interactions, how to incentive routing as well as data transfer and many other things. If I had the time, I'd implement it.
deleted 11y ago
[deleted]
deleted 11y ago
[deleted]
chmike 11y ago
This is a naive idea that is wrong is so many way. It keeps popping up recurrently and I don't understand why people didn't see yet that it won't be the next Internet revolution. Oh well! Let me try to explain it again. The idea of content base key is strongly narrowing the application domain of the system. Modifying your information (I.e. typo correction) invalidates all the keys. More precisely, people holding the old key won't be able to access the corrected information. of course this model fails completely with dynamic data. The other problem that is often overlooked with such idea is the function that translate your key into the location of the information. That is: determine the IP address of the server(s) hosting the information from the hash key of it. One needs something like a distributed index for that. Don't use an algorithm because you don't want to add the constrain that your informations can't be moved or replicated. Another problem is staying in control of your information. An owner of information want to be able to fix or modify it, or delete it when he wants. Finally, another naive idea is sharing a distributed storage. This is a nice idea but it simply doesn't work. Some People will abuse it. To avoid this you need a system to monitor and control usage. Accounting ? good luck. By the way, this is the problem of Internet today. It is a shared resource, but a few are consuming a hell lot of it and earning a lot of money without sharing back the profit. I'm looking at you google with YouTube. I'm thinking and working on this problem for a long time. My conclusions are : 1. Decouple keys from content 2. Make the distributed locating index your infrastructure 3. Optimize keys for index traversal 4. Owner of information must stay in control of Information 5. Owner of information must assume the cost of hosting
- david_ar 11y ago
  I suggest you read more about IPFS (or even just other comments on this post), as many of the issues you raise have already been solved (1-3), or have planned solutions (4-5).
- mirimir 11y ago
  > 4. Owner of information must stay in control of Information Once you put something on the Internet, you no longer own it.
  - 11y ago
mirimir 11y ago
I wonder if IPFS can access resources via Tor onion services.
- david_ar 11y ago
  https://github.com/ipfs/notes/issues/37 https://github.com/ipfs/notes/issues/37
  - mirimir 11y ago
    Thanks. I'll test.
ilaksh 11y ago
You can tell this is a great idea by how much some people hate it.

341 points by confiscate 11y ago

It seems like a pastime of TC is to write sensational filler articles out of the more questionable bile emanating from HN. See also "Death to C": https://news.ycombinator.com/item?id=9477878 https://news.ycombinator.com/item?id=9477878

_prometheus 11y ago
(IPFS dev here) "questionable bile"? we'd love your technical feedback. We know we have a ton of things to do better -- but what sort of things are you thinking about? what can we improve on?
- larzang 11y ago
  The core of the modern web isn't static: it's transactional, it's personalized, and it's end-to-end encrypted for both security and privacy. How does a distributed caching model accommodate such things? If it can't, how can it be a replacement for http? If it's not a replacement for http, is it at least better than current CDNs? If so, how? It's hard to imagine you haven't already thought of these questions and answered them somewhere, but this non-article certainly doesn't address it or point to better resources (besides "buy my book!").
  - _prometheus 11y ago
    IPFS is not _just_ a static system. See the comments above: https://news.ycombinator.com/item?id=10329221 https://news.ycombinator.com/item?id=10329221 -- note that git ant bitcoin are "just static systems" as much as ipfs :). It turns out all content is "static", it's just "static content" that is "dynamically generated" at times-- the point being that you can still do all the dynamic stuff just fine, and use IPFS as a transport for the data.

smegger001 11y ago

IPFS is essentially web (here meaning htmls css & javascript) over torrent + a distributed hash table. Correct me if I am wrong but isn't this just freenet without the anonymity or kind of like tribler but for websites instead of pirated files?

mburns 11y ago
IPFS shares a lot of similarities to the Freenet datastore, yes. Both support static and dynamic keys that map to content-addressed URIs and can be accessed via a DHT. IPFS lets you selectively (opt-in) save data locally and host it, unlike Freenet which can cache and serve that data to other clients without user action. The lack of Freenet's anonymity feature means it can perform much faster and even be implemented in client-side javascript (Work In Progress) which will drastically increase user adoption.
- _prometheus 11y ago
  (IPFS Dev) Yeah, we have many similarities with Freenet, but we also have significant differences. You mention the lack of anonymity, and yep, that's definitely one -- to speak more about this: this parts from our content model. Among explicit constraints, we have: - content must be able to move as fast as the underlying network permits. this rules out designs like freenet's and other oblivious storage platforms, as the base case. Like you said, they're just way too slow for most of IPFS use cases. But, these can be implemented trivially with the use of privacy focused transports (like Tor and I2P -- there's actually work towards this and people are getting close), content encryption, and so on. - IPFS nodes should be able to only store and/or distribute content they _explicitly_ want to store and/or distribute. This means that computers that run IPFS nodes do not have to host "other people's stuff", which is a very important thing when you consider that lots of content in the internet is -- in some for or other -- illegal under certain jurisdictions. Legitimate companies have way too much in their plate to additionally worry about potentially be storing a ton of illegal stuff. For serious companies like Google to use IPFS, we need to have a mode of operation that allows implementations to ONLY move the content THEY want to move. - Websites/Webapps must be able to operate entirely disconnected -- this means that it should be possible to build applications which create data locally, signed by the user, which can be distributed encrypted end-to-end to other users, without needing to ever touch specific backbone servers. This means users can move the data end-to-end via the closest route possible and in disconnected networks (think users on mobile phones on a plane using messaging webapp or web game and moving the bits over bluetooth or an ad-hoc wifi network. And this also means users in the broader internet do not _need to_ rely on backbone servers -- the model we're going for is that dedicated computers in the backbone CAN of course and SHOULD help, but you shouldn't HAVE TO RELY on them. Can write more about this :)

AstroJetson 11y ago

"As I explain in my upcoming book...." now the article makes sense, TC is helping her pimp her book. What the internet needs is a new financial model since the one we are have now isn't working in the long term.

tim333 11y ago
Yeah - stuff like "Before It’s Too Late" seems like hype to sell books. As far as I can tell the internet gets faster and cheaper each year and is not about to have it's "own internal contradictions unravel it from within".
dfc 11y ago
> Now this article makes sense This is all TC does, you can call it pimping or fluffing. Before this article did you think TC was an investigative journalism powerhouse? http://www.paulgraham.com/submarine.html http://www.paulgraham.com/submarine.html

sklivvz1971 11y ago

Meh, it's a distributed static website thing. Of course if one doesn't need transactions, everything is easier even today -- I mean, who doesn't use a CDN? It's a very similar concept. The problem is much tougher when the content is personalized or when there needs to be a transaction. This makes distribution a huge huge problem no one has solved yet (even a blockchain, IIRC can't really scale up in transactions...)

_prometheus 11y ago
(IPFS dev here) Hey! Yeah, we're focusing right now on "distributed static websites" because those are really easy to get right and make work all over the network. However, what's _really_ interesting about IPFS is making completely distributed applications which can do their process unhinged from a single origin -- websites/webapps without an origin server. Posted more about this elsewhere on this thread, take a look: https://news.ycombinator.com/item?id=10329221 https://news.ycombinator.com/item?id=10329221 Also see the Neocities + IPFS blog post: https://ipfs.io/ipfs/QmTVcD87Ecjps6wv9jMaGhvMuzZ2BgP6NyXDcnMU74RELx/ https://ipfs.io/ipfs/QmTVcD87Ecjps6wv9jMaGhvMuzZ2BgP6NyXDcnM... Take a look also at this talk, which presents different use cases for this: https://www.youtube.com/watch?v=skMTdSEaCtA https://www.youtube.com/watch?v=skMTdSEaCtA Can also see some of the random crazy things we're planning in https://github.com/ipfs/notes/issues/ https://github.com/ipfs/notes/issues/ + https://github.com/ipfs/archives/issues/ https://github.com/ipfs/archives/issues/ + https://github.com/ipfs/apps/issues/ https://github.com/ipfs/apps/issues/ (Edit: fix links)
- supdog 11y ago
  Link to .../ipfs/notes/ appears to contain 37 bytes of nothing. Is the content available elseware? Would love to see the random crazy things you're planning.
  - lgierth 11y ago
    Yep sorry, it's in the issue tracker: https://github.com/ipfs/notes/issues https://github.com/ipfs/notes/issues

karissa 11y ago

This kinda makes me sick to my stomach -- essentially, she's saying that a peer-to-peer internet is great because companies will be able to have better uptime. She doesn't even cover the clearly obvious economic aspects of this -- why would I run an IPFS node if it just benefits the company and not me?

_prometheus 11y ago
(IPFS dev here) I think she does, but perhaps the discrepancy may be that she's got a different target audience: the established website owners? Also to be fair, there's not much space in these sorts of articles LOTS gets cut by editors and -- from having done something like this -- you have to hit this absuuuurdly high level and can't sink into details much at all. I'm actually pretty surprised at the level of technical detail in this article -- i would've expected much more to not make it past the "remove jargon - write for the average user" media filter. I've done some interviews / articles that ended up annoyingly waaaaay more high level and completely missed my expected mark. You might want to write to the author, too. :)
- esfandia 11y ago
  The OP asks an interesting question though: why would I run an IPFS node if it just benefits the company and not me?
  - reitanqild 11y ago
    Maybe you won't. IIRC local machines by default only caches stuff for a very short time which makes sense for personal computers. On company networks however I'd expect sysadmins to run nodes to reduce upstream bandwidth usage. Edit: see also https://news.ycombinator.com/item?id=10329262 https://news.ycombinator.com/item?id=10329262 the specific example is the exact opposite but I see no technical reason why you cannot do this the other way around.
  - david_ar 11y ago
    There will be incentives in the future, both in terms of promoting a good seeding/leeching ratio like bittorrent ("bitswap"), as well as http://filecoin.io/ http://filecoin.io/

tptacek 11y ago

I'm a believer in this idea generally --- that we should replace applications built directly on IP/TCP with applications built on a content-addressed overlay network running on top of IP/TCP --- but I think the logic used here is faulty. For instance: I'm not clear on how IPFS protects applications from DDOS. Systems like IPFS spread the load of delivering content, but applications themselves are intrinsically centralized.

on_ 11y ago
I totally agree. leaving aside the granular details in the article this is the future. Bitcoin protocol, ethereum, ipfs, openbazaar, bittorrent etc are the blocks that content will be rebuilt on. The SAFEnetwork is a cool project as well. Someone needs to hash whatever DNS becomes into the blockchain so ther are no more CAs. This is all coming. Can't wait.
_prometheus 11y ago
(IPFS Dev here) hey again tptacek! o/ > I'm not clear on how IPFS protects applications from DDOS. Systems like IPFS spread the load of delivering content, but applications themselves are intrinsically centralized. Think about an application whose content is moving around entirely distributed by IPFS as well -- think of apps who run mostly on clientside, with signed (+ maybe encrypted) data generated on the users' browsers, with maybe a few "non-browser" nodes contributing to building indices or providing trusted oracles. What we're taking about is a model for webapps in which not just the content, but the logic + processing is decentralized too. At one extreme are bitcoin/ethereum style applications, where everyone runs the same computation to verify it, and another extreme where everyone just computes on their own data + the data they care about, and sign all their updates. How to do this well is not easy-- distributing the content is one part, another is making a really good capabilities library (Tahoe-LAFS has done an excellent job with this, for example, and e-rights has tons more great ideas). Another part still is thinking about the sync models with ephemeral nodes which create tons of small pieces of data, blast them out to content bouncers, and go offline. Building scalable real-time indices on this sort of stuff is going to be tricky :) Another interesting area is thinking about how databases look once you do this-- think both NoSQL AND SQL models on top of IPFS. yep, may sound crazy, but we have some preliminary work towards this (NoSQL is easy, SQL is less easy, but very doable! -- after all a database is just a good datastructure and good algorithms for operating on it). Happy to write more about this, it's a super interesting model we're exploring.

Animats 11y ago

It's a lot like BitTorrent - everything is identified by its hash. Many of the claimed advantages of IPFS can be achieved with subresource integrity. If you use subresource integrity, files are validated by their hash. We just need some convention by which the hash is encoded into the URL. Then any caching server in the path can safely fulfill the request.

nicklaf 11y ago
This isn't really a valid criticism of IPFS, unless you can point to a competing initiative which implements 'subresource integrity'. Alan Kay said that each object on a webpage should have its own URI way back in 1997, and yet here we are.
- _prometheus 11y ago
  (IPFS dev) Yes! One of my explicit goals is to help get to Alan Kay's "one address for each object" goal.
_prometheus 11y ago
(IPFS dev here) I think subresource integrity is an _awesome_ thing that needs to be implemented and spread all over asap. (Actually, if you work on this, please ping us! we'd love to help!) However, this is very far from "many of the claimed advantages". I think it covers a few, but take a look at all the other stuff we can do, like making offline/disconnected webapps work! Think of IPFS like git or bitcoin, not just bittorrent.

euske 11y ago

This might mitigate some bandwidth problem, for sure, but how can this improve our privacy and the right to be forgotten? I think I'm missing something. Isn't this just a modern reimplementation of distributed hash tables that were researched circa 2005?

_prometheus 11y ago
It's more than that, take a look at the other comments in this thread, like https://news.ycombinator.com/item?id=10329236 https://news.ycombinator.com/item?id=10329236 and https://news.ycombinator.com/item?id=10329221 https://news.ycombinator.com/item?id=10329221 One interesting thing is you can have totally end-to-end encrypted applications, encrypt the application code and all the generated data. Also, remember that the HTTP Web itself was a reimplementation of decades-old hypertext systems. Hypertext hails all the way back to Xanadu (60-80s!).

Renaud 11y ago

Criticism of the article aside, having a decentralised, heavily redundant, safer and harder to disrupt web is an excellent idea worth pursuing. I can see how that could work for statics resources, but I don't get how you can decentralise the dynamic portion of a website without single points of failures to the backend.

dogma1138 11y ago
It's not really harder to disrupt by any means, at the end the internet is a network, or how congress thinks of it a series of pipes. Most internet censorship eventually works on L3, having a distributed alternative to the world wide web won't make the internet more resilient.
- sroerick 11y ago
  >Most internet censorship eventually works on L3 Could you explain a little more about this?
  - tadfisher 11y ago
    OSI Layer 3: https://en.wikipedia.org/wiki/Network_layer https://en.wikipedia.org/wiki/Network_layer
    - sroerick 11y ago
      I meant with regards to censorship.
  - dogma1138 11y ago
    It's simple most internet blocking just sinkhole's L3 traffic. Deep packet inspection does happen but it's usually very rare on a large scale even for big operations like the "Big Firewall of China". The most common form of internet censorship is through blocking IP addresses/ranges, and ports, the next common one is using DNS redirection for additional filtering. URL and packet inspection (keywords) happen but are quite rare, selective protocol filtering (deep packet inspection/IX) can happen but it's very rare on national scales. Complete internet blockage is usually done simply through BGP, the government simply publishes new routes for all of their ISP's which usually just direct to a blackhole, since BGP is insecure in general an ISP can technically redirect traffic on it's own if it wants too and it did happen several times before (hacking team allegedly published rogue BGP routes to sniff traffic).

nicklaf 11y ago

I view the favorable performance characteristics of IPFS at scale (over the current centralized client-server architecture of the present web) more as a general symptom of the pathology resulting from the incomplete, ill-planned architecture of the web in general. The greatest advantage of an architecture like IPFS instead lies in its friendliness to more a democratic, semantic web, in which users and programs may make use of URI's at a fine-grained and peer-to-peer level. If we can decouple resources from a central server, and then build programs around these static resources, the web will not only become more permanent, but also more robust against walled-gardens robbing us of programmatic control of those resources. To paraphrase the Plan 9 authors[1], it might make sense to say that a p2p, content addressable web should try to build a web out of a lot of little systems, not a system out of a lot of little webs. (Well, that doesn't completely make sense, and I'm starting to mix my metaphors, but my point is that what we have now is significantly hindered by lack of fine-grained semantics allowing interop. Hypermedia has the potential to be so much more than simply manipulating the DOM with some javascript!) [1] http://www.plan9.bell-labs.com/sys/doc/9.html http://www.plan9.bell-labs.com/sys/doc/9.html

_prometheus 11y ago
(IPFS Dev) Yes yes yes yes! :) This is very much what we're going for. Glad you mention semantic web-- it tends to be a very tricky word with most hacker cultures, because everyone loves to hate on failed attempts :( :( -- but in fact SW (Linked Data!) is a super interesting model that has made Google and Facebook tons and tons of money (Knowledge Graph and Open Graph!). One interesting fallout of IPFS is that we can make Linked Data waaaaay more robust and faster, because you no longer need the _insane_ tons of little HTTP hits to query, retrieve content, retrieve other content + definitions, etc., on each request. With IPFS we can turn everything into a (merkle)dag and distribute it as a single bundle of objects. think Browserify/Webpack but for Linked Data. All with valid content addressing and signatures :) And +1 to plan9 -- plan9 (in particular Fossil/Venti and 9p) have always been great inspirations for me, and their philosophy surfaces a lot in IPFS. From the fossil/venti (git) approach to content, to making everything mountable, to just using the path namespace for everything, etc. :) (ps: woah, a message positive on BOTH Linked Data AND plan9! unexpected find!)

kwijibob 11y ago

I think the IPFS vision is fantastic. The article misses the main advantage when it tries to say that IPFS would help corporations like Google/Youtube/Netflix. Big players will always be able to expertly run powerful distributed CDN's, but newer smaller websites will always start with one server under the current model. IPFS would help to level the playing field for distributed data services.

_prometheus 11y ago
(IPFS Dev) Thanks! :) Yeah, we want to help sites scale without needing massive infrastructure funding.

jsprogrammer 11y ago

>our rapidly dwindling connectivity ... Sorry to be extremely obtuse, but since when has our connectivity been rapidly dwindling? It seems to have only been growing since the network launched.

_prometheus 11y ago
(IPFS Dev here) Sort of, actually. The gist is that in general yes, the network is growing and getting better. But, there's actually a growing divide between the hyper-developed cities and ... the rest of the world. One example is this: if your bandwidth is too low, or your latency too high, you basically cannot use the Web as it is today. Native mobile apps do muuuch better, because you download them once and can use them mostly offline or with low network usage. The reason this is "getting worse" is more of a perception problem and an artifact of this fact: storage is getting cheaper faster than bandwidth is getting cheaper. Over time, our media usage (and thus perceived requirements) grows and grows, tons of websites today are _several megabytes!!_ and all the media we use keeps growing to fit our nicer screens. Also, as more and more people come on line, and their usage increases, the networks saturate. This causes the "perceived bandwidth" to decrease, meaning that the pipes (which are getting absolutely better) can handle a smaller percentage of our individual load and thus "feel worse". Then, there's are other stupid problems, like the fact that many major websites/webapps will be totally useless if you go over certain latencies (particularly with wireless meshes, meaning lots of packet loss). This is because the servers' request timeouts are way too low. I was recently traveling through _Europe_ and in many places (trains on the countryside or small cities) the mobile (and sometimes wired even!) latency was so high that i could not browse the web. TLS handshakes wouldn't complete. HTTP servers would give up on me. It was terrible-- as a person accustomed to LTE + fiber ("Aaaaahhhhhh the data!!!") I couldn't believe just how stupid and bad of an experience we're giving our users out there. This was Europe-- now transport yourself to Bangladesh, or many places in rural India where people are beginning to be plugged into the internet. Think of places where access to Wikipedia, to Khan Academy, to the messenger services, could make huge life-changing differences for people. Having a bad web _there_ is blocking people from having the amazing powers of communication and computing that we all get to enjoy. We must fix these problems. And we're going to fix them by improving the data model and the distribution protocols, not with optimistic policies.

zobzu 11y ago

Such articles should start by: IPFS = Internet Protocol File system Just because it makes stuff clearer. Heck the project page could use some of that too ;)

_prometheus 11y ago
(IPFS Dev) Duly noted! will do! :)
fiatjaf 11y ago
It's Interplanetary.
- zobzu 11y ago
  My point!
  - fiatjaf 11y ago
    Gah!
david_ar 11y ago
If you have any suggestions for improving the landing page, see https://github.com/ipfs/website/issues/54 https://github.com/ipfs/website/issues/54

skybrian 11y ago

For publishers, it's probably best thought of as a combination of a free CDN and the Internet Archive. But I don't think it's going to take off until there's a way to take down files you don't want published anymore. Even with the Internet Archive, if you add them to robots.txt, they will take it down. [1] Removing things from the Internet is always going to be imperfect since there will always be people who archive old copies of files (and that's a good thing). But the official software should honor retractions or mainstream publishers won't be interested. [1] https://archive.org/about/exclude.php https://archive.org/about/exclude.php

erikpukinskis 11y ago
Should be easy enough to add.
- toomuchtodo 11y ago
  You're going to be able to force me to remove a hash/file off my own IPFS node? Fat chance.
  - erikpukinskis 11y ago
    No, just publish a list and allow people to voluntarily ignore those files.
fiatjaf 11y ago
This is crazy. IPFS is a protocol, not a central authority. Also, it is impossible to know who owns a file. Since it is identified only by a hash, everybody possessing the same file at any point could claim that file is theirs and prove it by producing the hash. In this way, downloading a file is owning it. Also, the Internet Archive policy seems too naïve. So anyone who just bought a domain can exclude everything saved at that domain in the IA up to that day?
- skybrian 11y ago
  I think it's less about who owns the hash and more about who refers to it. Think about how git works. If more than one place refers to it then it's like it was copied. If you remove a hash from a directory, it's gone from that directory. If no other directory has it then it can be garbage collected. The key would be to convince publishers that removing the hash from their directory tree counts as deletion. Maybe the hashes shouldn't be that prominent? I don't see how the Internet Archive policy is naive. Yes, they're assuming whoever owns the domain owns the files and maybe sometimes that means someone deletes some files when they didn't have the right. But taking a stand against domain owners who want the files gone would probably land them in court.

nottednelson 11y ago

cf. http://named-data.net/ http://named-data.net/ and https://www.parc.com/work/focus-area/content-centric-networking/ https://www.parc.com/work/focus-area/content-centric-network...

_prometheus 11y ago
(IPFS dev here) +1 o/ NDN is awesome. If any NDN devs see this, please reach out. We'd love to work with NDN on things-- we think we can generate lots of demand for NDN

guessmyname 11y ago

"[...] Before it's too late"... But when will it be too late? EDIT: OP changed the title, it was originally written as "Why the Internet Needs IPFS Before It's Too Late"

wpietri 11y ago

Sigh. There's talk about how the Internet's "own internal contradictions [will] unravel it from within", but I'm not seeing it. The first time I can recall an "the Internet is doomed" prediction is 1995: https://en.wikipedia.org/wiki/Robert_Metcalfe#Incorrect_predictions https://en.wikipedia.org/wiki/Robert_Metcalfe#Incorrect_pred... At this point, there are circa 3 billion Internet users, nearly half the planet, so I think we're well past the point where "ZOMG GROWTH N STUFF" is a reasonable justification for this sort of hype. The Internet's growth rate peaked in the 90s: http://www.internetlivestats.com/internet-users/ http://www.internetlivestats.com/internet-users/ Now it's under 10% user growth, which seems entirely manageable.

nicklaf 11y ago
It hasn't been without significant abuse to the original vision of TCP/IP that the internet has grown to its present size. For example, where we needed IPv6, we got NAT. That said, there's no doubt that TCP/IP has been a runaway success, and I'm sure the credit largely belongs to some early visionary (who I should probably know about, but no name comes to mind). Edit: Vint Cerf and Bob Kahn.
swsieber 11y ago
New users is one way to measure growth. Total content or bandwoth consume give a different picture :)
- wpietri 11y ago
  So you're saying that although user growth is slowing, bandwidth growth is accelerating? I'd like to see your figures for that; although I'm sure that bandwidth use is growing, I doubt it's growing faster than it was 15 years ago.
  - chipsy 11y ago
    We aren't streaming 4K video regularly yet, and 8K is coming up. Each time we double the dimensions, we increase the pixel data four times. We also have higher frame rates and bit depths. This is only going to stabilize after we've reached some extreme resolution with lossless encoding - we're only just now getting there with audio data.

huslage 11y ago

There are better things than this hack on top of IP. It won't work. It cannot work because the basic underlying protocols mean that the overhead of routing and other meta-traffic will rapidly overwhelm the scale required for it to work. Can we please stop talking about IPFS as if it could actually work??? Check out Content Centric Networking (ccnx.org) from Parc for something that actually has a chance at being a real solution.

_prometheus 11y ago
(IPFS author here) It turns out that: - (a) IPFS already works - (b) IPFS layers over CCN extremely well AND generates demand FOR CCN. (i'm a big fan of CCN actually, and want to see it in more and more systems. But CCN has huge adoption problems. Look, we don't even have IPv6 fully deployed yet! So if all IPFS does is generate enough demand for CCN to be fully deployed, we've done a good job.) - (c) all the routing problems are very real, but can be solved just fine. If you haven't looked deeper at how IPFS actually works instead of short summaries, you'll realize that the IPFS specs define the routing layer as entirely pluggable for this reason: we need to evolve to better and better schemes over time. Instead of blindly saying "this cannot work", read more first. Understand the systems _goals_, decisions, and roadmap. Ask questions if you're unsure how something could possibly work. Lots of very smart people are working on IPFS and we're doing it because we see that it can indeed (and does) work. Blind negativity does not help CCN, and does not help anyone make better things.
- huslage 11y ago
  I see the goals and I've seen these exact problems attempting to be fixed in similar ways many times. I think the goals and roadmap are great, but overly idealized. I also think it's important to be skeptical of claims of this sort. That being said, I hope you're right.

firebones 11y ago

One milestone for IPFS: get past the point of the primary informational site about it (ipfs.io) being categorically blocked by corporate content filters as "blocked: peer-to-peer". It's hard to convey the vision within your corporation when you can't even share the primary informational site.

_prometheus 11y ago
(IPFS dev here) wow, that sucks. is this blocked in your corp firewall? Any chance of talking to the sysadmins? Blocking IPFS is like blocking HTTP :/ :/ :/ We may have to move our gateway back off our main domain, and have a "recommended blocker", so that people don't jump on this. Anyway, one piece of good news: despite everyone's belief about UDP transports never going through corp firewalls, QUIC now handles MOST (>60%, close to 80% i believe) of "Google Chrome<-->Google sites" traffic across the world. So, temp setbacks may be annoying, but in the end, once users install IPFS -- and once we have browser implementations, we can make all the traffic look like HTTP TLS flows, so blocking it will be very hard without also blocking regular HTTP TLS traffic.
- dingaling 11y ago
  > so blocking it will be very hard without also blocking regular HTTP TLS traffic. Many large or data-sensitive companies do SSL MiTM at their firewall so unfortunately that's not hard at all.
  - _prometheus 11y ago
    true, but then you're not in a {censorship, attacker} free environment at all -- MITMing TLS _is_ violating the security expectations of today's websites. I wouldn't even check my non-work email. i.e. all bets are off. there will always be pockets. but over time we could win even there, as the perf improvement will matter.
  - fulafel 11y ago
    This is a minority though and the users on such nets have pretty diminished expectations on the level of net access they have. (It's also illegal BTW in civilized countries! At least on networks with employees doing web browsing, as that's personal communications of the employee and and part of fundamental rights)

MCRed 11y ago

I recently learned about MAID SAFE-- http://maidsafe.net http://maidsafe.net -- but I haven't researched it much. I would be interested if someone could compare and contrast the two.

david_ar 11y ago
https://github.com/ipfs/faq/issues/2 https://github.com/ipfs/faq/issues/2

grayfox 11y ago

Hey, _prometheus! What's an email I can reach you at? I'd like to pick your brain about some of this stuff and how it relates to databases. Cheers!

crablar 11y ago

Podcast about IPFS with Juan Benet: http://softwareengineeringdaily.com/2015/08/25/interplanetary-file-system-ipfs-with-juan-benet/ http://softwareengineeringdaily.com/2015/08/25/interplanetar...

friendzis 11y ago

I see two core problems with content addressable web: * Dynamic content * Private/sensitive information Dynamic content on a distributed system poses rather hard challenges on data integrity - how would it be possible to ensure that different pieces of data you have are actually from the same [temporal] set? The very same HN - how would it be possible to see at least all the children of parent you want to comment to in a distributed manner? Anything I can come up with involves some sort of central dependency repository. Two child "commits" create branches in content tree. If these children are duplicates - we pick the better one and cast the other into abyss, the bigger challenge is how do we merge them together. This is a wide topic, so please excuse my brevity. Private information is yet another rake to step on. I would really love my bank account information to remain accessible to me and the bank only. I don't even trust myself not to lose decryption keys, I don't trust long-term security of encryption algorithms (flaws could be found, brute forcing might become viable), therefore the information better stays in as little places as possible. Another end of this stick is authentication/authorization. Encryption does not work, because access rights change. What I'm authorized to view/do today might not be the same tomorrow. The only solution is not to serve the content in the first place. As for authentication I don't see a solution at all. Although content addressable web is awesome solution for [more or less] static content - wikipedia, blogs, etc.

david_ar 11y ago
Re your first question, see https://github.com/ipfs/notes/issues/40 https://github.com/ipfs/notes/issues/40 In terms of revoking access rights, that's not really a problem unique to IPFS (cf. DRM)
einrealist 11y ago
I see more core problems: 1) This only scales if enough people are willing to give up a portion of their storage devices. I would think twice about that for my SSDs. 2) Private customer internet connections often have a lesser upload bandwidth and that is part of the overall traffic calculation of ISPs. So the networks have to change a lot to face changing traffic requirements in this regard. That is something that will not scale with the demand of a successful protocol (see BitTorrent). Also this IPFS traffic may harmful to other traffic like gaming sessions. 3) Illegal content will be the killer. Don't expect laws to change for IPFS. No one will permanently use VPNs / anonymizers permanently for using IPFS, because some law firms will specialise on gathering endpoint information like they already do with BitTorrent.

jerguismi 11y ago

“We use content-addressing so content can be decoupled from origin servers, and instead, can be stored permanently. " Why should I believe that claim? What are the incentives for storing my data in the network? As far as I understand, no incentivization is done for running the network. That's why I wouldn't trust it to store anything but a trivial amount of data.

david_ar 11y ago
Not yet, but this is planned: http://filecoin.io/ http://filecoin.io/
- repomies691 11y ago
  OK, why would I believe that IPFS is able to handle this issue better than the numerous other projects trying to implement incentivization? IPFS focus has been quite clearly elsewhere.
  - david_ar 11y ago
    You're right, this isn't the only possible incentive system, and ideally IPFS will be compatible with multiple ones. Could you link to the other projects you're talking about?
    - DanWaterworth 11y ago
      There are a few things that I see as less than ideal with IPFS, so I recently set myself the challenge of designing a better system. I really like the way IPFS breaks down the problem, so the components remain the same. Regarding incentivization, looking at the literature, you can split existing incentive schemes into two camps, local and global. Tit-for-tat would be an example of a local scheme, currency based systems would be global. Neither of these types of scheme really work, with local schemes, you have no way to use accrued reputation with new nodes. Global schemes require too much coordination and overhead to be suitable for small 'transactions'. As I understand it, IPFS deals with the problem by having a local scheme, but with a get out clause. If you want something you may have to do work on behalf of the node you want something from, but that work should be easier if you have a good reputation with other nodes in the network. Under my proposed system, nodes keep track of how much they are in debt or in credit with the nodes that they are in contact with. The amount that you can go into debt with another node is limited to 1% of the amount of useful data that you have transferred to that node. (This way, if a node terminates a connection whilst it is in debt to another node, that loss, amortized over the useful data that you have received is very small). Nodes are incentivized keep connections open, since short lived connections have very tight reins on how much debt/credit can be accrued. As this stands, this doesn't solve the problem. It's entirely local and there's no way to initiate connections since nodes, by default, don't trust each other. The solution is two-fold: credit transfer and proof of work. I have designed a protocol that allows nodes to transfer credit to cancel out 'cycles of debt'. Nodes can also gain credit with other nodes by solving pointless problems. The innovation here is that when confronted with a puzzle, a node is able to delegate the work to its debtors. Proof of work is useful since, given the choice, a node would probably prefer to use its upstream bandwidth than burn up CPU time. There are more details to this, like: the specifics of how to discover and execute a credit transfer along a long chain using only local interactions, how to incentive routing as well as data transfer and many other things. If I had the time, I'd implement it.

deleted 11y ago

[deleted]

deleted 11y ago

[deleted]

chmike 11y ago

This is a naive idea that is wrong is so many way. It keeps popping up recurrently and I don't understand why people didn't see yet that it won't be the next Internet revolution. Oh well! Let me try to explain it again. The idea of content base key is strongly narrowing the application domain of the system. Modifying your information (I.e. typo correction) invalidates all the keys. More precisely, people holding the old key won't be able to access the corrected information. of course this model fails completely with dynamic data. The other problem that is often overlooked with such idea is the function that translate your key into the location of the information. That is: determine the IP address of the server(s) hosting the information from the hash key of it. One needs something like a distributed index for that. Don't use an algorithm because you don't want to add the constrain that your informations can't be moved or replicated. Another problem is staying in control of your information. An owner of information want to be able to fix or modify it, or delete it when he wants. Finally, another naive idea is sharing a distributed storage. This is a nice idea but it simply doesn't work. Some People will abuse it. To avoid this you need a system to monitor and control usage. Accounting ? good luck. By the way, this is the problem of Internet today. It is a shared resource, but a few are consuming a hell lot of it and earning a lot of money without sharing back the profit. I'm looking at you google with YouTube. I'm thinking and working on this problem for a long time. My conclusions are : 1. Decouple keys from content 2. Make the distributed locating index your infrastructure 3. Optimize keys for index traversal 4. Owner of information must stay in control of Information 5. Owner of information must assume the cost of hosting

david_ar 11y ago
I suggest you read more about IPFS (or even just other comments on this post), as many of the issues you raise have already been solved (1-3), or have planned solutions (4-5).
mirimir 11y ago
> 4. Owner of information must stay in control of Information Once you put something on the Internet, you no longer own it.
- 11y ago

mirimir 11y ago

I wonder if IPFS can access resources via Tor onion services.

david_ar 11y ago
https://github.com/ipfs/notes/issues/37 https://github.com/ipfs/notes/issues/37
- mirimir 11y ago
  Thanks. I'll test.

ilaksh 11y ago

You can tell this is a great idea by how much some people hate it.