10 ms·
The Internet needs the InterPlanetary File System
- 29athrowaway 4y ago
- thrown_22 4y agoTor is a great way to keep out lame people from a website.
- Jtsummers 4y agoWith IPFS, you don’t have to have any data on your computer you don’t want. No one uploads the data to unsuspecting strangers, they announce the content (it uses content addressing, hashes). Then other people choose to download it.
- aabhay 4y agoYou can’t possibly hand approve everything so I’m sure the vast majority of node runners auto approve stuff and perhaps share some (non decentralized) black lists.
- rklaehn 4y agoIf you run an open gateway, people can indeed request content via your gateway unless you have explicitly blacklisted it. But if you run just a node, everything that is on the node is there because either you have added it or you have explicitly requested it. IPFS is not like freenet or other systems that shard data over all participants.
- jstanley 4y agoNothing gets on to your computer that you don't ask for. I'm not sure how you are imagining IPFS works.
- cowtools 4y ago>You can’t possibly hand approve everything so I’m sure the vast majority of node runners auto approve stuff and perhaps share some (non decentralized) black lists. This obviously hasn't been the case for bittorrent, so I don't know why you would assume that would be the case for IPFS. Even ignoring the legal risk of blindly re-hosting whatever illegal bootleg-ISIS-child-porn-malware.exe on the net, the use of a blacklist isn't sybil proof so you just get bogged down with spam by the first attacker and run out of disk space.
- Gigachad 4y agoNo, IPFS works like torrents. You see the content you download.
- 29athrowaway 4y agoThen if each file has to be added manually I don't see how this scales.
- Gigachad 4y agoPopular files gain more hosts seeding. Rather than a page getting hard to access when its getting slammed with viewers, it should in theory become even faster. You don't add them manually, its just when you open up a link in your browser or video player, it gets stored for some amount of time and seeded before perhaps being pruned or kept forever depending on the users preference. You don't have any issues with unknowingly hosting illegal files this way.
- prhrb 4y agoAt least read the article. "But is there a dark side to decentralization? We often hear concerns about how peer-to-peer networks may be used by bad actors to support illegal activity. These concerns are important but sometimes overstated. One area where IPFS improves on HTTP is in allowing comprehensive auditing of stored data. For example, thanks to its content-addressing functionality and, in particular, to the use of unique and permanent content identifiers, IPFS makes it easier to determine whether certain content is present on the network, and which nodes are storing it. Moreover, IPFS makes it trivial for users to decide what content they distribute and what content they stop distributing (by merely deleting it from their machines)."
- chaxor 4y agoI don't understand this position. The main use case is for serving machine learning models and similar types of data. A big use case is for researchers who don't want to constantly spend money to try to maintain a link to something they did at an institution they are no longer a part of. Many times they don't even have a mechanism to do so - so the link rot is guaranteed because of centralized systems.
- kordlessagain 4y agoAs I get older, these downvoted posts become almost invisible and are VERY hard to read.
- ezoe 4y agoThe Internet may need P2P file sharing protocol, but that protocol never be IPFS. BitTorrent is the closest practical protocol currently used. IPFS is just a joke.
- easrng 4y agoIPFS is just ... fine. It more or less works, but it's not very good at what it does, and the only advantage it has over BitTorrent v2 is marketing and web gateways.
- woojoo666 4y agoThis feels way too unsubstantiated for HN. There isn't enough information to disagree with you, so these threads just become echo chambers
- noncoml 4y ago> IPFS is just a joke Care to explain why? I only know the basic idea of IPFS and sounded great to me and something I was looking forward to get stable and widely adopted. In which way is it a joke?
- yjftsjthsd-h 4y agoI think the biggest complaint is performance, which is reported by some users to be amazingly poor (to the point that such a user would call the whole thing a joke on that account). YMMV.
- threeseed 4y agoThat's because everyone has forgotten the lessons from BitTorrent and the internet. If you want something to be fast then your content needs to be hosted on servers with fast connectivity to other servers. And that's not possible when at the same time you're trying to avoid legal jurisdictions that will censor e.g. EU, US, CN etc.
- trasz 4y ago
- aksss 4y ago> To understand the alternative approach we’re championing, though, you first need to recall how the Internet works. Keep Reading ↓ Mm, no thanks.
- endisneigh 4y agoPeople want HA, p2p will never work. As soon as you introduce nodes or machines that serve to increase availability you might as well drop the p2p and make the system more efficient. p2p makes most sense for ephemeral transactions.
- trasz 4y agoPeople want reliability, and reliability of commercial solutions like Apple Music or YouTube is already crap. We’ve just learned not to notice.
- bluehatbrit 4y agoI suppose it really depends on what your definition of "reliable enough" is. I've not used apple music, but as a consumer I don't remember the last time I was impacted in any significant way by a YouTube issue. If we have a couple minutes downtime a year from those services, like we seem to right now, I don't really see it as a problem. There are services where I don't want them going down ever of course. But those are more critical infrastructure than something like Apple Music or YouTube.
- zcw100 4y agoIPFS Cluster https://ipfscluster.io https://ipfscluster.io
- joshxyz 4y agoi love p2p and torrents but ipfs really looks like a solution looking for a problem. i still havent seen a user friendly implementation of it usable by the masses. (yes if you know some feel free to comment it, of course id love to check it out)
- zolland 4y agoI believe LibGen uses IPFS as it's backend
- aabhay 4y agoNo I believe someone has ported the library to IPFS but it’s not reliant on it in any way.
- IceWreck 4y agoIt doesn't, thats just a mirror and even then everyone just uses the cloudflare or ipfs.io gateway instead of actually using IPFS. Dont get me wrong I like IPFS and would love to see increased usage but if a libgen mirror is the biggest example then we need to find more applications for it.
- trasz 4y agoLet’s make a libgen for music. And run it all on IPFS.
- shp0ngle 4y agoYou will start running into exactly the same problem as with normal hosting. You need someone to “pin” the music for you, or you pin it yourself but with public IP and will get sued. Unlike with torrents, most people will be using the “gateways” - most people are not “seeding” - partly because IPFS itself is resource hungry and hard to use, unlike Torrent which is easy to use. In reality IPFS world is very centralized, you just have this resource hungry pretend-decentralisation on top. I don’t know why is it like that when BitTorrent is much older and much more actually decentralised.
- theGnuMe 4y agoGreat idea! This is basically pied piper right?
- fabrice_d 4y agoThe most important part of IPFS is Content Based Addressing, not p2p. Content Based addressing provides a natural way for data to be portable, because it's not tied to a storage location or to a transport protocol.
- toomim 4y agoAnd works great until your content changes, and then the address changes, and you end up wanting to have a way to name and address your content across the changes, and then you aren't using content addressing anymore.
- fritolaid 4y agothe metadata can be versioned and you can lookup by version, so names of stored file can be changed but every time there is a change to any of the metadata the version is updated too
- betwixthewires 4y agoIt works great even in that case, at the fundamental level it makes sense for content to be accessed in such a manner and then indexed on too of that by name.
- geysersam 4y agoWell, to me those sounds like different use cases. Pointing to the current up do date content requires a domain anyway. The domain is the users guarantee that a trusted party has made the decision to update the content.
- mynameisvlad 4y agoAn end-user doesn't want to manually update a domain each time they update their site. That's practically a non-starter for the technology. Especially since domain updates require transactions (on a L1 or L2 chain) to complete. As mentioned by another person, IPNS does solve this issue, though. My domains are pointed to an IPFS+IPNS-published version of my site, which is re-published on change from Github.
- emptysongglass 4y agoI think IPFS fails at the most basic tenet of file storage which is file permanence. It's touted as a decentralized approach but then anyone whose anyone ends up paying a centralized service to "pin" their content. Arweave is leagues better: I use it all the time to store articles I've written and very precious media. It couldn't be easier to use with its browser extension. It's also very cheap for true file permanence forever at least until all the storage nodes die out but given replication stands at some absurd multiplier there's little chance of it.
- colesantiago 4y ago
- emptysongglass 4y ago
- Arainach 4y agoTheir site links to their wiki: https://arwiki.wiki/#/en/Arweave https://arwiki.wiki/#/en/Arweave This is how they describe their project: "Arweave is a protocol that allows you to store data permanently, sustainably, with a single upfront fee." This is economically impossible. That is the description of a pyramid scheme, a startup without a revenue model trying to make up for the fact that they're losing money on every customer by getting lots of customers, a business major who skipped math class, or all of the above. You can't support indefinite retention on fixed cost.
- emptysongglass 4y agoMiners are incentivized to store data via a storage endowment, which pays a principle up-front and accrues interest over time. [1] As cost of storage decreases faster than the interest, storage is secured. Read the yellow paper for a technical deep dive. [2] [1] https://www.arweave.org/technology#blockweaves https://www.arweave.org/technology#blockweaves [2] https://www.arweave.org/yellow-paper.pdf https://www.arweave.org/yellow-paper.pdf
- sergiotapia 4y agoI wish there was a version of IPFS that distributed popular files as the swarm demands. Any singular person doesn't have whole pieces and doesn't know what they are distributing. Also when you help the swarm you get a crypto token of some sort, giving people a financial incentive to help the swarm. So you would say I have 16TB, feel free to use it as you see fit swarm. I can "mine" crypto by helping the swarm. The swarm benefits from a gigantic pool of storage and content. ISPs and government agencies can't do anything because it's all anonymous and any individual person is sharing encrypted pieces of a file anyways.
- Scoundreller 4y agoFor me, costing my ISP more money is sufficient incentive to do something like this.
- beardog 4y agoMinus the cryptocurrency part, Freenet has been doing this for 22 years https://freenetproject.org/ https://freenetproject.org/
- jeswin 4y ago> ISPs and government agencies can't do anything because it's all anonymous and any individual person is sharing encrypted pieces of a file anyways. It would need to be over Tor or similar, which is of course way slower. Otherwise there's nothing preventing "ISPs and government agencies" from going after people sharing copyrighted bits.
- chaxor 4y agoPeople keep mentioning the financial incentive as either something to take issue with and combat the entire system, or as something to hold up as a feature which can save a project. Perhaps both have some point; however, I don't see it really necessary at all for a very useful and thriving project to benefit from something like IPFS. Case in point - DVC. You don't need financial incentive for people to store ML models / weights on their computers - they need to use them there anyway. It makes great sense that the more popular the model, the faster it should load from more nodes serving you, rather than waiting for things to be downloaded from the one server the authors put it on (that will probably be deleted in a year or so anyway).
- 1letterunixname 4y agoI love IPFS for some things. It's sort of not anonymous and it's nondeterministically slow in starting and keeping itself going. When it works, it's fantastic. I think an overlay fs on top of bittorrent (and adopting existing torrents) would've been more useful than reinventing the wheel and fragmenting. Torrents already have CBA and work. It also has the problem of not enough people use it.
- rklaehn 4y agoIt has some performance and reliability problems. Some of them are just a matter of writing a better implementation, some are more conceptual. But the basic principle of a content-addressed, distributed file system is desperately needed, and there is a lot of work going on behind the scenes from both protocol labs and others to fix these problems... Stay tuned :-)
- cowtools 4y agoI like the IPNS idea. That would be useful even if it was just back-ported to bittorrent
- claudedopplr 4y agoI like the "immutability" of IPFS. Back in 2017 I tried it out and [Dat](https://www.datprotocol.com/ https://www.datprotocol.com/). Both IPNS and Dat seemed to break often (not finding the correct peer, which then lead to broken downloads). With IPFS, you know "it's just a hash of the content". That idea of content addressing is pretty powerful, and I'm starting to see it a lot more, particularly in the NixOS ecosystem. IPNS, on the other hand, seldom worked. The 24hr timeout made it unusable without an abstraction on top that kept it up to date. Dat had a similar problem, one of storing way too much historical data, and I could never be sure if the peers had the same version I las published. I think these are "theoretical problems". Content addressing is a great idea and it's reliable. Distributed hash tables are on the weaker technology spectrum. I'm rooting for IPFS to replace bittorrent, not IPNS/Hypercore, which I think will have serious scalability/security /privacy problems.
- easrng 4y ago
- srcnkcl 4y agoI did try to use ipfs but not having a search capability (something like torrent) is such a let down for the end user. I dont think it will stick just for this reason
- betwixthewires 4y agoI would absolutely love an IPFS based version control system, something superficially (or more) resembling fossil would be amazing. I think it's probably one of the best applications for IPFS, since a lot of the core data structures and primatives required for version control are already a part of IPFS. A version control system built on top of a distributed content addressed filesystem, what could be better?
- oefrha 4y agoA version control system with atrocious object retrieval performance, and with which you have pin everything yourself (or pay someone to pin everything) anyway to guarantee that all your stuff aren’t lost? No thanks.
- betwixthewires 4y agoYou have to have files on your system anyway to work on or use the files. How is it any worse to have it pinned? And if everyone working on a project keeps a copy of the canonical branch there's no central place you'd need to look for the files, just query for the hashes of the files. Github and the like are quick fixes for the shortcomings of things like git and mailing lists, I think IPFS could help actually resolve those shortcomings.
- pabs3 4y agoDoes this mean that your data requests become public to anyone listening for data requests?
- detaro 4y agoyes
- brunoqc 4y agoI wish the ipfs devs would stop wasting time with crypto and nfts. I mean it's their lives but I like their work, except the silly crypto stuff.
- celdon25 4y agoHN people: Crypto has no use cases! Also HN people: Stop wasting time trying to make legitimate use cases for crypto!
- colesantiago 4y agoyes, both of these statements are true for crypto. I don’t see the issue here.
- Dylan16807 4y ago...sure? You disprove "no use cases" by finding one where a blockchain is a natural fit. Not by taking a standalone idea and grafting a blockchain onto it.
- TomSwirly 4y agoThe Blockchain is almost as old as the smart phone. By now, given that not one single application, aside from cryptocurrencies, has appeared, it's very reasonable to be skeptical. Heck, IPFS is over seven years old!
- up2isomorphism 4y agoIf these people spend nearly as much time writing software than writing these blog posts, it will be a much better thing.
- Invictus0 4y agoHow many times do we need to read on that "build it and they will come" doesn't work?
- deleted 4y ago[deleted]
- madsbuch 4y agoIPFS is a great case study in a system that has issues running entirely on voluntary basis. Now they try different "product models". One is paid pinning services - services that ensure the existence and availability of files in the network. Then there i FileCoin - an augmentation with a specialised economy that tried to have files stored an available based on a market.
- cowtools 4y agoWell then maybe it might be worthwhile to start bittorent-like private trackers for IPFS?
- madsbuch 4y agoBittorrent has the same problem: files go stale and disappear from the network. On the other hand, bittorrent would probably benefit from a FileCoin like economy build on top of it.
- shp0ngle 4y agoAs I wrote elsewhere in the chat - FileCoin is essentially unrelated to IPFS and it’s really hard to make it work together.
- fxtentacle 4y agoIt's an opinion piece by the filecoin people, not an actual IEEE publication. TLDR: people whose salary depends on selling IPFS think everyone needs IPFS
- Animats 4y agoTrue. If you could actually pay once to have a file stored forever, that would be useful. "Perpetual care" for data would be valuable. There are Filecoin people who will take your money to do that, but don't have the credibility. Now, if Berenberg Bank (private bankers since 1590 and still a substantial bank) managed the funding for this, it might be credible. One of the things they do is handle money on the scale of centuries.
- fxtentacle 4y agoActually, I believe that BitTorrent comes pretty close. The magnet links are just the SHA-256 hash of the content (in truncated). Anyone can link to such content and as long as at least one computer worldwide has the data stored and is online, you can still download it. Modern BitTorrent doesn't even need real trackers anymore, due to its distributed hash table of peers.
- TomSwirly 4y agoMany hash IDs that used to be in BitTorrent no longer are there. If you need certainty that your data is still there, BitTorrent does not deliver.
- leppr 4y agoIPFS doesn't solve permanent data availability either, the closest to "solving" this is Arweave[1] but that comes with problems of its own. It's fundamentally impossible to correctly price the availability of a "service" in the far future. [1]: https://www.arweave.org/technology https://www.arweave.org/technology
- AgentME 4y ago
- jeroenhd 4y agoI love the concept of IPFS but every time I try to use it I run into very practical problems. The IPFS server takes up tons of CPU time seemingly doing nothing. Requesting a file from any public gateway to test availability takes minutes even for small files. I like the design concept of the main software (a drag and drop sharing application that happens to enable IPFS rather than some kind of specific server/client kit) but there needs to be better server software for this to ever be popular. You need IPNS at the very least to have a website equivalent and updating that automatically requires some custom tooling already. Then there's the privacy problem. In the default settings, you can query what IP read what content. You can run IPFS over Tor but that requires command line knowledge at the very least. The docs briefly mention it but don't go into detail. The NFT boom brought more attention to IPFS and I don't think that's necessarily a good thing; I don't think the association with cryptocurrency scams and IPFS will help the protocol forward.
- DoItToMe81 4y agoI like IPFS a lot and I'm very interested in seeing how the technology develops, but I'll note that this is written by ProtocolLabs, the founder of IPFS.
- Animats 4y agoPeertube is based on a related, but perhaps more practical idea. In Peertube, someone has the master copy, and watching nodes also cache the content and redistribute it. It's not distributed storage, just distributed caching. You can post a viral video on a small site without being overwhelmed by traffic demands. It's proper streaming, not just file hosting; you can jump around within the video. Most importantly, no annoying ads. It does an OK job. I put technical videos and rendering tests on Peertube.[1] If you don't need "discovery" or "monetization", it's a good option. [1] https://video.hardlimit.com/w/sFPkECUxRUSxbKXRkCmjJK https://video.hardlimit.com/w/sFPkECUxRUSxbKXRkCmjJK
- dschuetz 4y agoSounds good. But first, switch to IPv6 and make all broadband Internet access symmetrical, and nationalize all ISPs.
- sokoloff 4y agoWhat about this is interplanetary? It’s intraplanetary or perhaps pan-planetary.