7 ms·
Performance Evaluation of IPFS in Private Networks
- chockchocschoir 5y agoI'm not sure anyone would be surprised that IPFS's performance would be worse than FTP or any other protocol when it comes to transfers between two (or three) hosts. The benefits of distributes systems becomes bigger the larger the network. Replicate the same results with 100 nodes and we'll see. I still think torrents would be even more efficient though, than both FTP and IPFS.
- skyfaller 5y agoIsn't Hypercore (and its predecessor Dat), essentially Bittorrent + version control? https://hypercore-protocol.org/ https://hypercore-protocol.org/
- chrisco255 5y agoIs it possible to create an IPFS-based torrent network?
- georgyo 5y agoIPFS has very similar features as BitTorrent, you can remove torrent from that sentence and it would mean the same. You may be interested in BTFS, which is elements of IPFS but using BitTorrent as the protocol.
- somat 5y agobasically, if you don't look too hard, ipfs is bittorrent improved. The structures were cleaned up and given a shave, the trackers removed in favor of just putting everyone on the dht.
- epr 5y agoNot worse, worser
- DeathArrow 5y ago>The benefits of distributes systems becomes bigger the larger the network. Replicate the same results with 100 nodes and we'll see. That's true. I would be interested in an experiment to show how many IPFS host would be needed to reach parity to FTP and other protocols. I'm also wondering if IPFS can't be integrated in CDNs.
- nine_k 5y agoSince IPFS's performance is worser than FTP's, a client-server architecture is, apparently, betterer! I wonder how representative of a real-world scenario a three-node IPFS network is. I thought that file fragments are strewn across more nodes typically, with some redundancy. FTP offers nothing like this. So far this looks a bit too apples-to-oranges to me. Has anyone actually read the paper?
- edgyquant 5y agoI don’t think anyone thought decentralized architecture was faster. There are other benefits people think outweighs the trade offs.
- nl 5y ago> I thought that file fragments are strewn across more nodes typically, with some redundancy. In real world use, this is rarely the case on IPFS. Almost everyone who is trying to make files available via IPFS reliably is pinning the files themselves using a server they operate.
- deleted 5y ago[deleted]
- nl 5y agohttps://www.semanticscholar.org/paper/Performance-Evaluation-of-IPFS-in-Private-Networks-Lajam-Helmy/e18310f0250786be1f573359ceb6c421711c76af https://www.semanticscholar.org/paper/Performance-Evaluation... has the graphs showing just how bad IPFS is. People should judge IPFS on this, not p2p file systems or DHTs generally. Bitorrtent for example would do a lot better on those large file tests.
- olodus 5y agoI have heard of IPFS's bad performance quite often and as you point out it is weird because I know that isn't specific to distributed storage in general. How much of this is implementation specific though? Would it be possible for there to be a IPFS implementation that solves most of these performance problems? Since the protocol itself is so similar to bittorrent it would surprise me if it is protocol inherent, but I am not sure.
- anon_d 5y agoI don't have a big-picture answer for you, but I'll point out that there are a lot of performance-relevant details that are different between the two protocols. A torrent is basically a filesystem, dumped into a single large bytestring, broken into constant size chunks. Then there is a fairly small swarm of nodes trading those chunks. Most of those nodes are trying to download everything, and want to saturate inbound/outbound pipes. - IPFS is a single massive DHT for advertising who has what for every single blob in the entire network. Most nodes have a tiny subset of that data. - IPFS does not use a constant block size. - IPFS uses a "multihash" instead of hard-coding a specific hash function. This adds parsing time to hashes, and means that hashes are not a consistent size, and they have sizes that don't fit cleanly into words. Those are just some details that I happen to know off the top of my head, but you start to see the picture. In every architectural trade-off, IPFS goes for the more general thing, instead of the performant thing.
- nl 5y ago> How much of this is implementation specific though? I don't know for sure, but have tried the Go and Node IPFS versions and both are pretty bad. The node version did seem less reliable though. I don't think this reflects on the language or runtimes - just how they are coded.
- JaggerFoo 5y agoI noticed that public ipfs is used to host image files for NFTs. NFT marketplace websites serve the images quickly enough. I wonder how perkeep would perform in that situation.
- Animats 5y agoWell, we're going to find out how well IPFS does under heavy load. This crypto startup plans to host streaming porn on IPFS.[1] [1] https://sugarbounce.com/ https://sugarbounce.com/
- sneak 5y agoipfs is a transport, not a host.
- encryptluks2 5y agoAs with most tokens, seems like a big scam built on hype of Web3 technologies.
- hecturchi 5y agoIPFS is already used to stream (porn and not porn), mostly Chinese apps with 50k+ users at least. Not the kind of projects that make noise on HN etc though.
- yawnxyz 5y agoI think most NFT marketplaces cache the images on a server to serve them faster, only pulling from the IPFS file if they have to
- makeworld 5y agoFTP is obviously an unfair comparison, but I was surprised to see this: > the reading performance of IPFS degrades as the data become more popular in the network. I'd have expected the opposite.
- bb88 5y agoIs it? What about mirrored FTP Servers which are distributed or using a cached filesystem sitting behind a DNS round robin record. That was a thing 3 decades ago. I guess the question is, what value does IPFS bring that couldn't be done with 30 year old tech to a private network?
- noman-land 5y agoI think one important thing it brings is one-click joining of ad-hoc nodes to the global network. Also a suite of utilities for concocting your own p2p applications though standardized APIs. But that's more libp2p.
- viraptor 5y agoAdding an IPFS node does not require a new DNS round robin record, or any other synchronisation with the rest of the infra. It allows adding single files or even parts of them to the storage transparently. Also it eliminates the question of how do you keep the FTP nodes synchronised.
- klabb3 5y agoSeems highly dependent on the protocol implementation. In either case, if cpu or memory bottlenecks it's pretty bad.
- olodus 5y agoYeah I have sometimes thought about trying to make a IPFS implementation myself to test this theory.
- yawnxyz 5y agoI'm considering using IPFS for storing metadata (in a similar way that DOI tracks publications) for accessioning genomics and wetlab microbiology data. The data will still be stored in the institution (b/c of data governance laws). The reason I really like that IPFS exists is bc it's a really convenient, mostly cheap and mostly reliable place to put stuff for quick access, even more than running my own server (ew), or paying for hosting and/or using Supabase, Firebase, etc. Basically using IPFS as a convenient k/v store. The data is still stored locally, but it's just much less convenient to access. Is anyone else doing this kind of setup?
- kfarr 5y agoTo be clear, and perhaps why you’re being downvoted, IPFS stores nothing. It only guarantees that a given file will result in a certain hash id and that perhaps if there are servers available with assets that match that hash they might serve it. Hence the existence of pinning services which are paid platforms that do the actual hosting.
- encryptluks2 5y ago> The reason I really like that IPFS exists is bc it's a really convenient, mostly cheap and mostly reliable place to put stuff for quick access, even more than running my own server (ew), or paying for hosting and/or using Supabase, Firebase, etc. I think you're confusing pinning services or something else and it is actually quite a bit more expensive than Google Drive and other sync and storage solutions. IPFS doesn't store anything. You could just as easily setup Syncthing and it would perform the same functionality it you just want to address your files elsewhere.
- kkielhofner 5y agoNo mention of node performance? Running your own IPFS node with any non-personal amount of traffic is ROUGH. Poor storage quota support combined with a terrible garbage collector that consumes 1000% CPU on my Epyc cores every time it runs…
- kevincox 5y agoYeah, I had to take go-ipfs off my home server because it would grind it to a crawl. Even with attempts to memory limit it is seems that somehow it would constantly thrashy disk cache even when basically idle. The think the core IPFS technology is great. But they really need to settle down with changing the higher level protocols and the go-ipfs implementation is truly a piece of junk.
- hecturchi 5y agoThe upcoming release of go-libp2p (0.18.0) includes a resource manager that promises to improve this by effectively enforcing limits. Also, please re-implement it properly!
- testesttest 5y ago2nd this. I saw lots of packet loss on my gig connection when scraping a few TBs over IPFS. Turned off garbage collection which just deletes the whole cache when you go over a threshold?
- Proven 5y ago
- Proven 5y ago
- jaequery 5y agoAnyone know what happened to Filecoin ? Thought it was the faster version of IPFS.