4 ms·

I have heard of IPFS's bad performance quite often and as you point out it is weird because I know that isn't specific to distributed storage in general. How mu

by olodus 5y ago

I have heard of IPFS's bad performance quite often and as you point out it is weird because I know that isn't specific to distributed storage in general. How much of this is implementation specific though? Would it be possible for there to be a IPFS implementation that solves most of these performance problems? Since the protocol itself is so similar to bittorrent it would surprise me if it is protocol inherent, but I am not sure.

anon_d 5y ago
I don't have a big-picture answer for you, but I'll point out that there are a lot of performance-relevant details that are different between the two protocols. A torrent is basically a filesystem, dumped into a single large bytestring, broken into constant size chunks. Then there is a fairly small swarm of nodes trading those chunks. Most of those nodes are trying to download everything, and want to saturate inbound/outbound pipes. - IPFS is a single massive DHT for advertising who has what for every single blob in the entire network. Most nodes have a tiny subset of that data. - IPFS does not use a constant block size. - IPFS uses a "multihash" instead of hard-coding a specific hash function. This adds parsing time to hashes, and means that hashes are not a consistent size, and they have sizes that don't fit cleanly into words. Those are just some details that I happen to know off the top of my head, but you start to see the picture. In every architectural trade-off, IPFS goes for the more general thing, instead of the performant thing.
nl 5y ago
> How much of this is implementation specific though? I don't know for sure, but have tried the Go and Node IPFS versions and both are pretty bad. The node version did seem less reliable though. I don't think this reflects on the language or runtimes - just how they are coded.

4 ms·

I have heard of IPFS's bad performance quite often and as you point out it is weird because I know that isn't specific to distributed storage in general. How mu

by olodus 5y ago

anon_d 5y ago
I don't have a big-picture answer for you, but I'll point out that there are a lot of performance-relevant details that are different between the two protocols. A torrent is basically a filesystem, dumped into a single large bytestring, broken into constant size chunks. Then there is a fairly small swarm of nodes trading those chunks. Most of those nodes are trying to download everything, and want to saturate inbound/outbound pipes. - IPFS is a single massive DHT for advertising who has what for every single blob in the entire network. Most nodes have a tiny subset of that data. - IPFS does not use a constant block size. - IPFS uses a "multihash" instead of hard-coding a specific hash function. This adds parsing time to hashes, and means that hashes are not a consistent size, and they have sizes that don't fit cleanly into words. Those are just some details that I happen to know off the top of my head, but you start to see the picture. In every architectural trade-off, IPFS goes for the more general thing, instead of the performant thing.
nl 5y ago
> How much of this is implementation specific though? I don't know for sure, but have tried the Go and Node IPFS versions and both are pretty bad. The node version did seem less reliable though. I don't think this reflects on the language or runtimes - just how they are coded.