7 ms·

Willow solves the biggest problem I have always had with IPFS: it’s content addressable, which is nice for specific things but not generic enough to make the pr

Willow solves the biggest problem I have always had with IPFS: it’s content addressable, which is nice for specific things but not generic enough to make the protocol actually practical and usable in the real world outside of specific use cases. (Namely you can’t update resources or even track related updates to resources.) (Mathematically, a name-addressable system is actually a superset of content-addressable systems as you can always use the hash of the content as the name itself.)

nayuki 3y ago
> a name-addressable system is actually a superset of content-addressable systems as you can always use the hash of the content as the name itself It's a superset in that sense but not a superset in another sense. In a content-addressable system, if I post a link to another piece of content by hash, then no one can ever substitute a different piece of content. Like, if I reference the hash of a news article, no one can edit that article after the fact without being detected. This is a super-useful feature of CAS that is not a feature of NAS. Other implications: * I can review a piece of software, deem that it's not malware in my opinion, and link to the software by hash. No one can substitute a piece of malware without detection. * Suppose you get a link from a trusted source. Now you can download a copy of the underlying content from any untrusted source, without a care about authentication or trusted identities. This describes BitTorrent.
- ComputerGuru 3y ago
  It’s a protocol, not an implementation. If you say your implementation of the protocol says objects are defined by their hash, then your implementation can also assert that the hash of the retrieved file matches the hash you looked it up with^ (which you should do in your library/app regardless of what the protocol says will be returned, though ideally your protocol would define some sort of merkle tree or whatever to verify the hash piecemeal). ^ which you, by definition, already have
rklaehn 3y ago
To be fair, IPFS does offer not just content addressing but also a mechanism for mutability with IPNS. You can think of a willow namespace (or iroh document) as a key value store of IPNS entries. The problem with IPNS is that the performance is... not great... to put it politely, so it is not really an useful primitive to build mutability. You end up building your own thing using gossip, at which point you are not really getting a giant benefit anymore.
- unshavedyak 3y ago
  Is the performance a critical design flaw or just an implementation issue?
  - rklaehn 3y ago
    Difficult to answer. IPNS uses the IPFS kademlia DHT, which has some performance problems that you can argue are fundamental. For solving a similar problem with iroh, we would use the bittorrent mainline DHT, which is the largest DHT in existence and has stood the test of time - it still exists despite lots of powerful entities wanting it to go away. It also generally has very good performance and a very minimalist design. There is a rust crate to interact with the mainline DHT, https://crates.io/crates/mainline https://crates.io/crates/mainline , and a more high level idea to use DHTs as a kind of p2p DNS, https://github.com/nuhvi/pkarr https://github.com/nuhvi/pkarr
  - anacrolix 3y ago
    It's a design flaw.
  - wscott 3y ago
    Design flaw. In IPFS every piece of data (even every chuck of large files) is globally indexable on the same namespace. You need namespaces and/or a path to restrict yourself to just the subset of peers that might actually have the data. It would be possible to add a layer on top of IPFS to include some context with every hash lookup so the search can be more focused, but the original design just assumed it was ok to do a log2 search for every chunk.

nayuki 3y ago

> a name-addressable system is actually a superset of content-addressable systems as you can always use the hash of the content as the name itself It's a superset in that sense but not a superset in another sense. In a content-addressable system, if I post a link to another piece of content by hash, then no one can ever substitute a different piece of content. Like, if I reference the hash of a news article, no one can edit that article after the fact without being detected. This is a super-useful feature of CAS that is not a feature of NAS. Other implications: * I can review a piece of software, deem that it's not malware in my opinion, and link to the software by hash. No one can substitute a piece of malware without detection. * Suppose you get a link from a trusted source. Now you can download a copy of the underlying content from any untrusted source, without a care about authentication or trusted identities. This describes BitTorrent.

ComputerGuru 3y ago
It’s a protocol, not an implementation. If you say your implementation of the protocol says objects are defined by their hash, then your implementation can also assert that the hash of the retrieved file matches the hash you looked it up with^ (which you should do in your library/app regardless of what the protocol says will be returned, though ideally your protocol would define some sort of merkle tree or whatever to verify the hash piecemeal). ^ which you, by definition, already have

rklaehn 3y ago

To be fair, IPFS does offer not just content addressing but also a mechanism for mutability with IPNS. You can think of a willow namespace (or iroh document) as a key value store of IPNS entries. The problem with IPNS is that the performance is... not great... to put it politely, so it is not really an useful primitive to build mutability. You end up building your own thing using gossip, at which point you are not really getting a giant benefit anymore.

unshavedyak 3y ago
Is the performance a critical design flaw or just an implementation issue?
- rklaehn 3y ago
  Difficult to answer. IPNS uses the IPFS kademlia DHT, which has some performance problems that you can argue are fundamental. For solving a similar problem with iroh, we would use the bittorrent mainline DHT, which is the largest DHT in existence and has stood the test of time - it still exists despite lots of powerful entities wanting it to go away. It also generally has very good performance and a very minimalist design. There is a rust crate to interact with the mainline DHT, https://crates.io/crates/mainline https://crates.io/crates/mainline , and a more high level idea to use DHTs as a kind of p2p DNS, https://github.com/nuhvi/pkarr https://github.com/nuhvi/pkarr
- anacrolix 3y ago
  It's a design flaw.
- wscott 3y ago
  Design flaw. In IPFS every piece of data (even every chuck of large files) is globally indexable on the same namespace. You need namespaces and/or a path to restrict yourself to just the subset of peers that might actually have the data. It would be possible to add a layer on top of IPFS to include some context with every hash lookup so the search can be more focused, but the original design just assumed it was ok to do a log2 search for every chunk.