3 ms·

A Hypercore P2P innovation could bring more privacy to IPFS

32 points by pimterry 5y ago

cassonmars 5y ago
This doesn’t sound like it preserves privacy at all — hashes of interest will still be catalogued and monitored.
- panick21_ 5y ago
  Don't let the perfect be the enemy of the good.
native_samples 5y ago
Hypercore/DAT is a really nice bit of tech for anyone who hasn't seen it. It's the basis for Beaker Browser. Or at least an older version is.
antihero 5y ago
Is there basically any guarantee that if I use IPFS, my computer wont be used to store and distribute CSAM?
- mrtesthah 5y ago
  I believe your computer only caches/pins content that you've downloaded yourself.
  - antihero 5y ago
    In that case, how does it distribute the data?
    - detaro 5y ago
      by people caching or pinning content they want to help distribute.
      - antihero 5y ago
        I see, so a bit more like seeding?
        detaro 5y ago
        yes, very similar.
- creatonez 5y ago
  You're thinking of Filecoin and similar, not IPFS. IPFS only downloads what you tell it.
  - antihero 5y ago
    Ah, thanks. This makes much more sense, and might check it out now!
dividuum 5y ago
I use the exact same mechanism for four years now to locally distribute content among digital signage devices. The advantage of that scheme is that pairs that exactly know (by hash) which content they want to retrieve can do so from other pairs that already have the content locally available. All without revealing anything about the content to pairs that don't know the hash. I'm sure the same mechansim has been used before elsewhere?
kevincox 5y ago
If I want to discover the content hash can't I just advertise that I have the discovery hash then wait for someone to ask me for it? I think one approach here would be to encrypted the data with the content hash. Then you ask for the data by discovery hash, but if you don't actually know the content hash it is useless to you. This way the content hash is still critical to getting access to the content but never sent over the wire. I wrote a proposal for doing something similar to this encryption inside the UnixFS protocol that is usually used for serving websites across gateways but apparently there was no interest and they wanted to deprecate that protocol.
- nybble41 5y ago
  > If I want to discover the content hash can't I just advertise that I have the discovery hash then wait for someone to ask me for it? Yes, for this to work it is essential to avoid sending the content ID to a peer who may not already have it. > I think one approach here would be to encrypted the data with the content hash. Then you ask for the data by discovery hash, but if you don't actually know the content hash it is useless to you. That is how Hypercore/Dat works, except it uses a public key rather than a content hash. The request only includes the discovery hash, and the peer has to look up the corresponding content ID to encrypt the response.
nybble41 5y ago
> To download a file over Hypercore, you need to know the content hash. Without the original content hash, you can’t generate the discovery hash, and you won’t know which hash to ask other clients to transfer. Once you’ve found some peers using the discovery hash, you establish an encrypted connection to them and request the content hash. Hypercore (Dat) doesn't use content hashes, it uses ed25519 public keys (which work more like IPNS—the data they refer to can be changed). The discovery hash is a hash of the public key, which isn't sent to the peer as part of the request, it's looked up on the data provider's side using the discovery hash. If the provider doesn't already know the public key then it can't negotiate a successful connection since all subsequent messages are supposed to be encrypted with that key. If the public key (or IPFS content ID) were sent to the peer as part of the request then one could learn the content IDs for "interesting" discovery hashes simply by pretending to have the content, announcing the discovery hash to the network, and waiting for peers to show up and request the data by its content ID. At that point you could request the full content from other peers using the content ID. One could perhaps employ the Hypercore/Dat protocol for IPFS using the content ID for encryption instead of a public key, but it would be critical to avoid revealing the content ID itself as part of the request. Only the discovery hash can be sent to the peer. It might be safer to negotiate a session key with something like SRP[0], with the content ID as the "password". [0] https://en.wikipedia.org/wiki/Secure_Remote_Password_protocol https://en.wikipedia.org/wiki/Secure_Remote_Password_protoco...
vivegi 5y ago
I implemented a slightly different method for my prototype multimedia chat application (text/images/audio/video) over IPFS. I used an envelope format for permissioned encrypted content: { perms: 'ipns name of permission list' payload: [encrypted payload] } This ensured that the content hash of the envelope remains the same even when peers are added/removed from the perms list. The permission list had entries of the form: { salt: [a random salt], peer_pubkey_hash: [salted hash of the public key of the permitted peer], wrapped_enc_key: [payload private key that is encrypted using the peer's private key] } The content hash shared is that of the encrypted envelope and a permitted peer will get the perm list, lookup its entry using the peer_pubkey_hash and unwrap the wrapped_enc_key to obtain the encryption key that is used to decrypt the payload. I am oversimplifying, but that was the general idea.

32 points by pimterry 5y ago

cassonmars 5y ago

This doesn’t sound like it preserves privacy at all — hashes of interest will still be catalogued and monitored.

panick21_ 5y ago
Don't let the perfect be the enemy of the good.

native_samples 5y ago

Hypercore/DAT is a really nice bit of tech for anyone who hasn't seen it. It's the basis for Beaker Browser. Or at least an older version is.

antihero 5y ago

Is there basically any guarantee that if I use IPFS, my computer wont be used to store and distribute CSAM?

mrtesthah 5y ago
I believe your computer only caches/pins content that you've downloaded yourself.
- antihero 5y ago
  In that case, how does it distribute the data?
  - detaro 5y ago
    by people caching or pinning content they want to help distribute.
    - antihero 5y ago
      I see, so a bit more like seeding?
      - detaro 5y ago
        yes, very similar.
creatonez 5y ago
You're thinking of Filecoin and similar, not IPFS. IPFS only downloads what you tell it.
- antihero 5y ago
  Ah, thanks. This makes much more sense, and might check it out now!

dividuum 5y ago

I use the exact same mechanism for four years now to locally distribute content among digital signage devices. The advantage of that scheme is that pairs that exactly know (by hash) which content they want to retrieve can do so from other pairs that already have the content locally available. All without revealing anything about the content to pairs that don't know the hash. I'm sure the same mechansim has been used before elsewhere?

kevincox 5y ago

If I want to discover the content hash can't I just advertise that I have the discovery hash then wait for someone to ask me for it? I think one approach here would be to encrypted the data with the content hash. Then you ask for the data by discovery hash, but if you don't actually know the content hash it is useless to you. This way the content hash is still critical to getting access to the content but never sent over the wire. I wrote a proposal for doing something similar to this encryption inside the UnixFS protocol that is usually used for serving websites across gateways but apparently there was no interest and they wanted to deprecate that protocol.

nybble41 5y ago
> If I want to discover the content hash can't I just advertise that I have the discovery hash then wait for someone to ask me for it? Yes, for this to work it is essential to avoid sending the content ID to a peer who may not already have it. > I think one approach here would be to encrypted the data with the content hash. Then you ask for the data by discovery hash, but if you don't actually know the content hash it is useless to you. That is how Hypercore/Dat works, except it uses a public key rather than a content hash. The request only includes the discovery hash, and the peer has to look up the corresponding content ID to encrypt the response.

nybble41 5y ago

> To download a file over Hypercore, you need to know the content hash. Without the original content hash, you can’t generate the discovery hash, and you won’t know which hash to ask other clients to transfer. Once you’ve found some peers using the discovery hash, you establish an encrypted connection to them and request the content hash. Hypercore (Dat) doesn't use content hashes, it uses ed25519 public keys (which work more like IPNS—the data they refer to can be changed). The discovery hash is a hash of the public key, which isn't sent to the peer as part of the request, it's looked up on the data provider's side using the discovery hash. If the provider doesn't already know the public key then it can't negotiate a successful connection since all subsequent messages are supposed to be encrypted with that key. If the public key (or IPFS content ID) were sent to the peer as part of the request then one could learn the content IDs for "interesting" discovery hashes simply by pretending to have the content, announcing the discovery hash to the network, and waiting for peers to show up and request the data by its content ID. At that point you could request the full content from other peers using the content ID. One could perhaps employ the Hypercore/Dat protocol for IPFS using the content ID for encryption instead of a public key, but it would be critical to avoid revealing the content ID itself as part of the request. Only the discovery hash can be sent to the peer. It might be safer to negotiate a session key with something like SRP[0], with the content ID as the "password". [0] https://en.wikipedia.org/wiki/Secure_Remote_Password_protocol https://en.wikipedia.org/wiki/Secure_Remote_Password_protoco...

vivegi 5y ago

I implemented a slightly different method for my prototype multimedia chat application (text/images/audio/video) over IPFS. I used an envelope format for permissioned encrypted content: { perms: 'ipns name of permission list' payload: [encrypted payload] } This ensured that the content hash of the envelope remains the same even when peers are added/removed from the perms list. The permission list had entries of the form: { salt: [a random salt], peer_pubkey_hash: [salted hash of the public key of the permitted peer], wrapped_enc_key: [payload private key that is encrypted using the peer's private key] } The content hash shared is that of the encrypted envelope and a permitted peer will get the perm list, lookup its entry using the peer_pubkey_hash and unwrap the wrapped_enc_key to obtain the encryption key that is used to decrypt the payload. I am oversimplifying, but that was the general idea.