6 ms·

What I don't understand about Filecoin is that they advertise it is for "Private data" too, but there are no access control mechanisms and no ability to delete

by ahnick 6y ago

What I don't understand about Filecoin is that they advertise it is for "Private data" too, but there are no access control mechanisms and no ability to delete data from the network. (You can "unpin" it, but there's no mechanism in the clients to compel nodes to actively remove content afaik) If I put private data on the Filecoin network, I would like to be able to largely prevent most people from even pulling down the encrypted files. What happens when the encryption guarding those files is found to contain a flaw and gets compromised? If someone has an old copy of the encrypted files, then that data can now be unencrypted. These issues seem to go largely ignored by Filecoin as best as I can tell.

jluxenberg 6y ago
I'm not sure how you would implement this, since there's no way to provide a proof of deletion. Proof of storage is possible since you can do some operation on the stored data and then provide the result. However, I assume when a file is "unpinned" in IPFS you stop earning Filecoin for storing it? I'm not sure but I can't imagine why it would be any other way.
- ahnick 6y ago
  So I have read this is why they didn't implement deletion support before. With that said, I would still think that 1) It should be possible to guarantee that a particular object is not being actively served by a node on the network at any particular point in time and 2) If all the node software supports deletion, then even if you couldn't provide a 100% guarantee at least it would provide some mitigation.
  - derefr 6y ago
    Keep in mind also that Filecoin's storage layer is IPFS; and IPFS nodes don't have incentives that are perfectly aligned with those of Filecoin. There's maybe a reason for the owner of a "Filecoin node" to want to delete data "on the network." But that data ends up stored on many IPFS nodes, some (many?) of those hosted by people who've never even heard of Filecoin, but are just running on the public IPFS network for other reasons (usually the same reasons someone would donate to the Internet Archive, or host a public-access Debian APT mirror.) When you look up a file "on Filecoin", you're really looking it up from the IPFS network. Filecoin just incentivizes the insertion process. But data inserted "into Filecoin" is now on IPFS. So you have to think about what an IPFS node would (want to) do with your data. Why would these "pure" IPFS nodes that have your data, care about deleting it in response to requests? That's not what IPFS is "for." These nodes aren't participating in the filesystem-like storage paradigm that Filecoin represents. They're participating in a "content-addressable storage" paradigm. Deletion would be a violation of the ideology that likely motivates them to run their node.
    - ahnick 6y ago
      Yes, I understand that. I have this same issue with using just pure IPFS to store private data. IPFS for public data makes a lot of sense to me. When using IPFS for private data there seems like there is an inherent risk (hopefully only a small risk) that the encryption or its implementation can be broken at some point in the future potentially exposing the content that is hosted on the network. Normally, in this scenario the response is just run your own network of IPFS nodes if you want to restrict access, which is totally reasonable. I'm just expressing my desire to have something like Filecoin, but at the network level have the clients enforce access permissions and honor deletes. I may be the only one though. :)
      - derefr 6y ago
        I think the thing is that, because, as you say, there’s no true “Proof of Deletion” possible, then people who are considering alternatives when implementing a system that has private data, and needs a storage layer, just won’t choose IPFS/Filecoin, since it can’t offer them the same guarantees that e.g. using S3 can. Using S3 at least means you’re in a contractual relationship with Amazon, who thus has a monetary incentive from all its customers to “do what it says on the tin”, lest they all lose faith in its claims and boycott/switch to some other object-storage provider. Amazon can do this, because it's a single coherent closed-membership hierarchical organization, rather than an open-membership p2p network which people could join for reasons at odds with the network's "designed" incentives. Without a "Proof of Deletion", adding a deletion-request system to Filecoin would only be a mitigation against ciphertext-recovery at best—something that increases the probability that your file will be fully erased from the network—rather than a true guarantee that the entire system will attempt to achieve an explicit consensus state where the ciphertext has been purged from it. So, if you're a system architect, why would you choose to store private data on a service (Filecoin) that can only offer a high probability of erasure, rather than one (S3) that can offer a guarantee of non-recovery? And if it's clear that there's no reason to make that decision in favour of Filecoin, then turn that around: why, as the Filecoin or IPFS node-software authors, would you bother to add this feature, if it would be the game-theoretic dominant strategy for software architects that have this use-case to ignore Filecoin/IPFS in favor of some other solution, with or without the added feature? ----- All that being said, I'm ignoring a distinction here that's fairly important: there are two types of private data — timely and timeless private data. • Timeless private data would have just as much value if it was obtained years from now, as it would have if it were obtained today. (Someone's Bitcoin-wallet private keys, for example. Or compromising photos of a politician.) • Timely private data has high value to its owner upon creation, but that value erodes over time; until, at some point, even the creator themselves doesn't much care if it gets leaked. (Apple's next iPhone design, for example. Or the source code to Super Mario World. Or military intelligence.) There's actually far more timely private data in the world than timeless private data. And encryption, all by itself, basically solves the protection needs of timely private data. Cryptanalysis seems to only really advance in power alongside Moore's law; so anything encrypted using modern encryption technology, won't be brute-forced for a number of years yet. So you can take your timely private data, encrypt it, and stick a printout of the base64 ciphertext in the public square, if you like. By the time anyone can decrypt it, there'll be no value in doing so. Filecoin/IPFS works just fine to hold timely private data. And since that's most private data, it actually supports the needs of most system architects just fine. It's quite the rare use-case that requires decentralized storage of timeless private data. The fact that this type of data is so rare, though, is another strike against IPFS or Filecoin node-software developers being interested in introducing a feature specifically meant to help that use-case. Filecoin/IPFS would still be a non-dominant strategy for the timeless-private-data case, so there'd still be no point; but on top of that, it's a niche-within-a-niche.
gruez 6y ago
> If I put private data on the Filecoin network, I would like to be able to largely prevent most people from even pulling down the encrypted files. Isn't that fundamentally impossible? There's nothing preventing someone from setting up an archive site dedicated to storing/serving deleted content.
- ahnick 6y ago
  I think you might not be understanding what I mean. I'd like to be able to use Filecoin as a backend storage that will only serve up data to people in a particular group. Take the example of a traditional website or web application like Facebook. When I post content I can select who will be able to access that content and there is no way for anyone NOT in the groups I select to pull down that data even in an encrypted form. If Filecoin operated this way, then there would be no way for anyone to archive the data, because they could not access it in the first place.
  - gruez 6y ago
    Right but filecoin isn't a centralized service, it's a decentralized network. Just like how ultimately your facebook post is stored on a server somewhere, stuff you stored using filecoin is stored on a random computer somewhere. At least with facebook there's some central entity that gets to decide where their servers are located and who gets access to them. You don't get that with a decentralized network.
    - ahnick 6y ago
      Just because the storage network is decentralized (i.e. I don't get to choose where the files are hosted) doesn't mean that the nodes operating on that network can't be designed to honor a deletion request. For example, Tardigrade (https://tardigrade.io https://tardigrade.io) is an decentralized network that provides an S3-compatible storage layer and also provides a removal command for the client. (https://documentation.tardigrade.io/api-reference/uplink-cli/rm-command https://documentation.tardigrade.io/api-reference/uplink-cli...)
neckardt 6y ago
> If I put private data on the Filecoin network, I would like to be able to largely prevent most people from even pulling down the encrypted files. Why do you want this? The files are encrypted and thus illegible. To anyone without the key they look like random noise.
- ahnick 6y ago
  The same reason people set permissions on their encrypted S3 object blob or you have to login to download any encrypted file from a multitude of applications. As computing power increases and/or as flaws in encryption implementations are discovered, then what is encrypted today might not stay encrypted decades from now.
  - neckardt 6y ago
    What sort of adversary do you imagine would carry out an attack like that? If you're defending against your own government, they already control the network and could hold on to your S3 network traffic until they can decrypt it. If you're defending against an individual how would they know where to download your file from? I assume the network doesn't broadcast a connection between real life identity and the data you're storing. And if the individual can monitor your physical network, then they can again decrypt the network traffic eventually. If you're defending against the server hosting your content, they would have to host your old data for years past you stopping the service before they could decrypt the content, which would be a massive investment. Finally, do you have reason to believe that our current encryption algorithms will be cracked anytime soon? The last big innovation I heard was SHA-1 generating a hash collision in 2017, but even that took thousands of dollars in compute resources, and SHA-1 was known to be defective for a long time before that. I imagine if we use our best algorithms they'll hold up for quite a while longer.
    - ahnick 6y ago
      I assume the network doesn't broadcast a connection between real life identity and the data you're storing. So my understanding is that in the example of a decentralized app that uses Filecoin to store an IPFS object on the network the IPFS object id could be sniffed on the localhost by something like Wireshark. This way an attacker could identify the encrypted objects that are of interest to them and then independently request them from the IPFS network. In terms of threat, I'm primarily thinking of larger criminal organizations that might target things like files containing SSNs, credit history, etc. SHA-1 is an example, but my biggest concern comes from the potential of quantum computing to render what we consider secure today obsolete. (https://www.technologyreview.com/2019/05/30/65724/how-a-quantum-computer-could-break-2048-bit-rsa-encryption-in-8-hours/ https://www.technologyreview.com/2019/05/30/65724/how-a-quan...)
sneak 6y ago
This is not a use case anyone serious cares about. Data retention for ciphertext where the keys, stored and distributed elsewhere, have been deleted/zeroized, is a complete non-issue.
- ahnick 6y ago
  Why is that? I think it is entirely possible that in the future, encryption techniques used today will become compromised as both computing power improves and/or flaws are discovered. If someone is holding ciphertext encrypted with a compromised algorithm, then it could be decrypted. Basically, how do you know what we are encrypting today, won't be able to have the encryption broken 30 or 50 years from now?
  - folmar 6y ago
    You don't, but offline attacks tend to be relatively hard. 3DES is 25 years old now and not an easy target for a casual attacker.

by ahnick 6y ago

jluxenberg 6y ago

I'm not sure how you would implement this, since there's no way to provide a proof of deletion. Proof of storage is possible since you can do some operation on the stored data and then provide the result. However, I assume when a file is "unpinned" in IPFS you stop earning Filecoin for storing it? I'm not sure but I can't imagine why it would be any other way.

ahnick 6y ago
So I have read this is why they didn't implement deletion support before. With that said, I would still think that 1) It should be possible to guarantee that a particular object is not being actively served by a node on the network at any particular point in time and 2) If all the node software supports deletion, then even if you couldn't provide a 100% guarantee at least it would provide some mitigation.
- derefr 6y ago
  Keep in mind also that Filecoin's storage layer is IPFS; and IPFS nodes don't have incentives that are perfectly aligned with those of Filecoin. There's maybe a reason for the owner of a "Filecoin node" to want to delete data "on the network." But that data ends up stored on many IPFS nodes, some (many?) of those hosted by people who've never even heard of Filecoin, but are just running on the public IPFS network for other reasons (usually the same reasons someone would donate to the Internet Archive, or host a public-access Debian APT mirror.) When you look up a file "on Filecoin", you're really looking it up from the IPFS network. Filecoin just incentivizes the insertion process. But data inserted "into Filecoin" is now on IPFS. So you have to think about what an IPFS node would (want to) do with your data. Why would these "pure" IPFS nodes that have your data, care about deleting it in response to requests? That's not what IPFS is "for." These nodes aren't participating in the filesystem-like storage paradigm that Filecoin represents. They're participating in a "content-addressable storage" paradigm. Deletion would be a violation of the ideology that likely motivates them to run their node.
  - ahnick 6y ago
    Yes, I understand that. I have this same issue with using just pure IPFS to store private data. IPFS for public data makes a lot of sense to me. When using IPFS for private data there seems like there is an inherent risk (hopefully only a small risk) that the encryption or its implementation can be broken at some point in the future potentially exposing the content that is hosted on the network. Normally, in this scenario the response is just run your own network of IPFS nodes if you want to restrict access, which is totally reasonable. I'm just expressing my desire to have something like Filecoin, but at the network level have the clients enforce access permissions and honor deletes. I may be the only one though. :)
    - derefr 6y ago
      I think the thing is that, because, as you say, there’s no true “Proof of Deletion” possible, then people who are considering alternatives when implementing a system that has private data, and needs a storage layer, just won’t choose IPFS/Filecoin, since it can’t offer them the same guarantees that e.g. using S3 can. Using S3 at least means you’re in a contractual relationship with Amazon, who thus has a monetary incentive from all its customers to “do what it says on the tin”, lest they all lose faith in its claims and boycott/switch to some other object-storage provider. Amazon can do this, because it's a single coherent closed-membership hierarchical organization, rather than an open-membership p2p network which people could join for reasons at odds with the network's "designed" incentives. Without a "Proof of Deletion", adding a deletion-request system to Filecoin would only be a mitigation against ciphertext-recovery at best—something that increases the probability that your file will be fully erased from the network—rather than a true guarantee that the entire system will attempt to achieve an explicit consensus state where the ciphertext has been purged from it. So, if you're a system architect, why would you choose to store private data on a service (Filecoin) that can only offer a high probability of erasure, rather than one (S3) that can offer a guarantee of non-recovery? And if it's clear that there's no reason to make that decision in favour of Filecoin, then turn that around: why, as the Filecoin or IPFS node-software authors, would you bother to add this feature, if it would be the game-theoretic dominant strategy for software architects that have this use-case to ignore Filecoin/IPFS in favor of some other solution, with or without the added feature? ----- All that being said, I'm ignoring a distinction here that's fairly important: there are two types of private data — timely and timeless private data. • Timeless private data would have just as much value if it was obtained years from now, as it would have if it were obtained today. (Someone's Bitcoin-wallet private keys, for example. Or compromising photos of a politician.) • Timely private data has high value to its owner upon creation, but that value erodes over time; until, at some point, even the creator themselves doesn't much care if it gets leaked. (Apple's next iPhone design, for example. Or the source code to Super Mario World. Or military intelligence.) There's actually far more timely private data in the world than timeless private data. And encryption, all by itself, basically solves the protection needs of timely private data. Cryptanalysis seems to only really advance in power alongside Moore's law; so anything encrypted using modern encryption technology, won't be brute-forced for a number of years yet. So you can take your timely private data, encrypt it, and stick a printout of the base64 ciphertext in the public square, if you like. By the time anyone can decrypt it, there'll be no value in doing so. Filecoin/IPFS works just fine to hold timely private data. And since that's most private data, it actually supports the needs of most system architects just fine. It's quite the rare use-case that requires decentralized storage of timeless private data. The fact that this type of data is so rare, though, is another strike against IPFS or Filecoin node-software developers being interested in introducing a feature specifically meant to help that use-case. Filecoin/IPFS would still be a non-dominant strategy for the timeless-private-data case, so there'd still be no point; but on top of that, it's a niche-within-a-niche.

gruez 6y ago

> If I put private data on the Filecoin network, I would like to be able to largely prevent most people from even pulling down the encrypted files. Isn't that fundamentally impossible? There's nothing preventing someone from setting up an archive site dedicated to storing/serving deleted content.

ahnick 6y ago
I think you might not be understanding what I mean. I'd like to be able to use Filecoin as a backend storage that will only serve up data to people in a particular group. Take the example of a traditional website or web application like Facebook. When I post content I can select who will be able to access that content and there is no way for anyone NOT in the groups I select to pull down that data even in an encrypted form. If Filecoin operated this way, then there would be no way for anyone to archive the data, because they could not access it in the first place.
- gruez 6y ago
  Right but filecoin isn't a centralized service, it's a decentralized network. Just like how ultimately your facebook post is stored on a server somewhere, stuff you stored using filecoin is stored on a random computer somewhere. At least with facebook there's some central entity that gets to decide where their servers are located and who gets access to them. You don't get that with a decentralized network.
  - ahnick 6y ago
    Just because the storage network is decentralized (i.e. I don't get to choose where the files are hosted) doesn't mean that the nodes operating on that network can't be designed to honor a deletion request. For example, Tardigrade (https://tardigrade.io https://tardigrade.io) is an decentralized network that provides an S3-compatible storage layer and also provides a removal command for the client. (https://documentation.tardigrade.io/api-reference/uplink-cli/rm-command https://documentation.tardigrade.io/api-reference/uplink-cli...)

neckardt 6y ago

> If I put private data on the Filecoin network, I would like to be able to largely prevent most people from even pulling down the encrypted files. Why do you want this? The files are encrypted and thus illegible. To anyone without the key they look like random noise.

ahnick 6y ago
The same reason people set permissions on their encrypted S3 object blob or you have to login to download any encrypted file from a multitude of applications. As computing power increases and/or as flaws in encryption implementations are discovered, then what is encrypted today might not stay encrypted decades from now.
- neckardt 6y ago
  What sort of adversary do you imagine would carry out an attack like that? If you're defending against your own government, they already control the network and could hold on to your S3 network traffic until they can decrypt it. If you're defending against an individual how would they know where to download your file from? I assume the network doesn't broadcast a connection between real life identity and the data you're storing. And if the individual can monitor your physical network, then they can again decrypt the network traffic eventually. If you're defending against the server hosting your content, they would have to host your old data for years past you stopping the service before they could decrypt the content, which would be a massive investment. Finally, do you have reason to believe that our current encryption algorithms will be cracked anytime soon? The last big innovation I heard was SHA-1 generating a hash collision in 2017, but even that took thousands of dollars in compute resources, and SHA-1 was known to be defective for a long time before that. I imagine if we use our best algorithms they'll hold up for quite a while longer.
  - ahnick 6y ago
    I assume the network doesn't broadcast a connection between real life identity and the data you're storing. So my understanding is that in the example of a decentralized app that uses Filecoin to store an IPFS object on the network the IPFS object id could be sniffed on the localhost by something like Wireshark. This way an attacker could identify the encrypted objects that are of interest to them and then independently request them from the IPFS network. In terms of threat, I'm primarily thinking of larger criminal organizations that might target things like files containing SSNs, credit history, etc. SHA-1 is an example, but my biggest concern comes from the potential of quantum computing to render what we consider secure today obsolete. (https://www.technologyreview.com/2019/05/30/65724/how-a-quantum-computer-could-break-2048-bit-rsa-encryption-in-8-hours/ https://www.technologyreview.com/2019/05/30/65724/how-a-quan...)

sneak 6y ago

This is not a use case anyone serious cares about. Data retention for ciphertext where the keys, stored and distributed elsewhere, have been deleted/zeroized, is a complete non-issue.

ahnick 6y ago
Why is that? I think it is entirely possible that in the future, encryption techniques used today will become compromised as both computing power improves and/or flaws are discovered. If someone is holding ciphertext encrypted with a compromised algorithm, then it could be decrypted. Basically, how do you know what we are encrypting today, won't be able to have the encryption broken 30 or 50 years from now?
- folmar 6y ago
  You don't, but offline attacks tend to be relatively hard. 3DES is 25 years old now and not an easy target for a casual attacker.