5 ms·

I think this is missing the larger problem with the claim. Yes, using the term clashed with academic cryptography to some extent. But the larger issue was even

I think this is missing the larger problem with the claim. Yes, using the term clashed with academic cryptography to some extent. But the larger issue was even as they intended to use it, it's not a completely accurate description of their product. They learn quite a bit of information compared to you storing the data locally. Why the collision with academic cryptography doesn't matter: Anyone who had even a basic understanding of their product + some academic crypto background would get what they were going for: they have no knowledge of whats going on. Also, strictly speaking,the academic term is zero-knowledge proof or zero-knowledge proof of knowledge. Ie, zero-knowledge is an adjective used to describe a proof (and indeed, if you look at the history of how these evolved, that is exactly what happened). You could reasonably use zero-knowledge as a modifier for something else and it could be acceptable. Indeed, it's a fairly good shorthand for a particular class of definitions of privacy/confidentiality that require any transcript of the protocol can be produced by a simulator who has no knowledge of what transpired. The problem is Spider Oak's cloud backup cannot be zero-knowledge or no knowledge. It almost certainly leaks when you update files and when you delete them. Perhaps they don't log this or delete the logs, but they could. And this meta data could matter to businesses.

rarrrrrr 10y ago
Yes, of course there's some traffic analysis that would be possible, as there would be with any such service. But for the record: we keep logs for a limited time, and we don't just encrypt each file individually. Instead there's an encrypted journal and encrypted data blocks. (Having the additional layer of data blocks allows for better deduplicating one version of a file to the next.) So for each transaction that's uploaded to the servers, we know that the journal gets longer, and that data blocks are added or removed (or both.) All the database work for keeping track of the data blocks (reference accounting, garbage collection) is done client side. More details in this post from 2009: https://spideroak.com/articles/why--how-spideroak-architecture-is-different-than-other-online-storage-services https://spideroak.com/articles/why--how-spideroak-architectu...
- s-macke 10y ago
  This is pretty impressive. I thought that you only encrypt content and filename. But this goes way beyond what I expected from such a service.
- ianmiers 10y ago
  Right, some leakage is inherent and what you provide may be good enough or even the best you can reasonably do. However, There's a long history (even in academic crypto) of what seems like insignificant leakage being important. So it's good to be overt about it. So it looks you are doing blockwise encryption? Which means at least conceptually, not only do you leak when a file is updated, you leak what chunk? At least I'm assuming the journal isn't append only.
  - rarrrrrr 10y ago
    For what it's worth, I think of a journal as append only by definition and that's what SpiderOak does. Unless you have millions of very small files, the journal is going to be tiny relative to the backup content so this is fine. So the server doesn't have a concept of "an existing file was updated" vs "a new file was uploaded" etc. The server only knows "new blocks have arrived." All the "smarts" are on the client. In general operation, only new journal entries and new blocks are added. The only time blocks are removed is when the user intentionally chooses to remove data (we call that operation "purge") Intentional purges can also reduce the total size of the journal, and this the only operation that does so. Most backup software removes previous versions and deleted files after 30 days, but SpiderOak keeps these indefinitely by default, to allow for for point in time recovery, restore from ransom ware infections, mistakes you don't catch right away, etc. You can set a different retention policy if you prefer.
bo1024 10y ago
> Why the collision with academic cryptography doesn't matter[...] Good description and I agree that the collision is not confusing to someone who already knows what a zero-knowledge proof is. But I still appreciate the change because people who first hear the term from the website could be pretty confused if they hear about the academic term later. PS. I see your point about leaking of some metadata but it seems very difficult to expect any cloud service to avoid this. The only solution I see is to continually re-upload a re-encrypted version of all data whether it's been updated or not, and maybe pad the uploads so that they are all some maximum size regardless of how much data there actually is.
- ianmiers 10y ago
  Oh, there is no good solution. You need to do something using ORAM. There some clever tricks you can use that might be efficient (e.g this paper does some very interesting things: https://www.internetsociety.org/doc/oblivisync-practical-oblivious-file-backup-and-synchronization https://www.internetsociety.org/doc/oblivisync-practical-obl...), but I wouldn't realistically expect Spider Oak to cover it. What I would expect them to do is clearly state what they don't cover. Metadata is a thing people understand.

rarrrrrr 10y ago

Yes, of course there's some traffic analysis that would be possible, as there would be with any such service. But for the record: we keep logs for a limited time, and we don't just encrypt each file individually. Instead there's an encrypted journal and encrypted data blocks. (Having the additional layer of data blocks allows for better deduplicating one version of a file to the next.) So for each transaction that's uploaded to the servers, we know that the journal gets longer, and that data blocks are added or removed (or both.) All the database work for keeping track of the data blocks (reference accounting, garbage collection) is done client side. More details in this post from 2009: https://spideroak.com/articles/why--how-spideroak-architecture-is-different-than-other-online-storage-services https://spideroak.com/articles/why--how-spideroak-architectu...

s-macke 10y ago
This is pretty impressive. I thought that you only encrypt content and filename. But this goes way beyond what I expected from such a service.
ianmiers 10y ago
Right, some leakage is inherent and what you provide may be good enough or even the best you can reasonably do. However, There's a long history (even in academic crypto) of what seems like insignificant leakage being important. So it's good to be overt about it. So it looks you are doing blockwise encryption? Which means at least conceptually, not only do you leak when a file is updated, you leak what chunk? At least I'm assuming the journal isn't append only.
- rarrrrrr 10y ago
  For what it's worth, I think of a journal as append only by definition and that's what SpiderOak does. Unless you have millions of very small files, the journal is going to be tiny relative to the backup content so this is fine. So the server doesn't have a concept of "an existing file was updated" vs "a new file was uploaded" etc. The server only knows "new blocks have arrived." All the "smarts" are on the client. In general operation, only new journal entries and new blocks are added. The only time blocks are removed is when the user intentionally chooses to remove data (we call that operation "purge") Intentional purges can also reduce the total size of the journal, and this the only operation that does so. Most backup software removes previous versions and deleted files after 30 days, but SpiderOak keeps these indefinitely by default, to allow for for point in time recovery, restore from ransom ware infections, mistakes you don't catch right away, etc. You can set a different retention policy if you prefer.

bo1024 10y ago

> Why the collision with academic cryptography doesn't matter[...] Good description and I agree that the collision is not confusing to someone who already knows what a zero-knowledge proof is. But I still appreciate the change because people who first hear the term from the website could be pretty confused if they hear about the academic term later. PS. I see your point about leaking of some metadata but it seems very difficult to expect any cloud service to avoid this. The only solution I see is to continually re-upload a re-encrypted version of all data whether it's been updated or not, and maybe pad the uploads so that they are all some maximum size regardless of how much data there actually is.

ianmiers 10y ago
Oh, there is no good solution. You need to do something using ORAM. There some clever tricks you can use that might be efficient (e.g this paper does some very interesting things: https://www.internetsociety.org/doc/oblivisync-practical-oblivious-file-backup-and-synchronization https://www.internetsociety.org/doc/oblivisync-practical-obl...), but I wouldn't realistically expect Spider Oak to cover it. What I would expect them to do is clearly state what they don't cover. Metadata is a thing people understand.