3 ms·
the problem with this theory is that deduplication works so well with file backup as the largest files we have are most often music and video, something that is
by esutton 15y ago
the problem with this theory is that deduplication works so well with file backup as the largest files we have are most often music and video, something that is highly replicated amongst file sharing users.
i.e millions of users have the same Beatles album.
Yet, when each persons album is encrypted it will be entirely different than every other persons album. If they were in any way similar it was not correctly encrypted. Thus the hit rate will be almost negligible.
So than you say OK, but maybe the block level code of the encryption of my Beatles album is similar to the block level code of the encryption of my Eminem album, and than we can save storage there. The problem with this is that encryption algorithms aim to make their input files output as close to a random distribution as possible. Meaning not only are those two copies of the beetles albums going to be as different from each other, but they will be different from any other file.
To sum it up, a well encrypted file should be uniformly random -> meaning it is uncompressible - > two different uniformly random files are still random relative to each other and when put together still cannot be compressed.
- indrax 15y agoThe easy solution is to segment easily deduplicated data from 'private' data. Perhaps sufficiently deduplicated data doesn't count against your quota. (as amazon does with music)
- evgen 15y agoThe problem with this is that the data which you want to de-dupe are the big files that everyone has a copy of; files like mp3s, videos, etc. Unfortunately these are the same files that users are most likely to not want the RIAA/MPAA to know about and keep private. The short version of all of this is that Bitcasa can't actually pull off what they are claiming and still maintain any user security/privacy.
- count 15y agoAt the 'infinite' scale of storage that they're discussing, you should still see a savings. It's not going to be 30% or 80%, but it will be savings. De-duplication CAN still take place.
- esutton 15y agoactually it should be close to nothing, encrypted data cannot be compressed as it is supposed to be uniformly random. If it can be compressed, the encryption algorithm is weak. also on stage at tech crunch disrupt the founders were talking about the majority of the data be deduplicated