3 ms·
Are you using a cloud service or trying to host your own server? If you're on AWS, for example, there's built-in versioning for S3 buckets: https://docs.aws.am
by solardev 2y ago
Are you using a cloud service or trying to host your own server?
If you're on AWS, for example, there's built-in versioning for S3 buckets: https://docs.aws.amazon.com/AmazonS3/latest/userguide/versioning-workflows.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/versio.... GCP and Azure have similar versioning systems for their storage.
To save disk space, the term you probably want is "data deduplication" (https://en.wikipedia.org/wiki/Data_deduplication https://en.wikipedia.org/wiki/Data_deduplication), and there are various algorithms, filesystems, etc. that can help with that. However, it's somewhat slow and risky, because you have to recombine a final file from fragments spread across different versions. And if a fragment gets corrupted, it could affect not just one version but several or even all.
Disk space is generally cheap these days, and it might not be worth sacrificing performance and integrity just to save a few cents per user.
Can you just give them a disk space limit and charge them for additional space if they need it?
- babyent 2y agoS3 versioning doesn’t do diffs. It stores distinct files and uses internal version number. I’m specifically curious about binary diff between files :sob: for text files it’s easy and not a problem. But I am wondering for other files. I will definitely charge them for disk space, but wondering about diff. Thanks!
- solardev 2y agoYeah, I understand. I was just arguing it might be more trouble than it's worth. Look into delta encoding and https://en.m.wikipedia.org/wiki/Data_differencing https://en.m.wikipedia.org/wiki/Data_differencing and xdelta Edit: more modern might be https://bup.github.io/ https://bup.github.io/
- babyent 2y agoThank you! I will check out bup.