5 ms·
AWS S3 object versioning makes it pretty easy to allow a server to add backup data without the ability to permanently modify or delete data that existed previou
by justin_oaks 5y ago
AWS S3 object versioning makes it pretty easy to allow a server to add backup data without the ability to permanently modify or delete data that existed previously.
For my backups I use restic and sync the restic repository data to S3. Even if the source data is corrupted, I can always roll back to the set of S3 object versions from a particular time.
The downside to using S3 object versioning is that I haven't found any good tools to work with object versions across multiple objects.
For example, I need to prune old backups from my restic repository. To do that I have to delete object versions that are no longer current (i.e. the latest version of an object). To accomplish this I had to write a script that uses Boto3 (AWS SDK for Python) that lets me delete non-current object versions and deletes any delete-marker versions.
The code was pretty straightforward, but I wish there was a tool that made it easier to do that.
- dageshi 5y agoNot 100% sure but can't you setup rules for that stuff in s3 itself? Like delete everything older than x date but never delete current version? I have a feeling I set something like this up but it's been a while since I did it.
- nijave 5y agoYeah you can use lifecycle policies https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/object...
- andrewguenther 5y agoYou are correct, S3 lifecycle policies support deleting non-current object versions once they're X days old as well as X versions old: https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-configuration-examples.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecy...
- justin_oaks 5y agoYou can, but automatically doing it would defeat the whole purpose of having versions I could roll back to. I need to confirm that that my recent backups are valid before deleting old backups. Otherwise I might as well not have used S3 versioning.