3 ms·

Anecdote: I was desktop client lead for Syncplicity, a major competitor to Dropbox, ect. One day a ticket was sent my way. A customer complained that some file

Anecdote: I was desktop client lead for Syncplicity, a major competitor to Dropbox, ect. One day a ticket was sent my way. A customer complained that some files were missing. Once I got logs from that customer's computer; it was clear, someone, or something, deleted those files... Four months ago. A recent cost-savings initiative bumped the deleted file retention to ~3 months. The retention window had come and gone, and we deleted the customers' files from their recycle bin. My response was basically: We need a longer retention plan if we're going to call ourselves "backup." The point of "backup" is to restore in cases like this, where someone (or something) accidentally deleted files. I even had that lesson with some important music files. I shared them with the person who originally recorded the files, and they dragged and dropped them out of the shared folder. It wasn't until a few years later that I realized I needed to restore them. Because I had the VIP flag, I was able to restore them.

skydhash 3y ago
> My response was basically: We need a longer retention plan if we're going to call ourselves "backup." The point of "backup" is to restore in cases like this, where someone (or something) accidentally deleted files. I'm using local backup software and it's pretty clear to me that versioned and trashed files still occupies space until I delete them permanently. If you provide 1TB of storage space, make it clear that the trash and versioning occupies some of that space instead of a random retention period. Ditto if it's unlimited and priced by usage.
refulgentis 3y ago
In your experience, at what point is the retention window sufficient? Naively, infinite must be the wrong answer, and any other answer from there seems like bikeshedding, but of course it does to me, who has 0 experience in the space
- mac-mc 3y ago
  You charge for deleted file retention like it's any other thing that takes up space and let the customer decide how much they want to pay for. Some will want to pay for "5 years" worth, others won't care. To be extra good about it, maybe do it like borg backup and others and save the changes with dedup and compression. You can have literal years of file retention and have it take up less space than you think it would.
  - rsync 3y ago
    This is the correct answer.
- MyFedora 3y ago
  If you can recover everything, then the data retention is too high. If you can't recover anything, then the data retention is too low. Find a sweet spot where you can satisfy most clients at a manageable cost. Offer problem clients a higher data retention for a premium.
- gwbas1c 3y ago
  A lot of responses in the thread say that deleted files should be counted in the quota. (And then stored until the user purges.) I'm general, I think this is a good approach. The consequence is that there is a certain class of user that won't understand this and will get confused when their account hits quota, but they don't see those files on their computer. Another consequence is that we don't sync recycle bins, so it wouldn't be straightforward for users to purge it. I personally think that moving deleted files to something like Glacier, (slow and cheap, think tape in the cloud,) and then having a separate recycle bin quota, is the best approach. (https://aws.amazon.com/pm/s3-glacier/?trk=75fef084-bf49-48b4-91ad-0db6e1da9c7e&sc_channel=ps&ef_id=Cj0KCQiAoeGuBhCBARIsAGfKY7yEHiD0BJV_iqVW_XrxAwHnN9VYwcC5dP4WZetmx4Sujse4ayKwIxUaAp2bEALw_wcB:G:s&s_kwcid=AL!4422!3!674509886397!e!!g!!amazon%20s3%20glacier!19852662443!154748795178 https://aws.amazon.com/pm/s3-glacier/?trk=75fef084-bf49-48b4...)
- treflop 3y ago
  As someone using cloud storage, the only answer is the amount I pay for. I pay for one year with Backblaze (on the cheaper plan, it’s much shorter but they’re explicit about it) and I expect one year and they hold themselves to that one year. It’s a number we both decided on ahead of time. For something like Google Drive or Dropbox, I expect no retention at all tbh. It’s a plus if I need it and they have it.
justinclift 3y ago
Isn't the retention window something that should be chosen by the customer?
- gwbas1c 3y ago
  Possibly, but: That's confusing for non-technical people, and it makes the product more complicated. Corner cases were the source of (almost) all of our bugs. (A configurable retention window will add corner cases.) What happens is that one customer will have a particular combination of corner cases that no one else has, and then that customer hits a bug that no one else will hit.
  - gwbas1c 3y ago
    > and then that customer hits a bug that no one else will hit. What I should also add is that bugs that only effect a few customers will be very low priority. IE: This is why software with lots of options is often buggy: Options create corner cases that aren't high priority to debug.

skydhash 3y ago

> My response was basically: We need a longer retention plan if we're going to call ourselves "backup." The point of "backup" is to restore in cases like this, where someone (or something) accidentally deleted files. I'm using local backup software and it's pretty clear to me that versioned and trashed files still occupies space until I delete them permanently. If you provide 1TB of storage space, make it clear that the trash and versioning occupies some of that space instead of a random retention period. Ditto if it's unlimited and priced by usage.

refulgentis 3y ago

In your experience, at what point is the retention window sufficient? Naively, infinite must be the wrong answer, and any other answer from there seems like bikeshedding, but of course it does to me, who has 0 experience in the space

mac-mc 3y ago
You charge for deleted file retention like it's any other thing that takes up space and let the customer decide how much they want to pay for. Some will want to pay for "5 years" worth, others won't care. To be extra good about it, maybe do it like borg backup and others and save the changes with dedup and compression. You can have literal years of file retention and have it take up less space than you think it would.
- rsync 3y ago
  This is the correct answer.
MyFedora 3y ago
If you can recover everything, then the data retention is too high. If you can't recover anything, then the data retention is too low. Find a sweet spot where you can satisfy most clients at a manageable cost. Offer problem clients a higher data retention for a premium.
gwbas1c 3y ago
A lot of responses in the thread say that deleted files should be counted in the quota. (And then stored until the user purges.) I'm general, I think this is a good approach. The consequence is that there is a certain class of user that won't understand this and will get confused when their account hits quota, but they don't see those files on their computer. Another consequence is that we don't sync recycle bins, so it wouldn't be straightforward for users to purge it. I personally think that moving deleted files to something like Glacier, (slow and cheap, think tape in the cloud,) and then having a separate recycle bin quota, is the best approach. (https://aws.amazon.com/pm/s3-glacier/?trk=75fef084-bf49-48b4-91ad-0db6e1da9c7e&sc_channel=ps&ef_id=Cj0KCQiAoeGuBhCBARIsAGfKY7yEHiD0BJV_iqVW_XrxAwHnN9VYwcC5dP4WZetmx4Sujse4ayKwIxUaAp2bEALw_wcB:G:s&s_kwcid=AL!4422!3!674509886397!e!!g!!amazon%20s3%20glacier!19852662443!154748795178 https://aws.amazon.com/pm/s3-glacier/?trk=75fef084-bf49-48b4...)
treflop 3y ago
As someone using cloud storage, the only answer is the amount I pay for. I pay for one year with Backblaze (on the cheaper plan, it’s much shorter but they’re explicit about it) and I expect one year and they hold themselves to that one year. It’s a number we both decided on ahead of time. For something like Google Drive or Dropbox, I expect no retention at all tbh. It’s a plus if I need it and they have it.

justinclift 3y ago

Isn't the retention window something that should be chosen by the customer?

gwbas1c 3y ago
Possibly, but: That's confusing for non-technical people, and it makes the product more complicated. Corner cases were the source of (almost) all of our bugs. (A configurable retention window will add corner cases.) What happens is that one customer will have a particular combination of corner cases that no one else has, and then that customer hits a bug that no one else will hit.
- gwbas1c 3y ago
  > and then that customer hits a bug that no one else will hit. What I should also add is that bugs that only effect a few customers will be very low priority. IE: This is why software with lots of options is often buggy: Options create corner cases that aren't high priority to debug.