7 ms·

Liberate - sure; Delete - maybe later (if ever).

by maxcap 17y ago

req2 17y ago
"A fanatic is one who can't change his mind and won't change the subject."
blasdel 17y ago
Their infrastructure makes it ridiculously inefficient to support on-demand erasure. In a system designed to never lose data, where most things are stored immutably in a log-structure, a delete feature is an unwelcome opportunity for massive failure. It's much better to delete things passively, even if it means waiting indefinitely and just not copying them in the next system migration.
patio11 17y ago
If this worries you, assume nothing you type can be ever deleted. I own a very, VERY small web application. It takes input from users and, potentially, saves it to the database, then performs operations on it. Suppose a user types "foobar" into my application, waits a while, and then tells me "Hey, I want you to totally erase any evidence that I ever typed "foobar". Well, um, I don't think it is physically possible for me to do so. Minimally, "foobar" is now present in my database. I can zap that fairly easily. Foobar might also be present at a few places in memcached, which are difficult to me to calculate but theoretically accessible to me. I suppose if they're theoretically accessible I could, with significant effort, call them up again, which means that with significant effort I can delete them. OK, zap. Then I have backups of my database. And here's where delete starts to become a matter of "Uh oh, now we're talking hard." I can't just blow away information in a database backup -- I'd have to unpack it, load it into a database (binary dumps = do not work on me with ad hoc tools!), blow away the record, then save. This would have to be done very carefully to avoid nasty effects if two or more people wanted to blow away data at the same time, since maintaining ACID guarantees is pretty difficult when you've got multiple independent copies of the same database running around. At this point, I'm already strongly inclined to say "If it ever hits the database backups, it potentially stays until doomsday." Then we have server backups. My server is a VPS. If you were in the database backup saved on disk at T1, when a server backup happened (which freezes an image of the VPS in time so that I can return to exactly the same state as T1), then even if I blow away the backup there is ANOTHER opaque backup in an even more opaque data structure. So I'd have to spin up another VPS from the image (for each image I have), deal with the pains of moving that to the present day/time, load each backup on the VPS, blow away your record, refreeze the database, then refreeze the VPS. All of this unfreezing and refreezing presents many, many opportunities for me to corrupt other users' data, which is the reason I have the backups in the first place. Corrupting data is unacceptable. Except, wait, I'm not the only one who has copies of my images! Slicehost also has redundant backups of the images because I can't lose an image if they lose the box the image is on! So I'd need to somehow gain access of their backups (which I have no control over) to spin up the VPS to spin up the backup to nuke the record to backup the DB to snapshot the VPS to remake the backup of the hard drive containing hundreds of images from people who are not me. EXCEPT WAIT! It is conceivable that, without notice to me, Slicehost has moved from owning their own backup media into a cloud storage solution. Which, for reliability, would duplicate the backup machine multiple times! So now we need to access each of the the copies of the backup machine to spin up each of the images to spin up the database so we can nuke the record to save the database to snapshot the image to back up the snapshot to replicate the backup machine which persists the backups of the snapshots containing the backups of the database which holds your record. So, yep, that's where I am. You probably did not read my privacy policy, but I'm pretty sure it says something to the effect of "I make no guarantees about being able to delete your data." That is as much as I am going to say of the matter. As soon as you give me the data to hold for a nanosecond it could very well be out of my control to ever delete again.
- maxcap 17y ago
  Liberation is good - Google is good - so are privacy agreements. A better name for the site could be "Grab a Copy of Your Data!" - much more accurate. Liberate implies that you are actually getting your data back - which you're not...you're getting...a.....copy.
- DenisM 17y ago
  Immediate deletion is hard. Retention policy is easy. At my company after 90 days the the backups are blown away. I presume Amazon S3 will take time to age out the data after I'm through with it, but they too will eventually delete it.

Liberate - sure; Delete - maybe later (if ever).

by maxcap 17y ago

Liberate - sure; Delete - maybe later (if ever).

req2 17y ago

"A fanatic is one who can't change his mind and won't change the subject."

blasdel 17y ago

Their infrastructure makes it ridiculously inefficient to support on-demand erasure. In a system designed to never lose data, where most things are stored immutably in a log-structure, a delete feature is an unwelcome opportunity for massive failure. It's much better to delete things passively, even if it means waiting indefinitely and just not copying them in the next system migration.

patio11 17y ago

If this worries you, assume nothing you type can be ever deleted. I own a very, VERY small web application. It takes input from users and, potentially, saves it to the database, then performs operations on it. Suppose a user types "foobar" into my application, waits a while, and then tells me "Hey, I want you to totally erase any evidence that I ever typed "foobar". Well, um, I don't think it is physically possible for me to do so. Minimally, "foobar" is now present in my database. I can zap that fairly easily. Foobar might also be present at a few places in memcached, which are difficult to me to calculate but theoretically accessible to me. I suppose if they're theoretically accessible I could, with significant effort, call them up again, which means that with significant effort I can delete them. OK, zap. Then I have backups of my database. And here's where delete starts to become a matter of "Uh oh, now we're talking hard." I can't just blow away information in a database backup -- I'd have to unpack it, load it into a database (binary dumps = do not work on me with ad hoc tools!), blow away the record, then save. This would have to be done very carefully to avoid nasty effects if two or more people wanted to blow away data at the same time, since maintaining ACID guarantees is pretty difficult when you've got multiple independent copies of the same database running around. At this point, I'm already strongly inclined to say "If it ever hits the database backups, it potentially stays until doomsday." Then we have server backups. My server is a VPS. If you were in the database backup saved on disk at T1, when a server backup happened (which freezes an image of the VPS in time so that I can return to exactly the same state as T1), then even if I blow away the backup there is ANOTHER opaque backup in an even more opaque data structure. So I'd have to spin up another VPS from the image (for each image I have), deal with the pains of moving that to the present day/time, load each backup on the VPS, blow away your record, refreeze the database, then refreeze the VPS. All of this unfreezing and refreezing presents many, many opportunities for me to corrupt other users' data, which is the reason I have the backups in the first place. Corrupting data is unacceptable. Except, wait, I'm not the only one who has copies of my images! Slicehost also has redundant backups of the images because I can't lose an image if they lose the box the image is on! So I'd need to somehow gain access of their backups (which I have no control over) to spin up the VPS to spin up the backup to nuke the record to backup the DB to snapshot the VPS to remake the backup of the hard drive containing hundreds of images from people who are not me. EXCEPT WAIT! It is conceivable that, without notice to me, Slicehost has moved from owning their own backup media into a cloud storage solution. Which, for reliability, would duplicate the backup machine multiple times! So now we need to access each of the the copies of the backup machine to spin up each of the images to spin up the database so we can nuke the record to save the database to snapshot the image to back up the snapshot to replicate the backup machine which persists the backups of the snapshots containing the backups of the database which holds your record. So, yep, that's where I am. You probably did not read my privacy policy, but I'm pretty sure it says something to the effect of "I make no guarantees about being able to delete your data." That is as much as I am going to say of the matter. As soon as you give me the data to hold for a nanosecond it could very well be out of my control to ever delete again.

maxcap 17y ago
Liberation is good - Google is good - so are privacy agreements. A better name for the site could be "Grab a Copy of Your Data!" - much more accurate. Liberate implies that you are actually getting your data back - which you're not...you're getting...a.....copy.
DenisM 17y ago
Immediate deletion is hard. Retention policy is easy. At my company after 90 days the the backups are blown away. I presume Amazon S3 will take time to age out the data after I'm through with it, but they too will eventually delete it.