4 ms·

No, I don't think that's disturbing. That's the point of backups - even when something is permanently and completely erased in the production database, it's st

No, I don't think that's disturbing. That's the point of backups - even when something is permanently and completely erased in the production database, it's still in the backup. Eventually it will get rotated out as the backups expire. Going back and purging things from the backups as part of the delete process would be overdoing it to a ridiculous degree.

yebyen 5y ago
I think that depends on what you mean by compliance. Some regulations require you to irreversibly destroy data when they prescribe the destruction of that data. That can mean as much as "you have to encrypt everything with a separate key, so that you can destroy the key for the given (say, personally identifiable) dataset making its retrieval irrecoverable" I'm not saying that's the particular compliance reason they had here, or that the analysis you're giving is wrong, either. There is an interpretation where either of these ideas could be the correct one.
- nindalf 5y ago
  This is why regulations specify that data must be destroyed within a time period, typically 90 days. It gives enough time for backups to rotate out. If this weren’t a concern, regulations would demand immediate deletion of data.
- a2800276 5y ago
  "permanently delete" strongly suggests to me that it was the "medical and financial data" kind of compliance. If data can be restored, it's not permanently deleted. But this was a statement from the CEO, so words can have arbitrary meaning :)
  - notatoad 5y ago
    "permanently delete" does not mean the same thing as "immediately delete". deleting from the live database is the first step of a permanent deletion, as long as the data exists somewhere the deletion process is still in-progress. there's a whole lot of people in here who are way too quick to assume that just because one part of a permanent deletion process was inadvertently triggered and then caught while they still had backups, their whole permanent deletion process is a lie.
    - voxic11 5y ago
      https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-regulation-gdpr/individual-rights/right-to-erasure/#ib5 https://ico.org.uk/for-organisations/guide-to-data-protectio... You seem to be right-ish, while the gdpr in certain circumstances allows you to keep backups of data that should have been deleted it seems like they are trying to discourage it in the future. > ...It is, however, important to note that where data put beyond use is still held it might need to be provided in response to a court order. Therefore data controllers should work towards technical solutions to prevent deletion problems recurring in the future.
      - deleted 5y ago
        [deleted]
  - jacobsenscott 5y ago
    A better way to do this sort of thing is not an actual "delete", but a "cryptographic delete". The data should be encrypted, and you just delete the key. The data is then unrecoverable everywhere, including backups. Of course you probably don't want to just nuke the key, but disable it for some period of time, and then nuke it.
    - notatoad 5y ago
      i don't see how that really changes anything - your keys should be backed up just as much as, if not more than your data. and any process for deleting the encryption keys should allow for restoring from backups for some period of time just the same as your process for deleting data should allow restoring from backups for some period of time. either way, permanently rendering data as unrecoverable takes time.
      - jacobsenscott 5y ago
        As an example, if you are using Amazon's KMS for key management and you destroy a key it gives you 7 days to undo before permanently destroying the key. Or you can disable they key and destroy it later as your retention policy permits. Surely they have some kind of key backup, but KMS users have no access to those backups.
Delitio 5y ago
Nope it's not ridiculous. If you are only allowed to store data for x month that's it. It's your job to use technics which allow you to do this like using encryption on your backup and deleting the keys for it, for example.
- scottlamb 5y ago
  Delitio> If you are only allowed to store data for x month that's it. Exactly. I'm not aware of any laws saying "you must delete this data immediately". More like "within X days or months". The permanently delete thing presumably skips some cooling-off period in the online database but not the backup, which seems perfectly appropriate, provided your backup retention is compliant. Google has a nice page describing out their deletion process. [1] It doesn't go into product-specific technical details/steps (like marked as deleted within the product, row deleted from Bigtable/Spanner, major compaction guaranteed to happen, backups guaranteed to be deleted or unusable) but it says this: Google> We then begin a process designed to safely and completely delete the data from our storage systems. Safe deletion is important to protect our users and customers from accidental data loss. Complete deletion of data from our servers is equally important for users’ peace of mind. This process generally takes around 2 months from the time of deletion. This often includes up to a month-long recovery period in case the data was removed unintentionally. This is a best practice. Delitio> It's your job to use technics which allow you to do this like using encryption on your backup and deleting the keys for it, for example. If they'd thrown away the encryption key immediately, this would have been much worse. Instead of "we're down for 2 weeks?!?" (already quite bad) it'd be "our data is gone forever?!?". You never want to delete anything too quickly for exactly this reason. [1] https://policies.google.com/technologies/retention?hl=en-US https://policies.google.com/technologies/retention?hl=en-US
usefulcat 5y ago
> Going back and purging things from the backups as part of the delete process would be overdoing it to a ridiculous degree. Also, modifying backups is a great way to inadvertently hose your backups.

yebyen 5y ago

I think that depends on what you mean by compliance. Some regulations require you to irreversibly destroy data when they prescribe the destruction of that data. That can mean as much as "you have to encrypt everything with a separate key, so that you can destroy the key for the given (say, personally identifiable) dataset making its retrieval irrecoverable" I'm not saying that's the particular compliance reason they had here, or that the analysis you're giving is wrong, either. There is an interpretation where either of these ideas could be the correct one.

nindalf 5y ago
This is why regulations specify that data must be destroyed within a time period, typically 90 days. It gives enough time for backups to rotate out. If this weren’t a concern, regulations would demand immediate deletion of data.
a2800276 5y ago
"permanently delete" strongly suggests to me that it was the "medical and financial data" kind of compliance. If data can be restored, it's not permanently deleted. But this was a statement from the CEO, so words can have arbitrary meaning :)
- notatoad 5y ago
  "permanently delete" does not mean the same thing as "immediately delete". deleting from the live database is the first step of a permanent deletion, as long as the data exists somewhere the deletion process is still in-progress. there's a whole lot of people in here who are way too quick to assume that just because one part of a permanent deletion process was inadvertently triggered and then caught while they still had backups, their whole permanent deletion process is a lie.
  - voxic11 5y ago
    https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-regulation-gdpr/individual-rights/right-to-erasure/#ib5 https://ico.org.uk/for-organisations/guide-to-data-protectio... You seem to be right-ish, while the gdpr in certain circumstances allows you to keep backups of data that should have been deleted it seems like they are trying to discourage it in the future. > ...It is, however, important to note that where data put beyond use is still held it might need to be provided in response to a court order. Therefore data controllers should work towards technical solutions to prevent deletion problems recurring in the future.
    - deleted 5y ago
      [deleted]
- jacobsenscott 5y ago
  A better way to do this sort of thing is not an actual "delete", but a "cryptographic delete". The data should be encrypted, and you just delete the key. The data is then unrecoverable everywhere, including backups. Of course you probably don't want to just nuke the key, but disable it for some period of time, and then nuke it.
  - notatoad 5y ago
    i don't see how that really changes anything - your keys should be backed up just as much as, if not more than your data. and any process for deleting the encryption keys should allow for restoring from backups for some period of time just the same as your process for deleting data should allow restoring from backups for some period of time. either way, permanently rendering data as unrecoverable takes time.
    - jacobsenscott 5y ago
      As an example, if you are using Amazon's KMS for key management and you destroy a key it gives you 7 days to undo before permanently destroying the key. Or you can disable they key and destroy it later as your retention policy permits. Surely they have some kind of key backup, but KMS users have no access to those backups.

Delitio 5y ago

Nope it's not ridiculous. If you are only allowed to store data for x month that's it. It's your job to use technics which allow you to do this like using encryption on your backup and deleting the keys for it, for example.

scottlamb 5y ago
Delitio> If you are only allowed to store data for x month that's it. Exactly. I'm not aware of any laws saying "you must delete this data immediately". More like "within X days or months". The permanently delete thing presumably skips some cooling-off period in the online database but not the backup, which seems perfectly appropriate, provided your backup retention is compliant. Google has a nice page describing out their deletion process. [1] It doesn't go into product-specific technical details/steps (like marked as deleted within the product, row deleted from Bigtable/Spanner, major compaction guaranteed to happen, backups guaranteed to be deleted or unusable) but it says this: Google> We then begin a process designed to safely and completely delete the data from our storage systems. Safe deletion is important to protect our users and customers from accidental data loss. Complete deletion of data from our servers is equally important for users’ peace of mind. This process generally takes around 2 months from the time of deletion. This often includes up to a month-long recovery period in case the data was removed unintentionally. This is a best practice. Delitio> It's your job to use technics which allow you to do this like using encryption on your backup and deleting the keys for it, for example. If they'd thrown away the encryption key immediately, this would have been much worse. Instead of "we're down for 2 weeks?!?" (already quite bad) it'd be "our data is gone forever?!?". You never want to delete anything too quickly for exactly this reason. [1] https://policies.google.com/technologies/retention?hl=en-US https://policies.google.com/technologies/retention?hl=en-US

usefulcat 5y ago

> Going back and purging things from the backups as part of the delete process would be overdoing it to a ridiculous degree. Also, modifying backups is a great way to inadvertently hose your backups.