4 ms·

Both AWS S3 and Google cloud storage buckets have an option that makes it impossible to delete stored objects for some period of time. The option was added for

Both AWS S3 and Google cloud storage buckets have an option that makes it impossible to delete stored objects for some period of time. The option was added for some legal compliance reasons, but I find it useful as an extra safeguard that important service data is not accidentally or maliciously deleted.

ramraj07 6y ago
You could always not pay the bill :)
chrismatheson 6y ago
I suppose that feature would be incompatible with user data in a GDPR world ??
- visarga 6y ago
  Git is also incompatible with GDPR, you can't simply delete a file from all history.
  - nix23 6y ago
    Oh man, all my WORM tapes are not GDPR compatible...thats terrible!!!
  - icebraining 6y ago
    You can, you just have to be willing to rewrite history. See git-filter-branch (and don't forget to read the fat WARNING).
  - msh 6y ago
    Git is not really intended for personal data
    - Hamuko 6y ago
      Yeah, I don't really understand why you would like to stuff personal data into a git repository (unless we're strictly speaking about author data). It's really not the tool for it. Now for those who decided that a blockchain is the perfect solution for storing personal data however...
    - syshum 6y ago
      Name and Email is personal data...
  - toong 6y ago
    Git allows you to rewrite history, that is not the problem. But you might have a hard time chasing down every copy, it's a distributed system by design.
    - wegs 6y ago
      Once you rewrite history, you've basically broken the git concept. It's an emergency feature if you've accidentally committed a security token or similar, or before you've pushed upstream, but if you're rewriting history with any sort of regularity, you're using git very, very wrong.
  - phatfish 6y ago
    Don't store your users PII in a git repository then.
    - visarga 6y ago
      The use case is to do dataset versioning for ML. The dataset itself is updated frequently. It would be nice to use a tool that can store efficiently when small changes are made yet allow versioning for reproducibility.
      - HelloNurse 6y ago
        If you do ML on sensitive data from actual users, reproducibility is a crime, not a feature. You need to ensure that your ML forgets deleted data completely and irreversibly.
- tyfon 6y ago
  Depends on how long it is stored, you have some time (one month) to clear your systems when you get a request for deletion.
- mixedbit 6y ago
  I'm not a lawyer, but my understanding is that GDPR requires companies to remove user data upon request in reasonable time-frame. If you keep not deletable backups for one month in order to improve reliability of your service, then my understanding is that it is fine to fulfill the GDPR data deletion requests withing one month period, not immediately.
  - microcolonel 6y ago
    Filtering this stuff from a master transaction log is kinda ludicrous, not looking forward to implementing that.
    - richardwhiuk 6y ago
      I don't think the GDPR imposes a requirement to actively filter backups.
      - microcolonel 6y ago
        It's not a backup, it's the authoritative dataset.
    - dividedbyzero 6y ago
      It's sometimes easier to encrypt everything sensitive with a user-specific key and destroy the key.
      - Ensorceled 6y ago
        That's how I've done it as well. The files are backed up for as long as we want, they are just useless once the user's secret is trashed. You can do the userid blocklist's on database restores etc. but for long term backup file storage it becomes silly.
- tokamak-teapot 6y ago
  No, as the right to be forgotten (if that’s what you’re thinking of) isn’t absolute. The example I’m aware of is where personal information is held due to ‘legitimate interest’ and the request to delete isn’t deemed (however that is defined) to override that. I won’t try to go further as it’s not 100% clear cut and IANAL. Have a look at the legislation or the various explainers that have been posted online if you’re interested. Edit: In this case I would guess that Canon would have trouble claiming a legitimate interest in keeping hold of your photos!
bbarnett 6y ago
Yes, but... This does nothing to protect against a bad cron job script. The web frontend, and even backend, are not linked to background maintenance tasks. And a field in a DB doesn't prevent a manual DB delete, or api call to a backing store. The real problem here, was a lack of backup testing / restore testing. If you have backups, you MUST manually verify they can be restored, in a desired state. And this means restores MUST be tested every time code changes happen, which effect backups.
canofbars 6y ago
I noticed that gitlab added a change so that once you delete a repo, it holds it for over a week and puts a big "Pending Deletion" banner on the repo.

ramraj07 6y ago

You could always not pay the bill :)

chrismatheson 6y ago

I suppose that feature would be incompatible with user data in a GDPR world ??

visarga 6y ago
Git is also incompatible with GDPR, you can't simply delete a file from all history.
- nix23 6y ago
  Oh man, all my WORM tapes are not GDPR compatible...thats terrible!!!
- icebraining 6y ago
  You can, you just have to be willing to rewrite history. See git-filter-branch (and don't forget to read the fat WARNING).
- msh 6y ago
  Git is not really intended for personal data
  - Hamuko 6y ago
    Yeah, I don't really understand why you would like to stuff personal data into a git repository (unless we're strictly speaking about author data). It's really not the tool for it. Now for those who decided that a blockchain is the perfect solution for storing personal data however...
  - syshum 6y ago
    Name and Email is personal data...
- toong 6y ago
  Git allows you to rewrite history, that is not the problem. But you might have a hard time chasing down every copy, it's a distributed system by design.
  - wegs 6y ago
    Once you rewrite history, you've basically broken the git concept. It's an emergency feature if you've accidentally committed a security token or similar, or before you've pushed upstream, but if you're rewriting history with any sort of regularity, you're using git very, very wrong.
- phatfish 6y ago
  Don't store your users PII in a git repository then.
  - visarga 6y ago
    The use case is to do dataset versioning for ML. The dataset itself is updated frequently. It would be nice to use a tool that can store efficiently when small changes are made yet allow versioning for reproducibility.
    - HelloNurse 6y ago
      If you do ML on sensitive data from actual users, reproducibility is a crime, not a feature. You need to ensure that your ML forgets deleted data completely and irreversibly.
tyfon 6y ago
Depends on how long it is stored, you have some time (one month) to clear your systems when you get a request for deletion.
mixedbit 6y ago
I'm not a lawyer, but my understanding is that GDPR requires companies to remove user data upon request in reasonable time-frame. If you keep not deletable backups for one month in order to improve reliability of your service, then my understanding is that it is fine to fulfill the GDPR data deletion requests withing one month period, not immediately.
- microcolonel 6y ago
  Filtering this stuff from a master transaction log is kinda ludicrous, not looking forward to implementing that.
  - richardwhiuk 6y ago
    I don't think the GDPR imposes a requirement to actively filter backups.
    - microcolonel 6y ago
      It's not a backup, it's the authoritative dataset.
  - dividedbyzero 6y ago
    It's sometimes easier to encrypt everything sensitive with a user-specific key and destroy the key.
    - Ensorceled 6y ago
      That's how I've done it as well. The files are backed up for as long as we want, they are just useless once the user's secret is trashed. You can do the userid blocklist's on database restores etc. but for long term backup file storage it becomes silly.
tokamak-teapot 6y ago
No, as the right to be forgotten (if that’s what you’re thinking of) isn’t absolute. The example I’m aware of is where personal information is held due to ‘legitimate interest’ and the request to delete isn’t deemed (however that is defined) to override that. I won’t try to go further as it’s not 100% clear cut and IANAL. Have a look at the legislation or the various explainers that have been posted online if you’re interested. Edit: In this case I would guess that Canon would have trouble claiming a legitimate interest in keeping hold of your photos!

bbarnett 6y ago

Yes, but... This does nothing to protect against a bad cron job script. The web frontend, and even backend, are not linked to background maintenance tasks. And a field in a DB doesn't prevent a manual DB delete, or api call to a backing store. The real problem here, was a lack of backup testing / restore testing. If you have backups, you MUST manually verify they can be restored, in a desired state. And this means restores MUST be tested every time code changes happen, which effect backups.

canofbars 6y ago

I noticed that gitlab added a change so that once you delete a repo, it holds it for over a week and puts a big "Pending Deletion" banner on the repo.