6 ms·

At work we dutifully delete all data on a GDPR request

by eurekin 1y ago

simonw 1y ago
Purely out of interest, how do you verify that the GDPR request is coming from the actual user and not an imposter?
- dijksterhuis 1y ago
  > The organisation might need you to prove your identity. However, they should only ask you for just enough information to be sure you are the right person. If they do this, then the one-month time period to respond to your request begins from when they receive this additional information. https://ico.org.uk/for-the-public/your-right-to-get-your-data-deleted/ https://ico.org.uk/for-the-public/your-right-to-get-your-dat...
- eurekin 1y ago
  In my domain, our set of services only authorizes Customer Centre system to do so. I guess I'd need to ask them for details, but I always assumed they have checks in place
sahila 1y ago
How do you manage deleting data from backups? Do you know not take backups?
- Gigachad 1y ago
  Probably most just ignore backups. But there were some good proposals where you encrypt every users data with their own key. So a full delete is just deleting the users encryption key, rendering all data everywhere including backups inaccessible.
  - liamYC 1y ago
    Smart, how do you backup the users encryption keys?
    - aiiane 1y ago
      A set of encryption keys is a lot smaller than the set of all user data, so it's much more viable to have both more redundant hot storage and more frequently rotated cold storage of just the keys.
  - jandrewrogers 1y ago
    Deletion via encryption only works if every user’s data is completely separate from every other user’s data in the storage layer. This is rarely the case in databases, indexes, etc. It also is often infeasible if the number of users is very large (key schedule state alone will blow up your CPU cache). Databases with data from multiple users largely can’t work this way unless you are comfortable with a several order of magnitude loss of performance. It has been built many times but performance is so poor that it is deemed unusable.
    - alisonatwork 1y ago
      Some of these issues could perhaps be addressed by having fixed retention of PII in the online systems, and encryption at rest in the offline systems. If a user wants to access data of theirs which has gone offline, they take the decryption hit. Of course it helps to be critical about how much data should be retained in the first place. It is true that protecting the user's privacy costs more than not protecting it, but some organizations feel a moral obligation or have a legal duty to do so. And some users value their own privacy enough that they are willing to deal with the decreased convenience. As an engineer, I find it neat that figuring out how to delete data is often a more complicated problem than figuring out how to create it. I welcome government regulations that encourage more research and development in this area, since from my perspective that aligns actually-interesting technical work with the public good.
      - jandrewrogers 1y ago
        > As an engineer, I find it neat that figuring out how to delete data is often a more complicated problem than figuring out how to create it. Unfortunately, this is a deeply hard problem in theory. It is not as though it has not been thoroughly studied in computer science. When GDPR first came out I was actually doing core research on “delete-optimized” databases. It is a problem in other domains. Regulations don’t have the power to dictate mathematics. I know of several examples in multiple countries where data deletion laws are flatly ignored by the government because it is literally impossible to comply even though they want to. Often this data supports a critical public good, so simply not collecting it would have adverse consequences to their citizens. tl;dr: delete-optimized architectures are so profoundly pathological to query performance, and a lesser extent insert performance, that no one can use them for most practical applications. It is fundamental to the computer science of the problem. Denial of this reality leads to issues like the above where non-compliance is required because the law didn’t concern itself with the physics of computation. If the database is too slow to load the data then it doesn’t matter how fast your deterministic hard deletion is because there is no data to delete in the system. Any improvements in the situation are solving minor problems in narrow cases. The core theory problems are what they are. No amount of wishful thinking will change this situation.
- alisonatwork 1y ago
  Backups can have a fixed retention period.
  - sahila 1y ago
    Sure, but now when the backup is restored two weeks later, is the user redeleted or just forgotten about?
    - alisonatwork 1y ago
      Depends on the processes in place at the company. Presumably if a backup is restored, some kind of replay has to happen after that, otherwise all the other users are going to lose data that arrived in the interim. A catastrophic failure where both two weeks of user data and all the related events get irretrievably blackholed could still happen, sure, but any company where that is a regular occurrence likely has much bigger problems than complying with GDPR. The point is that none of these problems are insurmountable - they are all processes and practices that have been in place since long before GDPR and long before I started in this industry 25+ years ago. Even if deletion is only eventually consistent, even if a few pieces of data slip through the cracks, it is not hard to have policies in place that at least provide a best effort at upholding users' privacy and complying with the regulations. Organizations who choose not to bother, claiming that it's all too difficult, or that because deletion cannot be done 100% perfectly it should not even be attempted at all, are making weak excuses. The cynical take would be that they are just covering for the fact that they really do not respect their users' privacy and simply do not want to give up even the slightest chance of extracting value from that data they illegally and immorally choose to retain.
- deleted 1y ago
  [deleted]
- crdrost 1y ago
  "When data subjects exercise one of their rights, the controller must respond within one month. If the request is too complex and more time is needed to answer, then your organisation may extend the time limit by two further months, provided that the data subject is informed within one month after receiving the request." Backup retention policy 60 days, respond within a week or two telling someone that you have purged their data from the main database but that these backups exist and cannot be changed, but that they will be automatically deleted in 60 days. The only real difficulty is if those backups are actually restored, then the user deletion needs to be replayed, which is something that would be easy to forget.
- Trasmatta 1y ago
  Most companies don't keep all backups in perpetuity, and instead have rolling backups over some period of time.
gruez 1y ago
That won't work in this case, because I doubt GDPR requests override court orders.

At work we dutifully delete all data on a GDPR request

simonw 1y ago

Purely out of interest, how do you verify that the GDPR request is coming from the actual user and not an imposter?

dijksterhuis 1y ago
> The organisation might need you to prove your identity. However, they should only ask you for just enough information to be sure you are the right person. If they do this, then the one-month time period to respond to your request begins from when they receive this additional information. https://ico.org.uk/for-the-public/your-right-to-get-your-data-deleted/ https://ico.org.uk/for-the-public/your-right-to-get-your-dat...
eurekin 1y ago
In my domain, our set of services only authorizes Customer Centre system to do so. I guess I'd need to ask them for details, but I always assumed they have checks in place

sahila 1y ago

How do you manage deleting data from backups? Do you know not take backups?

Gigachad 1y ago
Probably most just ignore backups. But there were some good proposals where you encrypt every users data with their own key. So a full delete is just deleting the users encryption key, rendering all data everywhere including backups inaccessible.
- liamYC 1y ago
  Smart, how do you backup the users encryption keys?
  - aiiane 1y ago
    A set of encryption keys is a lot smaller than the set of all user data, so it's much more viable to have both more redundant hot storage and more frequently rotated cold storage of just the keys.
- jandrewrogers 1y ago
  Deletion via encryption only works if every user’s data is completely separate from every other user’s data in the storage layer. This is rarely the case in databases, indexes, etc. It also is often infeasible if the number of users is very large (key schedule state alone will blow up your CPU cache). Databases with data from multiple users largely can’t work this way unless you are comfortable with a several order of magnitude loss of performance. It has been built many times but performance is so poor that it is deemed unusable.
  - alisonatwork 1y ago
    Some of these issues could perhaps be addressed by having fixed retention of PII in the online systems, and encryption at rest in the offline systems. If a user wants to access data of theirs which has gone offline, they take the decryption hit. Of course it helps to be critical about how much data should be retained in the first place. It is true that protecting the user's privacy costs more than not protecting it, but some organizations feel a moral obligation or have a legal duty to do so. And some users value their own privacy enough that they are willing to deal with the decreased convenience. As an engineer, I find it neat that figuring out how to delete data is often a more complicated problem than figuring out how to create it. I welcome government regulations that encourage more research and development in this area, since from my perspective that aligns actually-interesting technical work with the public good.
    - jandrewrogers 1y ago
      > As an engineer, I find it neat that figuring out how to delete data is often a more complicated problem than figuring out how to create it. Unfortunately, this is a deeply hard problem in theory. It is not as though it has not been thoroughly studied in computer science. When GDPR first came out I was actually doing core research on “delete-optimized” databases. It is a problem in other domains. Regulations don’t have the power to dictate mathematics. I know of several examples in multiple countries where data deletion laws are flatly ignored by the government because it is literally impossible to comply even though they want to. Often this data supports a critical public good, so simply not collecting it would have adverse consequences to their citizens. tl;dr: delete-optimized architectures are so profoundly pathological to query performance, and a lesser extent insert performance, that no one can use them for most practical applications. It is fundamental to the computer science of the problem. Denial of this reality leads to issues like the above where non-compliance is required because the law didn’t concern itself with the physics of computation. If the database is too slow to load the data then it doesn’t matter how fast your deterministic hard deletion is because there is no data to delete in the system. Any improvements in the situation are solving minor problems in narrow cases. The core theory problems are what they are. No amount of wishful thinking will change this situation.
alisonatwork 1y ago
Backups can have a fixed retention period.
- sahila 1y ago
  Sure, but now when the backup is restored two weeks later, is the user redeleted or just forgotten about?
  - alisonatwork 1y ago
    Depends on the processes in place at the company. Presumably if a backup is restored, some kind of replay has to happen after that, otherwise all the other users are going to lose data that arrived in the interim. A catastrophic failure where both two weeks of user data and all the related events get irretrievably blackholed could still happen, sure, but any company where that is a regular occurrence likely has much bigger problems than complying with GDPR. The point is that none of these problems are insurmountable - they are all processes and practices that have been in place since long before GDPR and long before I started in this industry 25+ years ago. Even if deletion is only eventually consistent, even if a few pieces of data slip through the cracks, it is not hard to have policies in place that at least provide a best effort at upholding users' privacy and complying with the regulations. Organizations who choose not to bother, claiming that it's all too difficult, or that because deletion cannot be done 100% perfectly it should not even be attempted at all, are making weak excuses. The cynical take would be that they are just covering for the fact that they really do not respect their users' privacy and simply do not want to give up even the slightest chance of extracting value from that data they illegally and immorally choose to retain.
deleted 1y ago
[deleted]
crdrost 1y ago
"When data subjects exercise one of their rights, the controller must respond within one month. If the request is too complex and more time is needed to answer, then your organisation may extend the time limit by two further months, provided that the data subject is informed within one month after receiving the request." Backup retention policy 60 days, respond within a week or two telling someone that you have purged their data from the main database but that these backups exist and cannot be changed, but that they will be automatically deleted in 60 days. The only real difficulty is if those backups are actually restored, then the user deletion needs to be replayed, which is something that would be easy to forget.
Trasmatta 1y ago
Most companies don't keep all backups in perpetuity, and instead have rolling backups over some period of time.

gruez 1y ago

That won't work in this case, because I doubt GDPR requests override court orders.