11 ms·

The author uses the "no one ever undeleted anything" as the primary justification. I think this is the part they miss. I've never undeleted a user either, but

The author uses the "no one ever undeleted anything" as the primary justification. I think this is the part they miss. I've never undeleted a user either, but there have been many times I've gone back to look at something. Either a complaint finally gets around to me as to why the user wanted their account deleted (e.g. feature not working) and it helps to figure out why. Or they're returning and want things set up like they were. Or someone is taking over their roll and needs to be set up like the last person who's already gone. Though you really shouldn't be relying on a database for an audit trail. It might help find some issues, but things actually used for security shouldn't be writable so easily.

alerighi 4y ago
It may be convenient, but under the GDPR is illegal. When an user deletes an account, all the personal data associated with that user must be deleted (or anonymize it in a way that it's no longer possible to associate it back to the particular user). You cannot just keep user information forever "just in case" they are useful again.
- changoplatanero 4y ago
  User deleting an account is just one way that stuff gets deleted. There are other deletion scenarios where it is appropriate to keep the information after its deleted.
  - Akronymus 4y ago
    Yeah. Basically anything that an employee can mess up, should be reversible IMO. But actions such as deleting an account should have the option of "YES, DELETE IT PERMANENTLY, THIS CAN'T BE UNDONE"
    - closeparen 4y ago
      Often it’s mark for deletion after some number of days.
    - charcircuit 4y ago
      The only way that would make sense in if it was a cancelable timed deletion. People make mistakes. People change their minds. People get hacked.
    - Gigachad 4y ago
      You probably want to put the account on hold for a period where it can be undone. And then clean it up after.
      - Akronymus 4y ago
        Not always possible in a legal way. For example, GDPR compliance.
        Gigachad 4y ago
        GDPR gives you a pretty large period to do it within. Easily enough to fit this feature in.
- smolder 4y ago
  I wonder what percentage of companies who even appear to comply with deletion requests actually do full deletion in practice. I suspect it's small, knowing how many things are coded to fake-delete for convenience. Businesses also tend to keep cold data backups around. (Maybe backups are exempted? I don't know.) There might even be cases where ostensibly deleted data can still be recovered from a disk, if they haven't overwritten.
  - avianlyric 4y ago
    Guess we’ll find out when a leaks happen, and companies start getting fines. GDPR does differentiate between structured data (I believe it uses the term “identifiable records” or similar), and huge heap of unstructured data where an individuals data can’t be quickly retrieved as it’s own atomic unit. With much stricter requirements for anything structured. So for data on a HDD that could be recovered, but is an unstructured mess. You’re probably ok, as long as you took reasonable steps to protect the data, like full disk encryption (which could occurs below the file system, and thus allow you to recover data deleted from the FS with a full disk scan, as long as you still had the keys). If you just had peoples data unencrypted on a HDD, and didn’t securely erase it before dispose, then that’s probably still a GDPR violation.
    - JohnBooty 4y ago
      What does GDPR say about obscuring data? Instead of hard-deleting a structured record for Bob Smith, can you leave the record intact and scrub it of identifying data so that I don't e.g. break all of the records for orders that Bob Smith made?
      - alerighi 4y ago
        You can as long as you delete all the personal data of the user and make sure that other remaining data is anonymized in a way that you can no longer associate it with the user.
      - deleted 4y ago
        [deleted]
- dataflow 4y ago
  It's illegal but companies don't necessarily care to avoid soft deletes regardless. I think companies wait to get sued so they can try to argue in court why their soft deletions are reasonable and why it's too technically difficult for them to do hard deletes. To be honest, in the age of modern overprovisioned storage drives that remap blocks frequently, I'm not really sure you can implement genuine "hard" deletes without choosing significantly unorthodox hardware (or destroying a drive every time you need a single bit erased), no matter how much you want to in software. One of those details that I'm both surprised and unsurprised doesn't seem to have been addressed legally. I feel like a court ought to at least buy this aspect of the argument, so maybe they'll buy that it can be difficult in terms of the software too? Who knows. My guess is that a reasonable court would accommodate something that's reasonable for a given company, but there are lots of variations that could fall into that category.
  - jcheng 4y ago
    GDPR fines are structured to make it extremely risky/expensive to take the “ask for forgiveness not permission” approach you’re suggesting. I think for this reason, the fine is designed to scale with the size of the offending company’s revenue, not so much with the actual damages suffered by anyone. Take a look at some of these fines, tens of millions of euros cause the UI for your cookie consent dialog is poorly designed? https://www.tessian.com/blog/biggest-gdpr-fines-2020/ https://www.tessian.com/blog/biggest-gdpr-fines-2020/ When the GDPR rules came out our company (pretty small at the time) pulled out all the stops to get ourselves into compliance, due to the above and other teeth put into the regulations. It seemed utterly irrational not to. Were we the exception, or the rule?
    - dataflow 4y ago
      I don't know if you were the exception or the rule, but I know for a fact some do go the forgiveness approach for requirements that they believe would be disproportionately burdensome. Probably almost all companies actually, including yourself (just to a different extent) - what did you do about the storage remapping thing I mentioned? Did it come up/did you guys discuss it? Do you believe you're in compliance despite your hardware (most likely) not guaranteeing erasure or overwriting of existing data? I'm curious how your assessment of that went, because I doubt one can be in strict compliance without guarantees from the hardware.
- afiori 4y ago
  I would guess that there are quite a few limits to that... A user has a long history of participating on your forum and other users have quoted their messages far and wide. Collectively all of the messages posted on your forums (with or without timestamps) reveal a few PII about the user. Do you have to delete those? The user filed a bug report about a functionality not working, do you have to delete the text of the bug report? Arguably if your user table look like [user_id, creation_date, deletion_date, status, account_type] then this table does not contain any PII. Assuming that user content is not automatically PII, whose responsibility is it to track where PII can be? Sometimes users doxx themselves (like mistakenly sharing tax return forms instead cat pics), in such a case it is the user responsibility to signal this to you. If the user filed an issue saying "when I insert my name (Abe Cox) the input validation fails" is it you or the user that need to read through all the issues to find this case? My point is that GDPR + right to be forgotten cannot make it look like you never had an account at all, especially without user assistance.
  - hnbad 4y ago
    user_id is PII if the identity of a user can be narrowed down sufficiently. For example, if you created a social media app and later left the company, it could still be determined that if you had an account it likely has one of the lowest user ID values if those were assigned sequentially. So even if your name was removed from it and none of your posts provided any identifying information about you, the user ID could still qualify as PII. Likewise if the user ID is public and non-PII content associated with that ID is linked elsewhere identifying the author of that content (e.g. a news article embedding a post) the identity of the author is still compromised. The problem is not just that users may submit PII in unexpected places, the problem is that even if the data type could not possibly allow them to, metadata beyond your control may actually still taint your supposedly anonymous data.
    - afiori 4y ago
      The question is not whether "spooky deanonymization as a distance" is possible; the question is how much you should proactively do to avoid this. As an extreme example, if user A takes a screenshot if their own user setting page on a image sharing site, then sends it to user B that uploads it to the same site. Who is responsible for this piece of data? does share-a-pic.web need to scan all images for the username of a user that is deleting an account? This is a convoluted example where the answer is obviously no, but it shows that a purely principled answer might be unsuitable. Especially since non PII data you own mixed with non PII data someone else owns can become PII and you do not know what data others have. (you could also replace the user ID in all locations with a new tagged UUID to preserve referential integrity in the DB, but there not much more than 2^32 humans in total, it s very hard not to be deanonymizable)
- 411111111111111 4y ago
  Stop spreading misinformation. Nobody is required to hard delete the very second the [delete] button is pressed. You're following the law as long as that mentioned reaper job exists and runs at least once per month, to be safely within the allowed timeframe
  - brigandish 4y ago
    I didn't see a claim that deletion had to be "the very second the [delete] button is pressed". This is the actual text of the law[1]: > The data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay There is a one month limit of the right of access[2] and right to be informed[3], which flow into some of the articles of the law; and certain articles also mention a month's limit on being informed, e.g. from article 12 parts 3 and 4 [4]: > The controller shall provide information on action taken on a request under Articles 15 to 22 to the data subject *without undue delay and in any event within one month* of receipt of the request. That period may be extended by two further months where necessary, taking into account the complexity and number of the requests. The controller shall inform the data subject of any such extension within one month of receipt of the request, together with the reasons for the delay. Where the data subject makes the request by electronic form means, the information shall be provided by electronic means where possible, unless otherwise requested by the data subject. If the controller does not take action on the request of the data subject, the controller shall inform the data subject without delay and at the latest within one month of receipt of the request of the reasons for not taking action and on the possibility of lodging a complaint with a supervisory authority and seeking a judicial remedy. I've marked out those phrases because they show the language of the law that you will see again and again in the articles - without undue delay and within one month. The latter does not obviate the former. Hence, and in fact, there is no part of the law that explicitly gives you a month to delete something. If you could delete something right now but you leave it hanging around because you have arbitrarily chosen a job to run once a month and there's a leak in that time, that would be undue delay. [1] https://gdpr-info.eu/art-17-gdpr/ https://gdpr-info.eu/art-17-gdpr/ [2] https://gdpr-info.eu/issues/right-of-access/ https://gdpr-info.eu/issues/right-of-access/ [3] https://gdpr-info.eu/issues/right-to-be-informed/ https://gdpr-info.eu/issues/right-to-be-informed/ [4] https://gdpr-info.eu/art-12-gdpr/ https://gdpr-info.eu/art-12-gdpr/
- twelve40 4y ago
  You can most definitely keep around the non-PII data you've generated for that user in your app, such as synthetic user ID's and other stuff. Furthermore, for some apps (fintech) in some countries (US) you are actually obligated to keep around certain information for several years. It really doesn't have to be all or nothing.
  - bpicolo 4y ago
    Insurance has to keep around info for the lifetime of the policy. For some products, that's decades.
  - hnbad 4y ago
    Synthetic user IDs can still qualify as PII since a human may still know which person that ID was associated with and data associated with that ID may be sufficient to narrow down who it was even if no human (or system) knows the exact mapping. Pseudonymous data is not anonymous data. But yes, the GDPR obviously allows for legal requirements of record keeping. It does however require you to delete the data once those requirements no longer apply (i.e. you have to ensure data is still deleted after those several years, you can't just not delete it because the deletion time is years in the future).
- lazide 4y ago
  This should have the asterisk of * after the period of time you’ve said it will take for it to be deleted. It’s almost impossible to guarantee instant delete, folks just care if it’s gone in a documented and/or reasonable amount of time, and predictably. Correct?
  - avianlyric 4y ago
    Sure, but that time period is usually limited to 30 days, with an additional 30 day extension allowed if you contact the person and tell them you need more time. You can’t just declare that it take years to delete data, GDPR sets reasonable limits on how long a company can delay true deletion.
    - LunaSea 4y ago
      The GDPR gives limits on answering, I'm not sure it gives a limit on actually deleting the data everywhere. Various types of data have various retention times for various reasons, some being legal reasons.
    - EUROCARE 4y ago
      Even if the company just issued you an invoice, that makes it obligated by law to hold onto your PII for the next 5 years - and GDPR of course has an exception for that.
      - avianlyric 4y ago
        Sure, but that data doesn’t fall under your right-to-be-forgotten, so any obligations that apply to the right-to-be-forgotten are irrelevant.
    - yuliyp 4y ago
      The GDPR says it's 30 days with a 2 month extension. This allows for having safe soft deletion and backup retention policies without running afoul of it.
      - 4y ago
- astura 4y ago
  >under the GDPR [soft delete] is illegal This is obviously not always true. Any European can't, for example, delete their online account with their mortgage company and demand that the mortgage company deletes the records saying that they owe them money so they can get their house for free. Nor can anyone call up their previous employer and require them to delete all the work they ever did from their company's computer systems. A company can't respond to a tax audit with "well we lost a billion dollars this year but all our customers wanted their accounts deleted so we can't provide any documentation of it." I mean, you aren't even going to get to the audit if you can't fill out your taxes in the first place because you had to delete all your financial records. Most computer systems/software products in existence aren't free media apps, yet the "privacy circlejerk" constantly talk in generalities as if they are. I'm sick of this, most applications have a business justification for using some form of soft delete for at least some items.
  - he0001 4y ago
    > Nor can anyone call up previous employer and require them to delete all the work they ever did from their company’s computer systems. You can’t delete IP as this is not covered by GDPR, but you sure can ask them to delete your identification data from their records as GDPR also works for employees not just customers. It’s a problem with for example vcs. Unless there’s another law in ruling, GDPR is the baseline. It’s for any identification of an individual, regardless of usage. Most “media apps” don’t have any other law that’s in effect, so GDPR is mandating what you need to do. (IANAL but worked on implementing GDPR with lawyers).
    - jsnell 4y ago
      The right to erasure to is not unlimited (few things in GDPR are as absolute as people imagine). Yes, is personal data is retained based on consent, that consent can be withdrawn and the data must be deleted. But there are many other bases for collecting and retaining data as well. The examples in the GP were mostly around the data being necessary for fulfilling a contract (can't request deletion of your mortgage records) or meeting legal obligations (can't delete tax records). But there's also legitimate interest. Legitimate interest is vague and open to interpretation, but I'd be really surprised if there were any decisions from a DPA suggesting that a VCS commit log would be problematic. The legitimate interest seems really strong in that case: being able to audit who actually made each change, e.g. for security reasons.
- EUROCARE 4y ago
  GDPR explicitly says that this doesn't apply when a more specific law requires you to store something - logically then you also need to know who the user was. Things like transaction records, network traffic logs, invoices, pay slips, workplace chat... So yeah, you're right that not deleting user data just because it's inconvenient is wrong and illegal, but that doesn't mean there aren't legitimate usecases for soft deletions of users.
- zrobotics 4y ago
  What we did (and legal was OK with it) was to create a "GDPR Delete" function in the admin backend. It doesn't actually delete, but sets a soft delete identical to the regular user delete function and overwrites any PII with stuff like name:gdpr_delete_name_xxxx That way log and system context is available for debugging, but the system no longer holds any PII. It does delete some logs, but anonymizing data seemed better than hard deletion. It wasn't hard to do, it took legal longer to approve than writing and testing the code and has come in handy a few times for debugging errors. Edit: typo
- spamizbad 4y ago
  Any reasonably big system will require you to invest in that anonymization. Doing cascading hard deletes and making sure any missing references don’t trip up your system or mess up reporting is much more difficult than just flagging fields that need to be scrambled with randomized data (most libraries have fake data generators that make doing these updates easy) Long before GDPR was a thing we had this implemented allowing us to securely analyze production dumps. This is one of those things that I think libraries need to mature to support and then it will be effortless. Right now it requires a modest, but hardly overwhelming, engineering effort to implement.
- pmarreck 4y ago
  What if you anonymize it with a random key and then give the user that key in the event they want to undelete their deletion?
- loeg 4y ago
  Not every delete is a GDPR delete. You could have a hard-delete GDPR flow, obviously.
- avereveard 4y ago
  Not all personal information storage needs consent, gdpr art 6 comma b and c cover such cases as applicable to private companies. Users can retract willing consent, but willing consent is not the only thing that allow companies to store user data. Soft deletion of users whom had an account with transactions on it is fine Art 17 comma b explain this quite clearly Gdpr is amazingly readable.
- petre 4y ago
  We don't do it for users but for certain user assets that are a pain to remove consistently without a background job or a job queue. Like user devices that have associated dependencies. The background job is automatic, it prunes objects evey x hours.
- pmontra 4y ago
  Last month I had to implement customer deletion on a new project I started working on recently. The basics were: anonymize the user personal data, delete subscriptions, keep anything else. I believe that payments and invoices are not stored in that database so I don't know what happens to them but in general you must not delete that kind of data because GDPR notwithstanding the law requires that they are available for inspection for a long time. Of course they cannot be anonymized in a way that cannot be reversed. Being able to connect them with a trail of operations in the main database is probably useful too. So, soft delete.
- Groxx 4y ago
  You can do both. Soft delete immediately when requested. Hard delete (if still soft deleted) when GDPR requires it. Besides it's not like all data is the same. Use what's appropriate when it's appropriate. Nobody's saying "you must always soft delete, and then immediately tweet the audit trail, print that tweet, put that printout onto a rocket, and launch it to the moon for later retrieval. with a blockchain for extra GDPR hate".
- kgeist 4y ago
  >It may be convenient, but under the GDPR is illegal. When an user deletes an account, all the personal data associated with that user must be deleted There's a grace period of 30 days. We hard-remove after 2 weeks allowing a user to change their mind if they deleted it by accident. More than enough.
Akronymus 4y ago
I messed up a mass update query enough times to leave myself SOME provision to undo it. The exceptions are when there is a well tested query that affects a single account or something. Like GDPR
coldtea 4y ago
>I've never undeleted a user either, but there have been many times I've gone back to look at something. I, for one, have undeleted things tons of times, taking them of the trash can before emptying it, undoing the delete action (in apps where this is possible), and so on.
JohnBooty 4y ago
I think this is the part they miss. I've never undeleted a user either, but there have been many times I've gone back to look at something. Yeah. As far as a user-facing "Undelete" button existing or being used... that's very rare in my experience. What's much more common is a user accidentally deletes some data. They deny they made an error. The developers are blamed. You then have to go on a wild goose chase figuring out if it was possible for the app to actually screw up in that way. There's usually no definitive answer, and even if there is, management can't understand it. And regardless of how any of that plays out, you still probably have to try and recover the data from backups or something. Alternately, maybe it was the app's fault. Still plays out nearly the same! Soft deletes and/or audit trails save you from all of that. Though you really shouldn't be relying on a database for an audit trail. It might help find some issues, but things actually used for security shouldn't be writable so easily. I mean, at some level you need to trust the database right? Been ages since I did it, but it's usually possible to set up a "secure" audit trail with use of database permissions. For example, the application's DB credentials can have SELECT and INSERT permissions on the audit trail table, but no UPDATE or DELETE perms. How would you set up a secure audit trail that didn't rely on the application and/or database at some level? Even if it lives outside of the database, that data came from the database. Not a rhetorical question. Genuinely curious!
- gmiller123456 4y ago
  How would you set up a secure audit trail that didn't rely on the application and/or database at some level? A database is fine for an audit trail. But the application shouldn't have permissions to cover up a change. Likewise, the audit trail should record that a record was created, deleted, maybe undeleted, etc and when, by who. Relying only on the "deleted" flag only shows you the current state, not what and when was done and who did it.
  - sverhagen 4y ago
    Any tools you recommend here? Or just roll your own with triggers in the database?
deleted 4y ago
[deleted]
pustan 4y ago
>The author uses the "no one ever undeleted anything" as the primary justification. I think this is the part they miss. But did they though? "Although I’ve never seen an undelete work in practice, soft deletion wasn’t completely useless because we would occasionally use it to refer to deleted data – usually a manual process where someone wanted to see to a deleted object for purposes of assisting with a support ticket or trying to squash a bug."

alerighi 4y ago

It may be convenient, but under the GDPR is illegal. When an user deletes an account, all the personal data associated with that user must be deleted (or anonymize it in a way that it's no longer possible to associate it back to the particular user). You cannot just keep user information forever "just in case" they are useful again.

changoplatanero 4y ago
User deleting an account is just one way that stuff gets deleted. There are other deletion scenarios where it is appropriate to keep the information after its deleted.
- Akronymus 4y ago
  Yeah. Basically anything that an employee can mess up, should be reversible IMO. But actions such as deleting an account should have the option of "YES, DELETE IT PERMANENTLY, THIS CAN'T BE UNDONE"
  - closeparen 4y ago
    Often it’s mark for deletion after some number of days.
  - charcircuit 4y ago
    The only way that would make sense in if it was a cancelable timed deletion. People make mistakes. People change their minds. People get hacked.
  - Gigachad 4y ago
    You probably want to put the account on hold for a period where it can be undone. And then clean it up after.
    - Akronymus 4y ago
      Not always possible in a legal way. For example, GDPR compliance.
      - Gigachad 4y ago
        GDPR gives you a pretty large period to do it within. Easily enough to fit this feature in.
smolder 4y ago
I wonder what percentage of companies who even appear to comply with deletion requests actually do full deletion in practice. I suspect it's small, knowing how many things are coded to fake-delete for convenience. Businesses also tend to keep cold data backups around. (Maybe backups are exempted? I don't know.) There might even be cases where ostensibly deleted data can still be recovered from a disk, if they haven't overwritten.
- avianlyric 4y ago
  Guess we’ll find out when a leaks happen, and companies start getting fines. GDPR does differentiate between structured data (I believe it uses the term “identifiable records” or similar), and huge heap of unstructured data where an individuals data can’t be quickly retrieved as it’s own atomic unit. With much stricter requirements for anything structured. So for data on a HDD that could be recovered, but is an unstructured mess. You’re probably ok, as long as you took reasonable steps to protect the data, like full disk encryption (which could occurs below the file system, and thus allow you to recover data deleted from the FS with a full disk scan, as long as you still had the keys). If you just had peoples data unencrypted on a HDD, and didn’t securely erase it before dispose, then that’s probably still a GDPR violation.
  - JohnBooty 4y ago
    What does GDPR say about obscuring data? Instead of hard-deleting a structured record for Bob Smith, can you leave the record intact and scrub it of identifying data so that I don't e.g. break all of the records for orders that Bob Smith made?
    - alerighi 4y ago
      You can as long as you delete all the personal data of the user and make sure that other remaining data is anonymized in a way that you can no longer associate it with the user.
    - deleted 4y ago
      [deleted]
dataflow 4y ago
It's illegal but companies don't necessarily care to avoid soft deletes regardless. I think companies wait to get sued so they can try to argue in court why their soft deletions are reasonable and why it's too technically difficult for them to do hard deletes. To be honest, in the age of modern overprovisioned storage drives that remap blocks frequently, I'm not really sure you can implement genuine "hard" deletes without choosing significantly unorthodox hardware (or destroying a drive every time you need a single bit erased), no matter how much you want to in software. One of those details that I'm both surprised and unsurprised doesn't seem to have been addressed legally. I feel like a court ought to at least buy this aspect of the argument, so maybe they'll buy that it can be difficult in terms of the software too? Who knows. My guess is that a reasonable court would accommodate something that's reasonable for a given company, but there are lots of variations that could fall into that category.
- jcheng 4y ago
  GDPR fines are structured to make it extremely risky/expensive to take the “ask for forgiveness not permission” approach you’re suggesting. I think for this reason, the fine is designed to scale with the size of the offending company’s revenue, not so much with the actual damages suffered by anyone. Take a look at some of these fines, tens of millions of euros cause the UI for your cookie consent dialog is poorly designed? https://www.tessian.com/blog/biggest-gdpr-fines-2020/ https://www.tessian.com/blog/biggest-gdpr-fines-2020/ When the GDPR rules came out our company (pretty small at the time) pulled out all the stops to get ourselves into compliance, due to the above and other teeth put into the regulations. It seemed utterly irrational not to. Were we the exception, or the rule?
  - dataflow 4y ago
    I don't know if you were the exception or the rule, but I know for a fact some do go the forgiveness approach for requirements that they believe would be disproportionately burdensome. Probably almost all companies actually, including yourself (just to a different extent) - what did you do about the storage remapping thing I mentioned? Did it come up/did you guys discuss it? Do you believe you're in compliance despite your hardware (most likely) not guaranteeing erasure or overwriting of existing data? I'm curious how your assessment of that went, because I doubt one can be in strict compliance without guarantees from the hardware.
afiori 4y ago
I would guess that there are quite a few limits to that... A user has a long history of participating on your forum and other users have quoted their messages far and wide. Collectively all of the messages posted on your forums (with or without timestamps) reveal a few PII about the user. Do you have to delete those? The user filed a bug report about a functionality not working, do you have to delete the text of the bug report? Arguably if your user table look like [user_id, creation_date, deletion_date, status, account_type] then this table does not contain any PII. Assuming that user content is not automatically PII, whose responsibility is it to track where PII can be? Sometimes users doxx themselves (like mistakenly sharing tax return forms instead cat pics), in such a case it is the user responsibility to signal this to you. If the user filed an issue saying "when I insert my name (Abe Cox) the input validation fails" is it you or the user that need to read through all the issues to find this case? My point is that GDPR + right to be forgotten cannot make it look like you never had an account at all, especially without user assistance.
- hnbad 4y ago
  user_id is PII if the identity of a user can be narrowed down sufficiently. For example, if you created a social media app and later left the company, it could still be determined that if you had an account it likely has one of the lowest user ID values if those were assigned sequentially. So even if your name was removed from it and none of your posts provided any identifying information about you, the user ID could still qualify as PII. Likewise if the user ID is public and non-PII content associated with that ID is linked elsewhere identifying the author of that content (e.g. a news article embedding a post) the identity of the author is still compromised. The problem is not just that users may submit PII in unexpected places, the problem is that even if the data type could not possibly allow them to, metadata beyond your control may actually still taint your supposedly anonymous data.
  - afiori 4y ago
    The question is not whether "spooky deanonymization as a distance" is possible; the question is how much you should proactively do to avoid this. As an extreme example, if user A takes a screenshot if their own user setting page on a image sharing site, then sends it to user B that uploads it to the same site. Who is responsible for this piece of data? does share-a-pic.web need to scan all images for the username of a user that is deleting an account? This is a convoluted example where the answer is obviously no, but it shows that a purely principled answer might be unsuitable. Especially since non PII data you own mixed with non PII data someone else owns can become PII and you do not know what data others have. (you could also replace the user ID in all locations with a new tagged UUID to preserve referential integrity in the DB, but there not much more than 2^32 humans in total, it s very hard not to be deanonymizable)
411111111111111 4y ago
Stop spreading misinformation. Nobody is required to hard delete the very second the [delete] button is pressed. You're following the law as long as that mentioned reaper job exists and runs at least once per month, to be safely within the allowed timeframe
- brigandish 4y ago
  I didn't see a claim that deletion had to be "the very second the [delete] button is pressed". This is the actual text of the law[1]: > The data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay There is a one month limit of the right of access[2] and right to be informed[3], which flow into some of the articles of the law; and certain articles also mention a month's limit on being informed, e.g. from article 12 parts 3 and 4 [4]: > The controller shall provide information on action taken on a request under Articles 15 to 22 to the data subject *without undue delay and in any event within one month* of receipt of the request. That period may be extended by two further months where necessary, taking into account the complexity and number of the requests. The controller shall inform the data subject of any such extension within one month of receipt of the request, together with the reasons for the delay. Where the data subject makes the request by electronic form means, the information shall be provided by electronic means where possible, unless otherwise requested by the data subject. If the controller does not take action on the request of the data subject, the controller shall inform the data subject without delay and at the latest within one month of receipt of the request of the reasons for not taking action and on the possibility of lodging a complaint with a supervisory authority and seeking a judicial remedy. I've marked out those phrases because they show the language of the law that you will see again and again in the articles - without undue delay and within one month. The latter does not obviate the former. Hence, and in fact, there is no part of the law that explicitly gives you a month to delete something. If you could delete something right now but you leave it hanging around because you have arbitrarily chosen a job to run once a month and there's a leak in that time, that would be undue delay. [1] https://gdpr-info.eu/art-17-gdpr/ https://gdpr-info.eu/art-17-gdpr/ [2] https://gdpr-info.eu/issues/right-of-access/ https://gdpr-info.eu/issues/right-of-access/ [3] https://gdpr-info.eu/issues/right-to-be-informed/ https://gdpr-info.eu/issues/right-to-be-informed/ [4] https://gdpr-info.eu/art-12-gdpr/ https://gdpr-info.eu/art-12-gdpr/
twelve40 4y ago
You can most definitely keep around the non-PII data you've generated for that user in your app, such as synthetic user ID's and other stuff. Furthermore, for some apps (fintech) in some countries (US) you are actually obligated to keep around certain information for several years. It really doesn't have to be all or nothing.
- bpicolo 4y ago
  Insurance has to keep around info for the lifetime of the policy. For some products, that's decades.
- hnbad 4y ago
  Synthetic user IDs can still qualify as PII since a human may still know which person that ID was associated with and data associated with that ID may be sufficient to narrow down who it was even if no human (or system) knows the exact mapping. Pseudonymous data is not anonymous data. But yes, the GDPR obviously allows for legal requirements of record keeping. It does however require you to delete the data once those requirements no longer apply (i.e. you have to ensure data is still deleted after those several years, you can't just not delete it because the deletion time is years in the future).
lazide 4y ago
This should have the asterisk of * after the period of time you’ve said it will take for it to be deleted. It’s almost impossible to guarantee instant delete, folks just care if it’s gone in a documented and/or reasonable amount of time, and predictably. Correct?
- avianlyric 4y ago
  Sure, but that time period is usually limited to 30 days, with an additional 30 day extension allowed if you contact the person and tell them you need more time. You can’t just declare that it take years to delete data, GDPR sets reasonable limits on how long a company can delay true deletion.
  - LunaSea 4y ago
    The GDPR gives limits on answering, I'm not sure it gives a limit on actually deleting the data everywhere. Various types of data have various retention times for various reasons, some being legal reasons.
  - EUROCARE 4y ago
    Even if the company just issued you an invoice, that makes it obligated by law to hold onto your PII for the next 5 years - and GDPR of course has an exception for that.
    - avianlyric 4y ago
      Sure, but that data doesn’t fall under your right-to-be-forgotten, so any obligations that apply to the right-to-be-forgotten are irrelevant.
  - yuliyp 4y ago
    The GDPR says it's 30 days with a 2 month extension. This allows for having safe soft deletion and backup retention policies without running afoul of it.
    - 4y ago
astura 4y ago
>under the GDPR [soft delete] is illegal This is obviously not always true. Any European can't, for example, delete their online account with their mortgage company and demand that the mortgage company deletes the records saying that they owe them money so they can get their house for free. Nor can anyone call up their previous employer and require them to delete all the work they ever did from their company's computer systems. A company can't respond to a tax audit with "well we lost a billion dollars this year but all our customers wanted their accounts deleted so we can't provide any documentation of it." I mean, you aren't even going to get to the audit if you can't fill out your taxes in the first place because you had to delete all your financial records. Most computer systems/software products in existence aren't free media apps, yet the "privacy circlejerk" constantly talk in generalities as if they are. I'm sick of this, most applications have a business justification for using some form of soft delete for at least some items.
- he0001 4y ago
  > Nor can anyone call up previous employer and require them to delete all the work they ever did from their company’s computer systems. You can’t delete IP as this is not covered by GDPR, but you sure can ask them to delete your identification data from their records as GDPR also works for employees not just customers. It’s a problem with for example vcs. Unless there’s another law in ruling, GDPR is the baseline. It’s for any identification of an individual, regardless of usage. Most “media apps” don’t have any other law that’s in effect, so GDPR is mandating what you need to do. (IANAL but worked on implementing GDPR with lawyers).
  - jsnell 4y ago
    The right to erasure to is not unlimited (few things in GDPR are as absolute as people imagine). Yes, is personal data is retained based on consent, that consent can be withdrawn and the data must be deleted. But there are many other bases for collecting and retaining data as well. The examples in the GP were mostly around the data being necessary for fulfilling a contract (can't request deletion of your mortgage records) or meeting legal obligations (can't delete tax records). But there's also legitimate interest. Legitimate interest is vague and open to interpretation, but I'd be really surprised if there were any decisions from a DPA suggesting that a VCS commit log would be problematic. The legitimate interest seems really strong in that case: being able to audit who actually made each change, e.g. for security reasons.
EUROCARE 4y ago
GDPR explicitly says that this doesn't apply when a more specific law requires you to store something - logically then you also need to know who the user was. Things like transaction records, network traffic logs, invoices, pay slips, workplace chat... So yeah, you're right that not deleting user data just because it's inconvenient is wrong and illegal, but that doesn't mean there aren't legitimate usecases for soft deletions of users.
zrobotics 4y ago
What we did (and legal was OK with it) was to create a "GDPR Delete" function in the admin backend. It doesn't actually delete, but sets a soft delete identical to the regular user delete function and overwrites any PII with stuff like name:gdpr_delete_name_xxxx That way log and system context is available for debugging, but the system no longer holds any PII. It does delete some logs, but anonymizing data seemed better than hard deletion. It wasn't hard to do, it took legal longer to approve than writing and testing the code and has come in handy a few times for debugging errors. Edit: typo
spamizbad 4y ago
Any reasonably big system will require you to invest in that anonymization. Doing cascading hard deletes and making sure any missing references don’t trip up your system or mess up reporting is much more difficult than just flagging fields that need to be scrambled with randomized data (most libraries have fake data generators that make doing these updates easy) Long before GDPR was a thing we had this implemented allowing us to securely analyze production dumps. This is one of those things that I think libraries need to mature to support and then it will be effortless. Right now it requires a modest, but hardly overwhelming, engineering effort to implement.
pmarreck 4y ago
What if you anonymize it with a random key and then give the user that key in the event they want to undelete their deletion?
loeg 4y ago
Not every delete is a GDPR delete. You could have a hard-delete GDPR flow, obviously.
avereveard 4y ago
Not all personal information storage needs consent, gdpr art 6 comma b and c cover such cases as applicable to private companies. Users can retract willing consent, but willing consent is not the only thing that allow companies to store user data. Soft deletion of users whom had an account with transactions on it is fine Art 17 comma b explain this quite clearly Gdpr is amazingly readable.
petre 4y ago
We don't do it for users but for certain user assets that are a pain to remove consistently without a background job or a job queue. Like user devices that have associated dependencies. The background job is automatic, it prunes objects evey x hours.
pmontra 4y ago
Last month I had to implement customer deletion on a new project I started working on recently. The basics were: anonymize the user personal data, delete subscriptions, keep anything else. I believe that payments and invoices are not stored in that database so I don't know what happens to them but in general you must not delete that kind of data because GDPR notwithstanding the law requires that they are available for inspection for a long time. Of course they cannot be anonymized in a way that cannot be reversed. Being able to connect them with a trail of operations in the main database is probably useful too. So, soft delete.
Groxx 4y ago
You can do both. Soft delete immediately when requested. Hard delete (if still soft deleted) when GDPR requires it. Besides it's not like all data is the same. Use what's appropriate when it's appropriate. Nobody's saying "you must always soft delete, and then immediately tweet the audit trail, print that tweet, put that printout onto a rocket, and launch it to the moon for later retrieval. with a blockchain for extra GDPR hate".
kgeist 4y ago
>It may be convenient, but under the GDPR is illegal. When an user deletes an account, all the personal data associated with that user must be deleted There's a grace period of 30 days. We hard-remove after 2 weeks allowing a user to change their mind if they deleted it by accident. More than enough.

Akronymus 4y ago

I messed up a mass update query enough times to leave myself SOME provision to undo it. The exceptions are when there is a well tested query that affects a single account or something. Like GDPR

coldtea 4y ago

>I've never undeleted a user either, but there have been many times I've gone back to look at something. I, for one, have undeleted things tons of times, taking them of the trash can before emptying it, undoing the delete action (in apps where this is possible), and so on.

JohnBooty 4y ago

I think this is the part they miss. I've never undeleted a user either, but there have been many times I've gone back to look at something. Yeah. As far as a user-facing "Undelete" button existing or being used... that's very rare in my experience. What's much more common is a user accidentally deletes some data. They deny they made an error. The developers are blamed. You then have to go on a wild goose chase figuring out if it was possible for the app to actually screw up in that way. There's usually no definitive answer, and even if there is, management can't understand it. And regardless of how any of that plays out, you still probably have to try and recover the data from backups or something. Alternately, maybe it was the app's fault. Still plays out nearly the same! Soft deletes and/or audit trails save you from all of that. Though you really shouldn't be relying on a database for an audit trail. It might help find some issues, but things actually used for security shouldn't be writable so easily. I mean, at some level you need to trust the database right? Been ages since I did it, but it's usually possible to set up a "secure" audit trail with use of database permissions. For example, the application's DB credentials can have SELECT and INSERT permissions on the audit trail table, but no UPDATE or DELETE perms. How would you set up a secure audit trail that didn't rely on the application and/or database at some level? Even if it lives outside of the database, that data came from the database. Not a rhetorical question. Genuinely curious!

gmiller123456 4y ago
How would you set up a secure audit trail that didn't rely on the application and/or database at some level? A database is fine for an audit trail. But the application shouldn't have permissions to cover up a change. Likewise, the audit trail should record that a record was created, deleted, maybe undeleted, etc and when, by who. Relying only on the "deleted" flag only shows you the current state, not what and when was done and who did it.
- sverhagen 4y ago
  Any tools you recommend here? Or just roll your own with triggers in the database?

deleted 4y ago

[deleted]

pustan 4y ago

>The author uses the "no one ever undeleted anything" as the primary justification. I think this is the part they miss. But did they though? "Although I’ve never seen an undelete work in practice, soft deletion wasn’t completely useless because we would occasionally use it to refer to deleted data – usually a manual process where someone wanted to see to a deleted object for purposes of assisting with a support ticket or trying to squash a bug."