3 ms·

The problem is - imagine microservices - that data does not exist in one spot. And chances are no one actaually knows 100% where the data lives. It probably liv

The problem is - imagine microservices - that data does not exist in one spot. And chances are no one actaually knows 100% where the data lives. It probably lives in a prod db, an ETL data lake type platform (or two/three - and god knows if that has any kind of identifier to actually delete it) and chances are if you are big enough some 3rd party systems. So even if you delete it from prod, it still exists somewhere. In a perfect world there would be some way to snap your fingers and delete it from every system - but we do not live in a perfect world. There is absolutely no incentive to build systems with this kind of requirement in mind. It's a waste of time and effort. Europeans will say "but hey wait! GDPR!" meanwhile the world keeps spinning and no one gives a shit.

geoka9 2y ago
The big scary fine is a absolutely an incentive, unless you are huge company with an army of lawyers on standby. Some customers I've worked with submit GDPR requests on behalf of their users to all of their software vendors; there're even SaaS products for doing exactly that. They also get you to sign documentation to that extent if you want to keep them as a customer.
Eisenstein 2y ago
I don't understand. If they want the ability to ever update the data how do they expect to have it propagate if these information systems aren't connected? And if they are connected, why would 'delete entry' be harder than 'update entry'?
- photonthug 2y ago
  You’re assuming the data is actually handled with care, and lives in a system designed for this sort of thing, but it’s probably not. Some data scientist who wasn’t even supposed to be able to view stuff managed to copy data from prod to dev, maybe by accident. Then an engineer who intended to be testing an in place transform pipeline actually wrote a copy elsewhere, maybe once for each test, and didn’t even notice. This stuff happened off and on for like fifteen years while projects and people all changed and no one cares, because sure there’s some vague gestures in the direction of compliance, security, and privacy but it’s just a token effort because anyone who really starts asking questions is let go for not being a team player. Syncing updates to some “main” copy won’t make it easy to delete dupes.
  - al_borland 2y ago
    Not being able to delete every possible record that may have been used during testing doesn’t mean they shouldn’t delete the data they do know about in the main system. If I can change my username, password, or email address… and those functions work. A delete should also work. Why a prod db with sensitive user info is being used in testing is a whole separate issue. Those can be deleted in their entirety.

geoka9 2y ago

The big scary fine is a absolutely an incentive, unless you are huge company with an army of lawyers on standby. Some customers I've worked with submit GDPR requests on behalf of their users to all of their software vendors; there're even SaaS products for doing exactly that. They also get you to sign documentation to that extent if you want to keep them as a customer.

Eisenstein 2y ago

I don't understand. If they want the ability to ever update the data how do they expect to have it propagate if these information systems aren't connected? And if they are connected, why would 'delete entry' be harder than 'update entry'?

photonthug 2y ago
You’re assuming the data is actually handled with care, and lives in a system designed for this sort of thing, but it’s probably not. Some data scientist who wasn’t even supposed to be able to view stuff managed to copy data from prod to dev, maybe by accident. Then an engineer who intended to be testing an in place transform pipeline actually wrote a copy elsewhere, maybe once for each test, and didn’t even notice. This stuff happened off and on for like fifteen years while projects and people all changed and no one cares, because sure there’s some vague gestures in the direction of compliance, security, and privacy but it’s just a token effort because anyone who really starts asking questions is let go for not being a team player. Syncing updates to some “main” copy won’t make it easy to delete dupes.
- al_borland 2y ago
  Not being able to delete every possible record that may have been used during testing doesn’t mean they shouldn’t delete the data they do know about in the main system. If I can change my username, password, or email address… and those functions work. A delete should also work. Why a prod db with sensitive user info is being used in testing is a whole separate issue. Those can be deleted in their entirety.