5 ms·
"The concept behind soft deletion is to make deletion safer, and reversible." That's one part. The other part is that in many industries you have regulatory da
by giantg2 4y ago
"The concept behind soft deletion is to make deletion safer, and reversible."
That's one part. The other part is that in many industries you have regulatory data retention and audit requirements. This is arguably the most valuable and common reason to perform Logical deletes.
- dexwiz 4y agoI think billions have been spent bridging the gap between “ideal” software and what businesses actually need. Access control is another thing I see developers wanting to simplify or push to implement later, but is actually a key feature.
- dubswithus 4y agoThere's definitely perfect and perfect for the business. But when large companies have many developers they all have to do something. They will spend their time doing something unnecessary.
- jewayne 4y agoI would argue that in many cases the concept behind soft deletion is to make deletion permanent. Hard deletes retain no memory of what you wanted to be gone, so any malfunctioning sync process will continuously recreate the deleted record soon after it's deleted. Soft deletes are often the only way to make sure deleted records don't reappear.
- eeperson 4y agoCouldn't a malfunctioning sync process undo a soft delete as well?
- deleted 4y ago[deleted]
- jayd16 4y agoNull is more ambiguous than an explicit conflict. If there is literally no record, even of the delete action then there's no timestamp for last write wins.
- trinovantes 4y agoAssuming you're using a modern database, replication is done with paxos/raft and they are formally proven to not allow this to happen as both edits/deletions are both just entries in distributed event log.
- silisili 4y agoWhy not just use an audit table, to keep from littering your indices?
- jacobr1 4y agoYou still need a record of the data, which probably isn't fully captured in the audit table
- jelkand 4y agoAn audit table is a similar, but not entirely equivalent tool. There are circumstance where both audit tables / soft deletes are appropriate, and where only one of the two is. Other systems that work are append-only tables, event driven systems, and I'm sure there are more that I'm not aware of.
- brtkdotse 4y agoIn banking and bookkeeping, there’s no such thing as a “delete”. Once something is in the ledger you can’t undo it - you have to make a new entry that negates the old one.
- hammock 4y agoIs the same true in mass surveillance?
- i_hate_pigeons 4y agonot every thing is a transaction though, say a deleted piece of ref data that still has FKs to other stuff etc
- Hallucinaut 4y agoThese are still versioned in many serious finance systems
- munk-a 4y ago
- deleted 4y ago[deleted]
- jiggywiggy 4y agoHa, and then there is the opposite regulation that you have to delete user data.
- MonkeyMalarky 4y agoWell you see, you need a complete record of when it was created, every change that occurred, everyone who could view it and also log every access attempt. But it. You're not actually supposed to keep it. Just everything surrounding it.
- scott_w 4y agoNot quite. GDPR (and equivalents) have clear escape hatches to allow you to store data if you have good reason (even if the data subject requests its removal). Invoices, from the article, is a great example. That record must remain unchanged in most financial regulations. I’d wager a customer sending a deletion request for invoices will be met with raucous laughter from the legal and finance teams.
- giantg2 4y agoIn some industries the retention regulations trump the deletion ones. I believe finance is one area where they will delete some of your data, but are still required to maintain 7 years of specifically listed data.
- MonkeyMalarky 4y agoThen there's always the joy of a situation where your client is being sued by one of their clients and now needs your help recovering everything possible from your platform. And you're going to help them because you'd like to keep them as a client rather than let them be sued into oblivion.
- nirvdrum 4y agoMy experience at a few start-ups has been that account deletion just isn't prioritized. It's not a focus when building an MVP. If the application ever gains traction, everyone is then terrified they'll accidentally delete customer data that they never delete anything. It's a shame. As a user, when I delete my account or data in my account, I want you to permanently delete it, not keep it around and just make inaccessible to me.
- jrockway 4y agoIf it makes you feel better, the startups that don't have time to delete your data probably don't have a viable disaster recovery plan either. As we learned from the Atlassian snafu, even giant companies with billions in revenue often can't recover from disasters. (I try to test mine every 6 months. I've never had a test go perfectly.)
- tomcam 4y agoGood to know. What do you were tests look like?
- nerdponx 4y agoI've also seen businesses retain "deleted" data in order to support legitimate data analysis work in the future. And it actually can help significantly. Maybe scrubbing PII from deleted accounts is a good idea, but those deleted accounts are perfectly good data points, especially in smaller/newer companies with lower-volume data streams.
- nirvdrum 4y agoIf you want to retain anonymous statistics, fine. I'm not keen on you retaining my actual data so you can monetize it after our business relationship has concluded. The biggest thing a small team can learn is why they failed to retain a customer or a trial that didn't convert. For that, usage metrics are considerably more valuable than user data.
- dubswithus 4y ago
- necovek 4y agoAnd yet another part is making deletes (appear) instantaneous: useful when it involves cleaning up a bunch of "related" data possibly living on different services (eg. S3, ES...). This also helps with the original goal of making them safer by manually implementing "eventual consistency" for data living outside the transactional world.
- taeric 4y agoDon't make deletes appear instantaneous? If you have heavy weight systems, then it makes sense for provisioning and deleting entities is a process, that should be open to monitoring.
- necovek 4y agoExposing details of long lived background processes, and especially deletion processes to users (who in many cases couldn't care less) is a lot of work that's probably not worth it for the rare "hey, something went wrong" case — it's usually perfectly acceptable to raise that with the support team and let them use internal tools to debug and investigate. I am sure there are cases where it is worthwhile, but most of the times I've hit this in common web apps, it wasn't.
- taeric 4y agoI disagree. Any system that doesn't transition an entity to "deleting" is almost certainly going to be a pain point for me when I go to delete something. This can be seen as related to soft deletes. But I consider it more marking intent in the system. Lets you make "delete" a simple field update that will be carried out by the backend.
- necovek 4y agoSay you are attempting to delete a tweet you made: you click on the delete button, and you want to see that your tweet is in the process of being deleted, but not really deleted yet (as the platform goes and updates all the retweet references and such)? Really? What value do you as a user get from knowing the details of this transitory state? Do you also want to see a progress bar of how many references have been updated, what's the progress of evicting the tweet from search indexes, etc? What's wrong with all this appearing instantaneous and actual delete happening in the background over 5 minutes or 5 hours? It seems you are in favour of this approach, just don't want to call it "soft delete" (since it may be followed by a hard out-of-band delete).
- taeric 4y agoThis is almost certainly going to bite you if you don't push all customer identification data out of your main data stores. And it will go a long way to making your services harder to use if you don't allow users to associate friendly names with things. And to assume that the same friendly name will be used for a future item. (For example, if you name devices based on the room you put them in. Is reasonable to think that when you replace a device, that you are likely to want to reuse the name.)
- yen223 4y agoNowadays, regulations goes both ways. Data sovereignty regulations and GDPR-like laws can mandate that data must get hard-deleted.