3 ms·
Most problems with overgrowing data can be fixed by having data deletion rule. Many people just keep adding data and think "maybe it will be useful in future"
by faangguyindia 5mo ago
Most problems with overgrowing data can be fixed by having data deletion rule.
Many people just keep adding data and think "maybe it will be useful in future" till their system goes down.
Many of your data is essentially useless for anything in future.
You can simply have data retention policy and for most app this ensures your data does not grow top huge
- redwood 5mo agoThis is orthogonal to many classes of write throughput. And in fact in many cases deletes themselves are effectively adding to the write workload
- hasyimibhar 5mo agoIn some cases you have no choice but to retain the data, e.g. due to compliance. But the good thing is it doesn't have to be in Postgres. You can periodically offload data to a lakehouse, then delete it from Postgres. If the table is partitioned, delete should be cheap. I'm guessing with Neon, since their storage is a lakehouse, you get this for free.