3 ms·
Durability often is overrated. E.g. lets say your system is storing billing records for an API. Customers won't complain about being under-charged, so a loss
by vidarh 4y ago
Durability often is overrated.
E.g. lets say your system is storing billing records for an API. Customers won't complain about being under-charged, so a loss of data for a system like that is fine as long as reducing the durability saves more money than it costs in losts ability to charge your customers. E.g. if rolling up and syncing summaries to disk every 10 seconds were to on average lose you 0.1% of records but cut your server cost for that system by 80%, it's a simple question of whether those server costs are higher or lower than .1% of your revenue (all numbers taken out of thin air).
We make sacrifices in terms of both consistency and durability and all kinds of things all the time. The problem is that we often don't take the time to analyse what kind of inconsistency and loss is acceptable from a business point of view before designing a solution that often tries to aim for a level of consistency or durability that may not be needed.
You can afford to sacrifice all kinds of guarantees as long as you have reliable models for how sacrificing them impacts you and your customers so that you know the sacrifices represent acceptable tradeoffs. The risk you're taking is that you're sacrificing safety margins, and so it's something you should only do if you understand both the domain and your systems failure modes very well.