3 ms·
A few days ago I posted my frustration with Cassandra counters, this article articulates a solution elegantly: 'if you make a mistake or something goes wrong i
by iandanforth 15y ago
A few days ago I posted my frustration with Cassandra counters, this article articulates a solution elegantly:
'if you make a mistake or something goes wrong in the realtime layer, the batch layer will correct it.'
The resiliency of this protection however is limited by the time it takes to do a full re-processing of the historical data. If there is a processing mistake in your real-time layer, it is likely to exist in your batch layer as well, thus invalidating the pre-computed results as well.
- nathanmarz 15y agoBut you can always correct the mistake and recompute things. Unlike systems based on mutable data, mistakes aren't permanent.
- ww520 15y agoIf the transaction log is kept for the mutable system, you can always replay the transaction to arrive at the correct state. Most systems truncate the transaction log after a checkpoint to save space. If space is not an issue, you can always keep the transaction log since the beginning.