This article seems to propose log compaction as the answer to the question of size (i.e. how much historical data is going to have to be kept around and how muc

This article seems to propose log compaction as the answer to the question of size (i.e. how much historical data is going to have to be kept around and how much is that going to cost). However, log compaction is not well suited to many use cases: storing partial updates or diffs in the log, storing many trillions of tiny entries (as keys), multiple messages on the log corresponding to related (contingent) updates, and so on. Those are tractable but hard to solve; log compaction is not a silver bullet and unless you think really hard about how your data changes over time, you may end up storing more of it than you expect if you use the log as an eternal source of truth--compaction or not.