Perhaps store it out to hdfs/blob storage. Then use some kind of unified processing layer to reconstruct history or parts of history when you need it? Eg: https

Perhaps store it out to hdfs/blob storage. Then use some kind of unified processing layer to reconstruct history or parts of history when you need it? Eg: https://news.ycombinator.com/item?id=20059006#20062821 https://news.ycombinator.com/item?id=20059006#20062821 Or use a durable unbounded solution like Pravega instead of Kafka? https://news.ycombinator.com/item?id=20059006#20062821 https://news.ycombinator.com/item?id=20059006#20062821 edit: this could be problematic especially when data privacy laws are involved. See the comments from gopalv below.