The naive solution would be to split the input data into the batches. Compress the first batch using a random model, then train a new model based on it. Compres

The naive solution would be to split the input data into the batches. Compress the first batch using a random model, then train a new model based on it. Compress the second batch using the model trained on the first batch and so on. Depending on your data structure, you can use a model trained on all the previous batches (or just N previous batches) when compressing the i-th batch. This can be likely optimized (e.g. 7-Zip's arithmetical coder doesn't split data into batches, but rather uses exponentially filtered conditional probabilities that are instantaneously updated each time a new sample is processed). I'm no expert in RNNs, but my hunch is that you can adjust the model incrementally using backpropagation right after it processes another sample instead of completely retraining it.