The obvious improvement is to use a better format than JPEG. Maybe something that can include information about the encoding process so the decoder can use that

The obvious improvement is to use a better format than JPEG. Maybe something that can include information about the encoding process so the decoder can use that instead of guessing. Or maybe even a non-lossy format so we can focus more on data compression and less on visual compression.