4 ms·

I don't know what features AWS provides but in general terms logs benefit a lot from compression, so if I were to set up this on my own I'd probably want to try

by hnarn 5y ago

I don't know what features AWS provides but in general terms logs benefit a lot from compression, so if I were to set up this on my own I'd probably want to try something like a VDO or ZFS backed storage system as well as compressed transfers (perhaps in batch if that's required).

nuker 5y ago
> something like a VDO or ZFS backed storage system And what text search engine?
- hnarn 5y ago
  I have no idea, but I’m assuming that regardless of which one you use there will be disk level duplication that could be taken care of to alleviate uncontrolled growth somewhat.