3 ms·
I understand that generated log files are generated upstream of you, but how much does compressing them actually wins? I guess for most of us it's better to ju
by listic 11y ago
I understand that generated log files are generated upstream of you, but how much does compressing them actually wins?
I guess for most of us it's better to just search plain text files, if possible?
- sp332 11y agoAs a rule of thumb, English in ASCII compresses 8:1. I would guess that log files compress even better because they tend to be repetitive. Edit: well best case is 8:1 if you put a lot of time into it. Gzip with default settings is more like 4:1 which is still significant.
- listic 11y agoHow can you optimize for better compression ratios with gzip?
- sp332 11y agogzip's default is -6, but -9 gives better compression. If you can sort your data or otherwise organize it so that similar data is grouped together, it has a better chance of finding the redundancy. Some implementations get better compression than the GNU implementation. The best one I know of is from the 7zip project. Other compression formats can give better ratios, if that's an option for your project. bzip2, xz, 7z.
- _delirium 11y agoYes, log files definitely do better. Doing a few spot checks, my log files compress anywhere from 10:1 to 50:1 with default gzip settings, depending on the kind of log.