3 ms·
> a constant 5 GB penalty is enormous in practice > orders of magnitude bigger than anything used in the real world [root@archlinux mail]# pwd /mail
by zimpenfish 6y ago
> a constant 5 GB penalty is enormous in practice
> orders of magnitude bigger than anything used in the real world
[root@archlinux mail]# pwd
/mail
[root@archlinux mail]# du -skh .
68G .
And this is a tiny personal mailserver. There's loads of applications where a 5GB penalty* is well below the amount of text you're looking at (wikipedia springs to mind since they're in the same kind of size range for text.)
- goodside 6y agoObviously bodies of text bigger than 5GB exist. I was talking about static compressor dictionaries, which are tiny. Hence mentioning Brotli’s 122KB dictionary. Static dictionaries are an optimization to improve the compression of very small text files — they aren’t useful for compressing large files, because once you have lots of data you can build a more efficient dictionary at compression time and include it in the compressed stream.