4 ms·
What I noticed in the other text, Hamlet, is that HAMLET in the last sentence has no pointer to Hamlet, obviously. This seems like an opportunity for optimisati
by pdknsk 13y ago
What I noticed in the other text, Hamlet, is that HAMLET in the last sentence has no pointer to Hamlet, obviously. This seems like an opportunity for optimisation, for text at least.
Usually a word is either lowercase, capitalised or uppercase. In more complex and rare cases this could be efficiently encoded bitwise (0 = keep, 1 = swap case), so HAMLEt becomes 011110 plus a pointer to Hamlet.
I wonder if any compression algorithm does this. Probably not, because the benefit is likely minimal at significantly increased de/compression time.
- cbhl 13y agoOne problem is that way of thinking is distinctly ASCII-centric -- what happens when you have accents, umlats, Chinese, and the rest of the stuff that comes out of Unicode? I suspect the encoding you propose would actually increase the size of the average plaintext, simply because people rarely will type "Hamlet" and then later type "HaMlEt" in the same document. I think a far more common case would be "Hamlet" (beginning of sentence) and "hamlet" (inline), where you can reuse the substring "amlet". (Either that, or they're consistent with the usage of all-caps, like MICROSOFT(R).) Besides, if you know you're doing English, you can save 12.5% simply by using 7-bit ASCII instead of of 8-bit ASCII.
- apw 13y agoYour idea is good, and many modern compression systems, like the PAQ system, exploit it [1]. [1] https://en.wikipedia.org/wiki/PAQ#Text_preprocessing https://en.wikipedia.org/wiki/PAQ#Text_preprocessing