3 ms·
Ask HN: Compression with common binary database
Today I realized something. Humans compress their communication by knowing things mutually. For example, if you know a joke and if your friend knows the same joke too, you can remind him/her that joke any second by just telling a word of it.
Today our compression algorithms compress stuff by only thinking that nobody knows the joke we are gonna tell and tries to choose shorter words instead of just outputting a word of the joke.
What if, we would analyze many compressed files and find the most common binary patterns and create a new compression algorithm which uses these patterns to tell the joke with only a word? And the person who wants to decompress the file would need to have the database as well.
- throwaway_374 10y agoGreat in concept until you run into the pigeonhole principle. Such compression schemes are fundamentally flawed and you can spend years wasting time on them until some corner edge case will throw you. Nonetheless, I'd encourage you to satisfy yourself by trying to devise such an elaborate scheme... Pied Piper may have a job opening for you.
- PaulHoule 10y agoSee https://en.wikipedia.org/wiki/Brotli https://en.wikipedia.org/wiki/Brotli
- tarikozket 10y agoThis is exactly what I have thought :) Thinking this as a software and making the database bigger and cloud connected makes more sense.