4 ms·
I think a lot of people had similar ideas, I know I did. The idea of using Pi is essentially the same as having a shared dictionary as used in both regular com
by SillyUsername 3y ago
I think a lot of people had similar ideas, I know I did.
The idea of using Pi is essentially the same as having a shared dictionary as used in both regular compression and in lossy as waveforms (in layman's, terms - technically it's a best match) both ends have the dictionary and you simply index it.
In fact whole network protocols use this concept too such as protocolbuffers which are part of grpc.
The difference here being the dictionary is infinite and until we get quantum computers on the desktop indexing into pi will always be slow. Some of the address space may not be feasible too, e.g. your file may require a billion bit address/index.
It may still be feasible to do partial matches for a file, 50% one index, 50% another for performance improvement.
I guess the trade off is a balance between performance and number of indexes Vs the original file length, and where in Pi the address space becomes unfeasible.
- adamgordonbell 3y agoIt's interesting how zstd can use a shared dictionary to compress small amounts of data. It's a more practical ( then my idea ) shared dictionary.
- wahern 3y agozlib also supports dictionaries, as does the deflate algorithm. The problem is that zlib doesn't provide any tooling to generate an optimal dictionary from a large sample, whereas zstd does provide such a utility.