3 ms·
Thank you for posting, I enjoyed the article. In true cybernetics fashion, the maximum compression ratio of information seems to be the current grail of our ma
by dleeftink 3y ago
Thank you for posting, I enjoyed the article. In true cybernetics fashion, the maximum compression ratio of information seems to be the current grail of our machine learning efforts, whether that is through inferring heuristics or gradually incorporating knowledge about the world.
I'm interested to see where on the information density gradient various organisms exists in their ability to do so..DNA seems awfully wordy, but has proven to be a very dense storage medium.
- visarga 3y agoThere is also an approach to use a LLM and only compress the errors from LLM to the target text. You'd have to add the LLM to the final size, but you could have it already downloaded at the target, in which case I expect nothing will beat it for text compression. Of course it would be very slow.