2 ms·
More accurate would be “understanding sometimes enables compression”. If you understand the underlying regularities in some dataset, and if those regularities t
by foldr 4y ago
More accurate would be “understanding sometimes enables compression”. If you understand the underlying regularities in some dataset, and if those regularities themselves have a compact representation (which is in no way guaranteed - an equation can be bigger than any given finite data set which it describes), then you can use your understanding to compress the data. But the implications of that banal observation for AI are roughly nil. It certainly doesn’t imply any identity relation between compression and understanding.
A realistic example might be the Mod/RM byte in x86 instruction encoding. There are underlying regularities, but a lookup table could quite possibly be smaller than the code required to generate the correct Mod/RM byte given operands. So you can understand the Mod/RM encoding without thereby being able to compress anything.