3 ms·
> Also, ML models (probably) can't be compressed because they already are compressed; learning and compression are the same thing! I feel like they're kind of
by ColonelPhantom 4y ago
> Also, ML models (probably) can't be compressed because they already are compressed; learning and compression are the same thing!
I feel like they're kind of two sides of the same coin: learning is about putting more information in the same data, while compression is about putting the same information in less data.
I'm wondering if some lossy floating-point compressor (such as zfp) would work.
- astrange 4y ago> I'm wondering if some lossy floating-point compressor (such as zfp) would work. Well apparently this can work; StableDiffusion comes as 32-bit and 16-bit float versions. I'm kind of surprised they both work, but that's lossy compression.
- ColonelPhantom 4y agoSure, but 16-bit float is pretty primitive compression, as it does not exploit any redundancy in the input. zfp groups numbers together in chunks, which means that correlated numbers can be represented more precisely. Its algorithm is described here: https://zfp.readthedocs.io/en/release1.0.0/algorithm.html#lossy-compression https://zfp.readthedocs.io/en/release1.0.0/algorithm.html#lo... I would like to see if the zfp can be applied to something like Stable Diffusion (or other ML models) and give better results than regular floats at the same size.