3 ms·
That is a strange statement, most machine learning boils down to compression in essence (OK it is lossy, but any image quantisation is also lossy). E.g. autoenc
by riedel 5y ago
That is a strange statement, most machine learning boils down to compression in essence (OK it is lossy, but any image quantisation is also lossy). E.g. autoencoders are compression algorithms: take the model as a dictionary and the latent space weights as compressed representation. For most algorithms you are right though. Particularly it is worth the effort to put things in to Asics as the "wiring" normally stays stable and you do not need the flexibility of gpgpus . Still also there is nothing that keeps you from using them e.g. for efficient parallel decoding.
- astrange 5y agoI think it's axiomatically true and ML models get a pass on being inefficient because people are so busy being impressed by their magic tricks. In particular, using ML models for compression would get a free advantage because they're often several GBs where a traditional codec is KBs, so they can hide data in there, but most people wouldn't think of that as being part of the compressed message size. But they could have very differently shaped dependency trees, especially if you somehow managed to not run an entropy coder as the last step of compression.
- riedel 5y agoTotally agree, tgeneral compression that requires the dictionary to be transmitted on every message is not the best use case (if there even is any). Minimal description length is actually a nice information theoretical measure to describe power of ML models to compress. However I have also seen tons of theoretical proof that probably there is no optimal compression method ( I looked at grammar vs entropy coding some time ago) hence there is probably no optimal hardware in general. In the end it depends on your assumption about the the data (quality, structure, size, chunking,...) you want to compress.