4 ms·
Language Modeling Is Compression
- rg111 3y agoCan we please link directly to arXiv instead of HuggingFace?
- j-pb 3y agohttps://arxiv.org/abs/2309.10668 https://arxiv.org/abs/2309.10668
- nyrikki 3y agoI am confused about the lack of mention about PAC learnability, VC dimensional and the work of Littlestone and Warmuth from the 1980s PAC learnability is directly related to compressibility, with only ergotic systems being fully learnable as would be required to happen for lossless compression. Perhaps review of Ergotic theory and PAC learnability would help convince reviewed more. Comparing gzip, which was designed with time and space complexity constraints as StoA compared to a training transformers was called out on the author's previous work. https://openreview.net/forum?id=Hygi7xStvS https://openreview.net/forum?id=Hygi7xStvS If someone can educate me on what I am missing it would be appreciated