3 ms·

Lossless is actually the appropriate test. These models output probabilities and can easily be turned into very effective lossless compressors with the use of a

Lossless is actually the appropriate test. These models output probabilities and can easily be turned into very effective lossless compressors with the use of arithmetic encoding. For a very interesting discussion see: https://www.youtube.com/watch?v=dO4TPJkeaaU https://www.youtube.com/watch?v=dO4TPJkeaaU The real problem with the Hutter prize is that the data and compute are too small. It seems obvious now that the regime where compression turns into AGI is far beyond the scale of the current contest (1 GB data, 50 CPU hours). If you simply bumped those limits up five or six orders of magnitude then the Hutter prize could be relevant today. But you'd also have to increase the size of the prize a few orders of magnitude because the cost of competing would be more than the current prize money. Of course a prize isn't really a necessary incentive to develop AGI, since getting an AGI is about the best incentive you could possibly imagine.

optimalsolver 4y ago
>Lossless is actually the appropriate test Could you elaborate on why that is. It seems very non-obvious to me. In the realm of vision, after a certain point you would just be trying to compress noise that has nothing to do with the content of image/video.
- modeless 4y ago
  That's OK. The purpose of the test is to compare approaches to compression. Noise in the dataset is a handicap but it affects all approaches, so it doesn't invalidate the test as a way to compare approaches. And true white noise is very rare, almost all real noise has characteristics that are at least somewhat compressible. And hopefully whoever constructs the dataset tries not to include tons of noise in it. Lossless compression is a great objective because it's impossible to cheat. When you do lossy compression you have to define a quality metric and as soon as you do that the game becomes cheating the quality metric rather than actually compressing the data in a useful way. I highly recommend watching the video I linked. Arithmetic encoding reduces the task of lossless compression to assigning probabilities for the next token, which is the exact task these models are trained to do.
- inimino 4y ago
  Answered in detail in the FAQ: http://prize.hutter1.net/hfaq.htm#lossless http://prize.hutter1.net/hfaq.htm#lossless

optimalsolver 4y ago

>Lossless is actually the appropriate test Could you elaborate on why that is. It seems very non-obvious to me. In the realm of vision, after a certain point you would just be trying to compress noise that has nothing to do with the content of image/video.

modeless 4y ago
That's OK. The purpose of the test is to compare approaches to compression. Noise in the dataset is a handicap but it affects all approaches, so it doesn't invalidate the test as a way to compare approaches. And true white noise is very rare, almost all real noise has characteristics that are at least somewhat compressible. And hopefully whoever constructs the dataset tries not to include tons of noise in it. Lossless compression is a great objective because it's impossible to cheat. When you do lossy compression you have to define a quality metric and as soon as you do that the game becomes cheating the quality metric rather than actually compressing the data in a useful way. I highly recommend watching the video I linked. Arithmetic encoding reduces the task of lossless compression to assigning probabilities for the next token, which is the exact task these models are trained to do.
inimino 4y ago
Answered in detail in the FAQ: http://prize.hutter1.net/hfaq.htm#lossless http://prize.hutter1.net/hfaq.htm#lossless