2 ms·

Isn't noise in the data going to dominate output size of lossless compression? Wouldn't linguistics and vision be better off with direct measurements of predict

• d_burfoot 3y ago
  (author) Noise certainly affects the compression rate. But you are not concerned with the absolute compression rate, you are only concerned with the relative rate achieved by two theories A and B. Both theories will be negatively impacted to the same degree by the noise, so the comparison still works to select which theory is better.
  • harperlee 3y ago
    …to the extent that the difference between theories dominates over the possible noise variability.
    • gwern 3y ago
      You can easily quantify the variance and do standard model-comparison/hypothesis-testing if you want statistical-significance levels. For many datasets these days, this is hardly even a consideration: even a 1% compression improvement is clear.