3 ms·

That's fine, I agree. My point was not "His metric is the most awesome, shut up", it was "Despite only giving one kind of number, the author still included oth

by jfager 16y ago

That's fine, I agree. My point was not "His metric is the most awesome, shut up", it was "Despite only giving one kind of number, the author still included other analysis and subjective opinion in the writeup for each photo, so it's not as bad as you're trying to imply".

sp332 16y ago
You asked How else would you construct an objective test? And my answer is, just make sure you're being fair in measuring the same thing the encoders are optimising for.
- jfager 16y ago
  That was in response to the complaint that the post looked at images of the same size and compared their quality, rather than looking at images of the same quality and comparing their size. If you know of a practical way to objectively do the latter, please share.
- scott_s 16y ago
  If I only test the optimal circumstances for a new thing, then I have done a terrible job at trying to understand the consequences of adopting it.
  - sp332 16y ago
    On two occasions I have been asked,—"Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" .... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. --Babbage (1864), Passages from the Life of a Philosopher, ch. 5 "Difference Engine No. 1" You can't reasonably use an encoder optimized for PSNR and then "ding" it for producing output with bad SSIM, or vice-versa.
    - scott_s 16y ago
      I'm not familiar enough with the image encoding algorithms (and the tradeoffs they are faced with) to comment on that, specifically. But I've done enough research with systems in general to disagree with your statement. No matter what I optimize a system for, it's always fair for someone to test it under adverse conditions. If we're going to adopt something, we need to know its limitations. Further, my understanding of Google's intent with webp is that it is being offered as a replacement to jpeg. In that case, even if it's optimized for one thing, it's not just fair but necessary to see how it works under all other things.

by jfager 16y ago

sp332 16y ago

You asked How else would you construct an objective test? And my answer is, just make sure you're being fair in measuring the same thing the encoders are optimising for.

jfager 16y ago
That was in response to the complaint that the post looked at images of the same size and compared their quality, rather than looking at images of the same quality and comparing their size. If you know of a practical way to objectively do the latter, please share.
scott_s 16y ago
If I only test the optimal circumstances for a new thing, then I have done a terrible job at trying to understand the consequences of adopting it.
- sp332 16y ago
  On two occasions I have been asked,—"Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" .... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. --Babbage (1864), Passages from the Life of a Philosopher, ch. 5 "Difference Engine No. 1" You can't reasonably use an encoder optimized for PSNR and then "ding" it for producing output with bad SSIM, or vice-versa.
  - scott_s 16y ago
    I'm not familiar enough with the image encoding algorithms (and the tradeoffs they are faced with) to comment on that, specifically. But I've done enough research with systems in general to disagree with your statement. No matter what I optimize a system for, it's always fair for someone to test it under adverse conditions. If we're going to adopt something, we need to know its limitations. Further, my understanding of Google's intent with webp is that it is being offered as a replacement to jpeg. In that case, even if it's optimized for one thing, it's not just fair but necessary to see how it works under all other things.