7 ms·
> We use Netflix’s Emmy-winning Video Multimethod Assessment Fusion (VMAF) to assess visual quality, as it’s the best available tool at the moment. Maybe not?
by CharlesW 4y ago
> We use Netflix’s Emmy-winning Video Multimethod Assessment Fusion (VMAF) to assess visual quality, as it’s the best available tool at the moment.
Maybe not? From "Who Has the Best Hardware AV1 Encoder?", discussed in another HN thread (https://news.ycombinator.com/item?id=35587509 https://news.ycombinator.com/item?id=35587509):
"Everyone's favorite bad, obsolete, easily tricked metric is VMAF, which was good for a time but simply isn't good anymore."
- ksec 4y agoI think it is fair to say VMAF was never meant to measure still image quality. It is also not perfect and Netflix has plenty of example of areas known to be problematic. But it is still far better than say PSNR or SSIM. The major problem is when encoder optimise for VMAF only.
- Dalewyn 4y agoI like how the anime fansubbing community does it: Take random clips of anime, do various encodes under real world conditions, compare results.
- bick_nyers 4y agoHave any good examples of someone doing this?
- CharlesW 4y agoThe MSU Video Codecs Comparisons do this (see "Subjective Comparison Methodology"): https://www.compression.ru/video/codec_comparison/2021/subjective_report.html https://www.compression.ru/video/codec_comparison/2021/subje... As far as I know, human ABX testing is still the best way to measure the relative perceptual quality of different codecs.