4 ms·

So, I am very much not a video codec developer, so I feel I have to ask: in the comparison I can see a difference, but I have a hard time discerning whether one

So, I am very much not a video codec developer, so I feel I have to ask: in the comparison I can see a difference, but I have a hard time discerning whether one or the other is better. Maybe not when there are very visible artifacts, but say comparing Daala and HEVC. Apart from obvious artefacts, what are we looking for?

ZeroGravitas 10y ago
The visual quality is a bit of a red herring, as in most situations you'd just up the bitrate until it was acceptable. So quality and bitrate are in some senses interchangeable. The graph above the comparison suggests that at this low bitrate, HEVC/BGP will very slightly outperform Daala on average, but each individual image may be slightly better or worse. It'll more probably come down to which kinds of artifact you're more familiar with and so see more easily. Switching quickly between the original and the encoded version is probably the easiest way to spot the artifacts/compromises that are being made.
- jmvalin 10y ago
  Obviously, the rates for these images are the same across all codecs. Choosing the rate for such comparison is also a little tricky. If you choose a rate that people would use to make their image look really good (which how most people compress images), then you have a realistic use case, but nobody can see much difference. If you choose too low a rate, then all images look really ugly. So I tried picking a rate where the resulting images are tolerable, but low enough that differences across codecs are easily noticeable.
lambda 10y ago
As it says, at this point it's getting pretty subjective, and so you'll need a blinded subjective test to really judge which one is better: The improvement in quality compared to the previous status update is quite obvious. Daala is now much better than both WebP and JPEG (libjpeg-turbo is the most commonly used JPEG encoder). As for Daala vs BPG/HEVC, the artifacts are obviously different and hard to judge from just four images. Opinions are likely to vary based on the viewer and the input image. At this point, what we'd really need is a full subjective test. Fortunately, this is exactly what is going to take place shortly, as Daala has been submitted as a candidate for the Image Compression Grand Challenge at ICIP 2016. The results should be available in September. In the mean time, you can read the paper we will be presenting. What I generally do is toggle between three images, the original, Daala, and HEVC, and focus on a different area each time. There is always some obvious distortion from the original, so I try to determine for each area I'm focusing on, which one is closer to the original perceptually. It is a subjective process, but if you make this evaluation for each area of the image, count up your judgements over the whole image, do that over multiple images, and have multiple people all do that and combine their results, you can come up with a reasonable evaluation of the quality.
- FullyFunctional 10y ago
  One good example is the hair on the forehead of the "woman" image. IMO, Daala does better than BPG here, but obviously overall there's no clear overall winner (but JPEG is a clear looser :) Very excited to see another Daala update. I look forward to a competivite alternative to HEVC.

ZeroGravitas 10y ago

The visual quality is a bit of a red herring, as in most situations you'd just up the bitrate until it was acceptable. So quality and bitrate are in some senses interchangeable. The graph above the comparison suggests that at this low bitrate, HEVC/BGP will very slightly outperform Daala on average, but each individual image may be slightly better or worse. It'll more probably come down to which kinds of artifact you're more familiar with and so see more easily. Switching quickly between the original and the encoded version is probably the easiest way to spot the artifacts/compromises that are being made.

jmvalin 10y ago
Obviously, the rates for these images are the same across all codecs. Choosing the rate for such comparison is also a little tricky. If you choose a rate that people would use to make their image look really good (which how most people compress images), then you have a realistic use case, but nobody can see much difference. If you choose too low a rate, then all images look really ugly. So I tried picking a rate where the resulting images are tolerable, but low enough that differences across codecs are easily noticeable.

lambda 10y ago

As it says, at this point it's getting pretty subjective, and so you'll need a blinded subjective test to really judge which one is better: The improvement in quality compared to the previous status update is quite obvious. Daala is now much better than both WebP and JPEG (libjpeg-turbo is the most commonly used JPEG encoder). As for Daala vs BPG/HEVC, the artifacts are obviously different and hard to judge from just four images. Opinions are likely to vary based on the viewer and the input image. At this point, what we'd really need is a full subjective test. Fortunately, this is exactly what is going to take place shortly, as Daala has been submitted as a candidate for the Image Compression Grand Challenge at ICIP 2016. The results should be available in September. In the mean time, you can read the paper we will be presenting. What I generally do is toggle between three images, the original, Daala, and HEVC, and focus on a different area each time. There is always some obvious distortion from the original, so I try to determine for each area I'm focusing on, which one is closer to the original perceptually. It is a subjective process, but if you make this evaluation for each area of the image, count up your judgements over the whole image, do that over multiple images, and have multiple people all do that and combine their results, you can come up with a reasonable evaluation of the quality.

FullyFunctional 10y ago
One good example is the hair on the forehead of the "woman" image. IMO, Daala does better than BPG here, but obviously overall there's no clear overall winner (but JPEG is a clear looser :) Very excited to see another Daala update. I look forward to a competivite alternative to HEVC.