4 ms·
One thing I'd love to see follow from this is an image format based on its intra coding. The companies involved expressed interest (https://www.cnet.com/news/go
by twotwotwo 9y ago
One thing I'd love to see follow from this is an image format based on its intra coding. The companies involved expressed interest (https://www.cnet.com/news/google-mozilla-av1-photo-format-could-outdo-aging-jpeg/ https://www.cnet.com/news/google-mozilla-av1-photo-format-co...), and tests (https://people.xiph.org/~tdaede/av1stilldemo/ https://people.xiph.org/~tdaede/av1stilldemo/) look great.
The .heic format now used by iPhones is the same idea (https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format https://en.wikipedia.org/wiki/High_Efficiency_Image_File_For...). But wider use of that is going to be constrained by the cost problem HEVC has in general (two patent pools to deal with). Some of WebP is based on VP8 (not VP9) intra coding too.
There are a lot of tools packed in AV1's intra coding (and HEVC's, though I've read less about that). Block sizes range from 4x4 to 64x64 and there's a mix within one image, so the encoder can use the right size for the level of detail in each area. There are more ways to predict a block's content from what's to the top and left, which leaves less work for the JPEGish DCT part. There's clever de-ringing post-processing that, in effect, blurs away many of the attention-getting JPEG-y DCT artifacts around edges, while 1) being aware of the direction of the main edge itself to avoid blurring that away and 2) using contrast thresholds to preserve as much other legitimate detail as it can--more about deringing at https://people.xiph.org/~jm/daala/deringing_demo/ https://people.xiph.org/~jm/daala/deringing_demo/ .
(There's some good detailed discussion at https://parisvideotech.com/wp-content/uploads/2017/07/AOM-AV1-Video-Tech-meet-up.pdf https://parisvideotech.com/wp-content/uploads/2017/07/AOM-AV... and in Wikipedia's page at https://en.wikipedia.org/wiki/AV1 https://en.wikipedia.org/wiki/AV1 )
Relatedly, given the complexity, I wouldn't expect this to, like, take the world by storm in the next three months. The unoptimized encoder is still _really_ slow. Google has designed a hardware implementation, but of course hardware designs take time to get integrated, fabbed, and into shipping products. Given who's involved, I'm hopeful it does get wide support. (Would love to hear Apple's plans given their current support for x265; their decision to join AOMedia is a good sign at least.) Anyway, looking forward to seeing the results.
- futurix 9y agoGood image formats for movie frames aren't necessarily good for stand-alone still images though...
- twotwotwo 9y agoThere are differences--you're often willing to do more work decoding a still than a frame, say--and in the first story I linked there's a quote from a Googler wondering about whether to make the image format exactly track AV1 or to tweak it. So, there's a real question. That said, the still demo at https://people.xiph.org/~tdaede/av1stilldemo/ https://people.xiph.org/~tdaede/av1stilldemo/ is really encouraging, and Apple's seen a win in doing something parallel with HEVC/.heic. AV1's already gone through a lot of optimization and IP vetting (patents caused trouble for previously proposed replacements for JPEG), and has a hardware decoder design and a lot of companies signed on. Like everyone I just have to wait and see what they do, but I'm more or less on team "ship it" here.
- zokier 9y agoOn the other hand the bar for still images is very low due JPEGs dominance. That's why it makes sense to use video encoders for stills even if they are not ideal, you are still going to get much better results than JPEG.
- floatboth 9y ago> Some of WebP is based on VP8 (not VP9) intra coding too Quite an understatement. "Some" here means WebP is literally a single VP8 frame shoved into a RIFF container :D Back in the day (holy shit 8 years ago already!) there was a script that added WebP support to WebM-supporting browsers by literally changing the container. Here it is: https://github.com/antimatter15/weppy/blob/master/weppy.js https://github.com/antimatter15/weppy/blob/master/weppy.js
- clouddrover 9y agoThese days there's a WebAssembly build of libwebp which enables WebP support in browsers which don't support it natively: https://webmproject.github.io/libwebp-demo/webp_wasm/index.html https://webmproject.github.io/libwebp-demo/webp_wasm/index.h... I think it's an ideal use case for WebAssembly. It'd be a good way to roll out any future AV1-based image format until native support arrived in all browsers.
- twotwotwo 9y ago> "Some" here means WebP is literally a single VP8 frame shoved into a RIFF container :D Though it's less used than the lossy format, lossless WebP is a thing too; if I said WebP was exactly VP8 intra someone might nitpick that. There's no winning, haha!
- twotwotwo 9y agoHeh, fun consequence I just realized is that, in theory, you don't even have to wait for video-based still formats to be standardized and deployed if you're willing to play some games with single-frame videos. So, like, h.264-based "images" can be shown in most browsers now, and VP9-based in many (but not Safari or IE). And you can fake AV1 images as soon as you get AV1 video. Using video decoding in an odd way like that probably isn't especially practical or wise, but fun that the capability's there/reachable.
- MikusR 9y agoThere are at least 3-4 pools plus individual patent holders.