13 ms·

AVIF – AV1 based image format

176 points by lcnmrn 9y ago

niftich 9y ago
This is a format combining HEIF as the container with AV1 I-frames as the payload. HEIF is the Nokia-developed, MPEG and ISO-blessed container for image sequences and transforms, which is published as MPEG-H Part 12, and is based on the good-old ISOBMFF, which you typically recognize as the ".MP4 container". HEIF's most wide deployment right now is on Apple iPhones, where it's the new default capture format for still images and 'live' images (effectively, a short video on either side of a high-res still). AV1 is the AOMedia-developed, Google/ex-On2/Xiph.Org/Cisco consensus video format that's the evolution of the VP8/VP9 line and Daala. This format takes the nice royalty-free state-of-the-art MPEG container and puts a royalty-free payload in it, making it a useful best-of-both-worlds.
- mtgx 9y ago
  Isn't HEIF the payload and HEIC the container? (also C seems to stand for container in HEIC...). It's confusing how people seem to use HEIF to mean container. And if HEIF is actually the container, then what is the payload in Apple's case, and what exactly is HEIC?
  - niftich 9y ago
    HEIF is the name of the container [1][2]. HEIC is an ISOBMFF 'brand' to be used inside the container to indicate that the payload's codec is HEVC (in 'Main' profile or 'Main Still Picture' profile), and that it's one still-image and not a sequence [1]. In most of the cases where the payload is the HEVC codec, the corresponding file extension is encouraged to be .heic and the MIME-type follows as well. Apple has a note about this in their transcript at WWDC 2017 [3]. [1] https://nokiatech.github.io/heif/technical.html https://nokiatech.github.io/heif/technical.html [2] https://mpeg.chiariglione.org/standards/mpeg-h/image-file-format https://mpeg.chiariglione.org/standards/mpeg-h/image-file-fo.... [3] https://developer.apple.com/videos/play/wwdc2017/513 https://developer.apple.com/videos/play/wwdc2017/513
  - deleted 9y ago
    [deleted]
- deleted 9y ago
  [deleted]
- twotwotwo 9y ago
  Comparing HEVC-based and AV1-based: there's a tech demo (that I've posted here...a lot now heh) at https://people.xiph.org/~tdaede/av1stilldemo/ https://people.xiph.org/~tdaede/av1stilldemo/. Both AV1 and x265 intra look great. If the artifacts when you compare to originals bug you, remember this is just a test at a level where JPEG is painfully blocky, and you can go higher! Visually, I think AV1 adds less DCT-ish noise, but tends to blur small, low-contrast details a little more than x265. (Compare the ends of tree branches on the default image, Crepuscular Rays, or faraway tiles and details in Mercado dos Lavradores. Haven't done objective measurements though--you may disagree.) I, personally, like AV1's tradeoffs better, at least from what I see there. Patches of ringing draw attention, whereas slight blurring is something we're used to ignoring since it comes from lots of things besides codecs; natural vision and camera images aren't completely sharp either. (Dunno why AV1's that way, but might be that it does more aggressive deringing with that filter Monty's posting about next.) The photo caption on the comparison in this story suggested the same conclusion re the detail vs. noise tradeoff, for what it's worth: https://www.cnet.com/news/google-mozilla-av1-photo-format-could-outdo-aging-jpeg/ https://www.cnet.com/news/google-mozilla-av1-photo-format-co... The paper on the deringing filter included subjective tests, and it suggests that test subjects liked its results more consistently than you might guess from the changes in objective metrics--small sample size though: https://arxiv.org/pdf/1602.05975.pdf https://arxiv.org/pdf/1602.05975.pdf Sometimes you may want x265's bias towards more detail; in the Tiger image, AV1 "cuts" the whiskers near the top of the image shorter than x265 does. Tricky to 'teach' encoders that some detail is more important than others despite both being equally low size or contrast. Given they're both a ton better than JPEG, the big advantage to AV1 for a random developer is the openness; we can use it w/out needing to either take a legal risk or lean on the implementation of some company that's dealt with the HEVC patent mess for us.
  - niftich 9y ago
    To my eyes and preference, both x265 and AV1's stills are far less blocky and far too smoothed than I'd like, but then again I've been an unabashed proponent of less smoothing [1], having lived through early H.264 encoders' blurry messes. Once I get past that, some of the test images look better on AV1, and some look better on x265. Some of them kill detail here, some of them kill detail there. I think this is a victory -- both of them perform really well, considering, and allocating a bit more bits to get it looking nice is always an option. It's good to have a royalty-free codec that's actually good, and with much more profile and fanfare than just a weird garage-band secret project like VP9 was. And with widespread hardware support, it may actually make a dent in JPEG's marketshare. [1] https://hn.algolia.com/?query=niftich%20fake%20sharpness&sort=byDate&type=comment https://hn.algolia.com/?query=niftich%20fake%20sharpness&sor...
bhouston 9y ago
I was wanting this just a few week ago: https://news.ycombinator.com/item?id=16709626 https://news.ycombinator.com/item?id=16709626 I didn't realize it already existed. Amazing.
Adamantcheese 9y ago
So it's basically what webp is to webm? Also how does it compare to my favorite image format, FLIF? http://flif.info/ http://flif.info/
- bhouston 9y ago
  About 95% smaller than FLIF I would think.
  - adamzochowski 9y ago
    How is it 95% smaller? FLIF is a lossless compression, and AV1 in lossless mode can't do magic. It can't do 95% better. https://wyohknott.github.io/image-formats-comparison/report.html https://wyohknott.github.io/image-formats-comparison/report.... FLIF is reported at 59.57% avg space savings AV1 (2018-02-22) is reported at 66.02% avg space savings
    - notatoad 9y ago
      presumably it's 95% smaller because it's not limited to lossless mode?
      - simcop2387 9y ago
        FLIF is also not limited to lossless mode, http://flif.info/lossy.html http://flif.info/lossy.html
        zaarn 9y ago
        And FLIF's lossy mode is as good as AV1's?
        simcop2387 9y ago
        That part I don't know. I haven't seen a comparison between them, but it's compared well at least against BPG, or whatever the h264 based format was. I don't think i've seen a comparison against h265, let along AV1. It'd be interesting to see. Even if they're about as good, I like FLIF's setup better since it enables progressive decoding which could be a big step up for slower connections.
ComputerGuru 9y ago
I found no mention in the document of CMYK color space or mandatory dpi attributes, both of which are severe pain points for PNG images. Does anyone know about this?
- cbhl 9y ago
  AV1 (and thus AVIF) appears to be designed for screens (works in YUV color space). If you want to print, PDF is probably still your best bet.
  - paulkerr 9y ago
    AV1 can also encode RGB color space. Such usage would probably be more prevalent in the still image use case.
  - ComputerGuru 9y ago
    Actually I munged two issues. CMYK is for print but DPI is for retina images.
- awalton 9y ago
  They have a metadata block type defined and you can shove whatever metadata format you want into it (EXIF, etc), and it has "image properties" for the necessary bits (presumably DPI would land itself as an image property, but I can see leaving colorspace out by assuming a "sane" default of something like sRGB). Since there are plenty of reasons you don't care about this metadata (e.g. to reduce the transfer size on images that are requested millions or billions of times by a web browser), as much as possible it should be left optional. But all of this seems to be pretty much draftware at the moment, so we'll see how things shake out.
  - paulkerr 9y ago
    You are correct: the specification is a work in progress which we plan to wrap of quickly. The approach is not to over-specify the format with features/options initially, but begin with a functional sub-set of HIEF and MIAF. The target is to provide an HDR/WCG successor to JPEG. This work is being done by a working group in the Alliance for Open Media.
askvictor 9y ago
The name will just lead to confusion. AVI is already a well-known and much-used video (container) format, with no relation to AVIF or AV1. As discussed in AV1 discussion yesterday, AV1 will already cause confusion, and possibly coding and security errors. This name will lead to more of the same. Regular expressions, in particular.
runesoerensen 9y ago
Facebook also posted some performance tests comparing AV1 to x264 and libvpx-vp9: https://code.facebook.com/posts/253852078523394/av1-beats-x264-and-libvpx-vp9-in-practical-use-case/ https://code.facebook.com/posts/253852078523394/av1-beats-x2...
- nothis 9y ago
  Does any of this translate to still images? How much better would it be compared to, say, a JPG? I assume somewhat better but it would have to be magnitudes to take over.
jwatte 9y ago
Oh, look, another image format without support for stereo, cube layout, or MIP maps! I mean, it's not as if media is going 3D and VR right this moment (or even in the last few years) or anything...
pedro80 9y ago
It's absolutely irrelevant what container it uses (HEIF/ios/mp4), more over from my experience it was pure pain in the ass to deal with all these boxes and different ways different products produce them. I would never use it for an image format. Last time I checked it seemed like Nokia was a sinking ship, why would anyone board it?.. Keep it separate: make something that encodes image data separately and then use some container that to wrap it nicely. Same shit that works for jpeg should be reused IMO, it just works (all that metadata), simply replace coded image data with a better encoder (AV1)
- pedro80 9y ago
  when I see that nokiatech "tech" where some dudes make these nice documents on how to pack images in mp4, all reaction I get: WTF?! don't see no tech in there. All that shit exists for videos, treat image like a video file that has a single first i-frame and nothing else, that's what AV1 in mp4 is all about. Is their tech about limiting possibilities with mp4? I didn't ever read their tech and wouldn't bother spending even a minute. Like a year ago I started reading it and was WTF is this bullshit, a doc on image container format!??.. well, still all different devices will produce shit load of random mp4 structures, what's the point, you'll always have to deal with that pain in the ass that comes with all these possibilities with mp4 and you basically need a full blown lib to deal with it you pretty much cannot code it manually to parse mp4 to get coded image and show on the screen.

176 points by lcnmrn 9y ago

This is a format combining HEIF as the container with AV1 I-frames as the payload. HEIF is the Nokia-developed, MPEG and ISO-blessed container for image sequences and transforms, which is published as MPEG-H Part 12, and is based on the good-old ISOBMFF, which you typically recognize as the ".MP4 container". HEIF's most wide deployment right now is on Apple iPhones, where it's the new default capture format for still images and 'live' images (effectively, a short video on either side of a high-res still). AV1 is the AOMedia-developed, Google/ex-On2/Xiph.Org/Cisco consensus video format that's the evolution of the VP8/VP9 line and Daala. This format takes the nice royalty-free state-of-the-art MPEG container and puts a royalty-free payload in it, making it a useful best-of-both-worlds.

mtgx 9y ago
Isn't HEIF the payload and HEIC the container? (also C seems to stand for container in HEIC...). It's confusing how people seem to use HEIF to mean container. And if HEIF is actually the container, then what is the payload in Apple's case, and what exactly is HEIC?
- niftich 9y ago
  HEIF is the name of the container [1][2]. HEIC is an ISOBMFF 'brand' to be used inside the container to indicate that the payload's codec is HEVC (in 'Main' profile or 'Main Still Picture' profile), and that it's one still-image and not a sequence [1]. In most of the cases where the payload is the HEVC codec, the corresponding file extension is encouraged to be .heic and the MIME-type follows as well. Apple has a note about this in their transcript at WWDC 2017 [3]. [1] https://nokiatech.github.io/heif/technical.html https://nokiatech.github.io/heif/technical.html [2] https://mpeg.chiariglione.org/standards/mpeg-h/image-file-format https://mpeg.chiariglione.org/standards/mpeg-h/image-file-fo.... [3] https://developer.apple.com/videos/play/wwdc2017/513 https://developer.apple.com/videos/play/wwdc2017/513
- deleted 9y ago
  [deleted]
deleted 9y ago
[deleted]
twotwotwo 9y ago
Comparing HEVC-based and AV1-based: there's a tech demo (that I've posted here...a lot now heh) at https://people.xiph.org/~tdaede/av1stilldemo/ https://people.xiph.org/~tdaede/av1stilldemo/. Both AV1 and x265 intra look great. If the artifacts when you compare to originals bug you, remember this is just a test at a level where JPEG is painfully blocky, and you can go higher! Visually, I think AV1 adds less DCT-ish noise, but tends to blur small, low-contrast details a little more than x265. (Compare the ends of tree branches on the default image, Crepuscular Rays, or faraway tiles and details in Mercado dos Lavradores. Haven't done objective measurements though--you may disagree.) I, personally, like AV1's tradeoffs better, at least from what I see there. Patches of ringing draw attention, whereas slight blurring is something we're used to ignoring since it comes from lots of things besides codecs; natural vision and camera images aren't completely sharp either. (Dunno why AV1's that way, but might be that it does more aggressive deringing with that filter Monty's posting about next.) The photo caption on the comparison in this story suggested the same conclusion re the detail vs. noise tradeoff, for what it's worth: https://www.cnet.com/news/google-mozilla-av1-photo-format-could-outdo-aging-jpeg/ https://www.cnet.com/news/google-mozilla-av1-photo-format-co... The paper on the deringing filter included subjective tests, and it suggests that test subjects liked its results more consistently than you might guess from the changes in objective metrics--small sample size though: https://arxiv.org/pdf/1602.05975.pdf https://arxiv.org/pdf/1602.05975.pdf Sometimes you may want x265's bias towards more detail; in the Tiger image, AV1 "cuts" the whiskers near the top of the image shorter than x265 does. Tricky to 'teach' encoders that some detail is more important than others despite both being equally low size or contrast. Given they're both a ton better than JPEG, the big advantage to AV1 for a random developer is the openness; we can use it w/out needing to either take a legal risk or lean on the implementation of some company that's dealt with the HEVC patent mess for us.
- niftich 9y ago
  To my eyes and preference, both x265 and AV1's stills are far less blocky and far too smoothed than I'd like, but then again I've been an unabashed proponent of less smoothing [1], having lived through early H.264 encoders' blurry messes. Once I get past that, some of the test images look better on AV1, and some look better on x265. Some of them kill detail here, some of them kill detail there. I think this is a victory -- both of them perform really well, considering, and allocating a bit more bits to get it looking nice is always an option. It's good to have a royalty-free codec that's actually good, and with much more profile and fanfare than just a weird garage-band secret project like VP9 was. And with widespread hardware support, it may actually make a dent in JPEG's marketshare. [1] https://hn.algolia.com/?query=niftich%20fake%20sharpness&sort=byDate&type=comment https://hn.algolia.com/?query=niftich%20fake%20sharpness&sor...

bhouston 9y ago

I was wanting this just a few week ago: https://news.ycombinator.com/item?id=16709626 https://news.ycombinator.com/item?id=16709626 I didn't realize it already existed. Amazing.

Adamantcheese 9y ago

So it's basically what webp is to webm? Also how does it compare to my favorite image format, FLIF? http://flif.info/ http://flif.info/

bhouston 9y ago
About 95% smaller than FLIF I would think.
- adamzochowski 9y ago
  How is it 95% smaller? FLIF is a lossless compression, and AV1 in lossless mode can't do magic. It can't do 95% better. https://wyohknott.github.io/image-formats-comparison/report.html https://wyohknott.github.io/image-formats-comparison/report.... FLIF is reported at 59.57% avg space savings AV1 (2018-02-22) is reported at 66.02% avg space savings
  - notatoad 9y ago
    presumably it's 95% smaller because it's not limited to lossless mode?
    - simcop2387 9y ago
      FLIF is also not limited to lossless mode, http://flif.info/lossy.html http://flif.info/lossy.html
      - zaarn 9y ago
        And FLIF's lossy mode is as good as AV1's?
        simcop2387 9y ago
        That part I don't know. I haven't seen a comparison between them, but it's compared well at least against BPG, or whatever the h264 based format was. I don't think i've seen a comparison against h265, let along AV1. It'd be interesting to see. Even if they're about as good, I like FLIF's setup better since it enables progressive decoding which could be a big step up for slower connections.

ComputerGuru 9y ago

I found no mention in the document of CMYK color space or mandatory dpi attributes, both of which are severe pain points for PNG images. Does anyone know about this?

cbhl 9y ago
AV1 (and thus AVIF) appears to be designed for screens (works in YUV color space). If you want to print, PDF is probably still your best bet.
- paulkerr 9y ago
  AV1 can also encode RGB color space. Such usage would probably be more prevalent in the still image use case.
- ComputerGuru 9y ago
  Actually I munged two issues. CMYK is for print but DPI is for retina images.
awalton 9y ago
They have a metadata block type defined and you can shove whatever metadata format you want into it (EXIF, etc), and it has "image properties" for the necessary bits (presumably DPI would land itself as an image property, but I can see leaving colorspace out by assuming a "sane" default of something like sRGB). Since there are plenty of reasons you don't care about this metadata (e.g. to reduce the transfer size on images that are requested millions or billions of times by a web browser), as much as possible it should be left optional. But all of this seems to be pretty much draftware at the moment, so we'll see how things shake out.
- paulkerr 9y ago
  You are correct: the specification is a work in progress which we plan to wrap of quickly. The approach is not to over-specify the format with features/options initially, but begin with a functional sub-set of HIEF and MIAF. The target is to provide an HDR/WCG successor to JPEG. This work is being done by a working group in the Alliance for Open Media.

askvictor 9y ago

The name will just lead to confusion. AVI is already a well-known and much-used video (container) format, with no relation to AVIF or AV1. As discussed in AV1 discussion yesterday, AV1 will already cause confusion, and possibly coding and security errors. This name will lead to more of the same. Regular expressions, in particular.

runesoerensen 9y ago

Facebook also posted some performance tests comparing AV1 to x264 and libvpx-vp9: https://code.facebook.com/posts/253852078523394/av1-beats-x264-and-libvpx-vp9-in-practical-use-case/ https://code.facebook.com/posts/253852078523394/av1-beats-x2...

nothis 9y ago
Does any of this translate to still images? How much better would it be compared to, say, a JPG? I assume somewhat better but it would have to be magnitudes to take over.

jwatte 9y ago

Oh, look, another image format without support for stereo, cube layout, or MIP maps! I mean, it's not as if media is going 3D and VR right this moment (or even in the last few years) or anything...

pedro80 9y ago

It's absolutely irrelevant what container it uses (HEIF/ios/mp4), more over from my experience it was pure pain in the ass to deal with all these boxes and different ways different products produce them. I would never use it for an image format. Last time I checked it seemed like Nokia was a sinking ship, why would anyone board it?.. Keep it separate: make something that encodes image data separately and then use some container that to wrap it nicely. Same shit that works for jpeg should be reused IMO, it just works (all that metadata), simply replace coded image data with a better encoder (AV1)

pedro80 9y ago
when I see that nokiatech "tech" where some dudes make these nice documents on how to pack images in mp4, all reaction I get: WTF?! don't see no tech in there. All that shit exists for videos, treat image like a video file that has a single first i-frame and nothing else, that's what AV1 in mp4 is all about. Is their tech about limiting possibilities with mp4? I didn't ever read their tech and wouldn't bother spending even a minute. Like a year ago I started reading it and was WTF is this bullshit, a doc on image container format!??.. well, still all different devices will produce shit load of random mp4 structures, what's the point, you'll always have to deal with that pain in the ass that comes with all these possibilities with mp4 and you basically need a full blown lib to deal with it you pretty much cannot code it manually to parse mp4 to get coded image and show on the screen.