10 ms·
HEIF – High Efficiency Image File Format
- frik 9y agoiOS11 now takes photos in HEIF instead of JPEG by default. I think that's why it's posted here.
- 0x0 9y agoWonder how this will work with random websites' photo upload features. Will all other browsers break because they are serving HEVC user-uploaded content? Or will iOS transcode to JPEG on upload (triggering double lossy encoding?)
- roywiggins 9y agoHow many websites don't already re-encode most of the JPEGs they're handed? Facebook does- Flickr does, though sometimes you can request the original file specifically. Imgur probably does.
- dawnerd 9y agoThey mentioned you can share the photos like normal so I assume they'll do on the fly conversion.
- hutattedonmyarm 9y agoThey said you can share it as jpg. I assume it works like live photos: If the receiving app (or website) doesn't explicitely say it supports HEIF it'll receive a jpg
- fao_ 9y agoWhat benefits does this have over PNG, farbfeld, etc. ?
- agentgt 9y agoIf you bothered just navigating the site you would have found this page with a table (table II): https://nokiatech.github.io/heif/technical.html https://nokiatech.github.io/heif/technical.html
- fao_ 9y agoA lot of the items on that table really aren't very self-explanatory (Derived image, for example, obviously has a clearly defined meaning in this context, and it appears to be contrary to what I expect it to be). They assume a background that I don't have, and at the moment I don't feel that I have the time to spend obtaining it. I was asking for more of a layperson's comparison of the differences. Read it as "Why should I prefer HEVC over PNG, when they appear to be mostly identical".
- deleted 9y ago[deleted]
- masklinn 9y agoPNG is a lossless format, it's pretty much pointless for photos. IDK about farbfelt but it seems like a TIFF competitor more than anything else. This is an (significant) improvement over JPEG.
- fao_ 9y ago> a lossless format [is] pretty much pointless for photos Why is that? Every edit you make to that photo means that the photo will degrade in quality. I always thought JPEG artifacts and compression lossage was a bad thing for photos, not a good thing?
- deleted 9y ago
- vim_wannabe 9y agoIf Apple started to support natively this in Safari, which one would succeed: HEIF or WebP?
- douglasfshearer 9y agoApple just announced support at WWDC.
- floatboth 9y agoJust look at Chrome (and everything Chromium based) vs Safari market share… https://caniuse.com/#feat=webp https://caniuse.com/#feat=webp
- cflat 9y agoMy bet is on HEIF. It's far superior. Full subsampling support, >24bit colours, etc, etc
- amluto 9y agoIt's based on HEVC. I wonder how bad the patent situation is.
- MontagFTB 9y agoIt would seem the Nokia open source implementation is what's limited by the patent rider. I don't think that applies to the format at large.
- BillinghamJ 9y agoNokia and Apple just recently resolved their differences on patents it seems
- TD-Linux 9y agoPretty bad. https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding#Patent_License_Royalties https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding#P... If you choose to implement this format, I hope you enjoy negotiating licenses with at least four different entities so that you can display pictures.
- vbernat 9y agoThis seems pretty similar to BPG: https://bellard.org/bpg/ https://bellard.org/bpg/
- masklinn 9y agoThe core idea does, but according to the comparison table HEIF seems to go quite a bit beyond BPG.
- Adamantcheese 9y agoI wonder how it compares to FLIF, which looks like it has a formal release finally. http://flif.info/ http://flif.info/
- agentgt 9y agoIt looks like FLIF doesn't really support lossy well (its sort of fake lossy). "FLIF does not have a lossy mode, but interlaced files can be decoded progressively so we can simply use something like dd if=lossless.flif of=lossy.flif bs=1024 count=5" You can see this in the examples where BPG does well in stream loading: https://nokiatech.github.io/heif/technical.html https://nokiatech.github.io/heif/technical.html HEIF would probably be similar to BPG in performance. Still FLIF looks pretty darn interesting.
- sprash 9y ago> It looks like FLIF doesn't really support lossy well I don't know what you mean by well. As far as I'm concerned lossy FLIF beats JPEG easily. Here is a comparision: http://flif.info/lossy-artifacts.html http://flif.info/lossy-artifacts.html
- agentgt 9y agoWhoops looks like I pasted the wrong link. Here is the correct link: http://flif.info/example.html http://flif.info/example.html And yes "well" is relative and what I meant by that is that its not readily easy for a normal user to do it. BTW I think Flif is damn good particularly progressive decoding. Its unclear how good HEIF progressive decoding is.
- deleted 9y ago[deleted]
- emn13 9y agoYeah, even on that fair; jpg-unfriendly example jpeg still clearly beats flif above 22kb. And below 22kb - sure, FLIF is less bad, but it's still unusable. Also, the examples below 22kb are a little unfair to most codecs in practice, since you'd get far better results by downscaling first.
- tehabe 9y agoSo HEIF is for HEVC what WebP is for VP9.
- niftich 9y agoRoughly, BPG [1] is for HEVC as what WebP is for VP8 or VP9, in the sense that you take an I-frame out of a video format and stuff it into an established, classic image container. Compared to this, HEIF is a different way of packaging HEVC frames; namely into ISOBMFF (MPEG-4 Part 12), the most basic level of MPEG container. This comes with benefits [2] that come along with using that particular container, but also drawbacks, because with some of the advanced features of HEIF you're essentially describing a sequence of frames -- which is almost like video -- but you're doing it in a way incompatible with the way you'd describe video. I'd be curious if the two "representations" are losslessly convertible, for example. [1] https://bellard.org/bpg/ https://bellard.org/bpg/ [2] https://nokiatech.github.io/heif/technical.html https://nokiatech.github.io/heif/technical.html
- jpap 9y ago> I'd be curious if the two "representations" are losslessly convertible, for example. They are: you could just extract the HEVC NAL units, and re-write them into a MP4 or QuickTime container, making sure to properly place the codec configuration box, etc. HEIF also goes beyond a sequence of frames, in that it can describe alpha planes, depth maps, tiling, etc. In that case there might not be an analog with a standard video. If you really wanted to decompose a HEIF container, you might choose to extract the raw media into elementary streams (for HEVC or AVC; or if you're using the JPEG codec, just plain JPEG files) adjacent to any metadata like Exif, etc. This is essentially what Nokia's conformance files are [1]. [1] https://github.com/nokiatech/heif_conformance/tree/master/bitstreams https://github.com/nokiatech/heif_conformance/tree/master/bi...
- threeseed 9y agoVP9 only really compares with H.264. It's more like HEIF is for HEVC what WebP is for AV1.
- amelius 9y agoWhy don't we store a url to the decoder inside images and other compressed files for maximum flexibility? I mean, looking at WASM, sandbox technology is sufficiently strong. And performance and availability aren't really an issue either, because for specific cases we can fall back to what we are doing now.
- masklinn 9y ago> Why don't we store a url to the decoder inside images and other compressed files for maximum flexibility? Image formats are already some of the biggest attack surface of modern systems, "executable" image formats (hello PDF) have an absolutely ghastly track record there. And not being able to open an image if you don't have an internet connection sounds dreadful.
- repsilat 9y ago> not being able to open an image if you don't have an internet connection sounds dreadful Caching the decoder smells like having already downloaded the image library. Mostly just different in that "first run time" is now always the same thing as "install time".
- derefr 9y agoRe: attack surface—PDF is complex because it requires a ton of interaction (you can fill PDF forms, etc.) Image decoders should just be pure functions—binary stream in, matrix of pixel structs out. Easily sandboxed—you shouldn't need access to any system calls while doing that decoding. Re: requiring an Internet connection—you still do require an Internet connection to display an image, if it's in a format whose decoder you don't currently have installed; you just currently have to manually 1. figure out what the format even is, 2. select a library package for a decoder for that format, and 3. install that package. Presumably such libraries would be able to register decoder-URLs they are (non-reference) implementations of—like they register MIME types today—so your system would be able to display standard formats no problem; it'd just be the weird rare or new ones that would trigger the zero-install process for a decoder.
- amelius 9y ago
- niftich 9y agoThis seems conceptually most similar to Fabrice Bellard's proposal 'BPG' [1], which was essentially a lightweight wrapper around an HEVC I-frame. The format got small amounts of attention in encoding circles, but didn't pick up mainstream traction. The image encoding layer of HEIF is the same HEVC, but the container is MPEG-4 Part 12 (the Quicktime-descendant ISO Base Media Format; the core behind .mp4, .3gp, etc.), which theoretically gives it wide support. In this structure, HEIF supports image sequences, which will help it compete with animated GIF, GIFV (which is really just a short MPEG-4 Part 14 file containing usually an H.264 video track), animated WebP (who uses these?), and WebM. This format doesn't really blaze new ground (EDIT: it does in the sense that it defines a container format to express image-y constructs like still images and sequences of images in an ISO media container, but see my other comment that asks how this is similar but different to video [3]), but if this repackaging and the resulting code donation and political clout helps it gain traction, we still would gain a lot. The problem, of course, is always with backwards-compatibility. WebP was aggressively promoted by Google the same way Microsoft used to promote its quirky Windows Media formats back in the early 2000s, but the WebP and non-WebP camps are still largely separate. This is unfortunate, because WebP in my opinion really isn't very good, and a backwards-compatible compressor on top of JPEG, such as Dropbox's Lepton [2] achieves similar results. Any HEVC-based image compressor is necessarily incompatible with JPEG; so broad buy-in would be required from makers of software and hardware products, from operating systems, file managers, image viewers, image editors, cameras, and the like, to really enjoy its improved compression rates, vs. being just another incompatible format that only functions inside controlled ecosystems. The video codec AV1 currently in development was supposed to be a grand alliance of disparate companies to agree on a common format for the future, and rectify the WebM vs. non-WebM split, but continuing to promulgate sophisticated formats based on work outside of this scope (such as MPEG- or ITU-derived custom formats) just muddies this further. [1] https://bellard.org/bpg/ https://bellard.org/bpg/ [2] https://news.ycombinator.com/item?id=13230805 https://news.ycombinator.com/item?id=13230805 [3] https://news.ycombinator.com/item?id=14490734 https://news.ycombinator.com/item?id=14490734
- zerocrates 9y agoIt mentions that it supports not only sequences but inter-frame prediction (so P- and possibly B- frames I assume), which is sort of surprising to me but of course makes sense especially given Apple's "image burst" usage. I'm also suprised to learn that WebP apparently supports P-frames in its animations. What's really different between this and the existing usages of the MP4 container for images, though? Is it just the packaging as an "image" format, or specification of more image-y metadata?
- namingwaysway 9y agoWhat's the support for popular image process libs?
- zokier 9y agoWhile the basic idea of using HVEC for still image compression is good, I'm somewhat concerned on how complex the whole format, especially the container, for HEIF is. It is kinda cute and from one viewpoint pragmatic to reuse existing container format, but I'm afraid the flexibility might have unforeseen and undesirable side-effects. And of course there is the problem of different implementations supporting different subsets and extensions of the format.
- TD-Linux 9y agoThey've already specified three different codec mappings - JPEG, AVC, and HEVC - so to open all HEIF files, you presumably need to implement all of the decoders. Worse, there is a financial incentive to implement only a subset of the decoders - HEVC is more expensive than AVC which is more expensive than JPEG. Even things like 4:4:4 color are more expensive than 4:2:0 color for HEVC.
- abritinthebay 9y agoIf PSD's have shown us anything: it's that complexity of the container doesn't matter to adoption, software support does. It's a standard format, it's faster than the competition, it has just as good (or better) quality vs compression metrics... ... all it needs is support and it'll take off. Apple is a good start for that.
- lsiebert 9y agolike usb-c and thunderbolt compatibility, I am worried there will be a lot of confusion and what works with what. Less in the area of devices that attach to your hw direction, but more when it comes to consumer devices with HW decoder support and getting these formats to display.
- hoschicz 9y agoIt's really sad that Apple decided to support the proprietary H.265 and this image format instead of the royalty-free WebP and WebM.
- threeseed 9y agoI think you're confused. WebM is a container not a codec. And either way VP9 and AV1 are simply inferior codecs to H.265 in almost every way. And as for royalty free well that's because MPEG-LA has't ever sued but they did give a royalty free license to Google for patents infringed by VP9. And since Google accepted it does indicate that AV1 is likely to also be patent encumbered. We would all like a royalty free codec but it isn't happening anytime soon. And so failing that I would prefer to go with the codec with the best support which by far is H.265.
- Veratyr 9y ago> And either way VP9 and AV1 are simply inferior codecs to H.265 in almost every way. Source? AV1 is already beating HEVC in compression and it's not even finalized yet: https://youtu.be/lzPaldsmJbk?t=2416 https://youtu.be/lzPaldsmJbk?t=2416
- threeseed 9y agoVP9 versus H.265: http://www.streamingmedia.com/Articles/Editorial/Featured-Articles/Netflix-Finds-x265-20-More-Efficient-than-VP9-113346.aspx http://www.streamingmedia.com/Articles/Editorial/Featured-Ar... Also the link you posted is using x265 which as the article suggests was ranked 4th out the 6 available encoders: http://compression.ru/video/codec_comparison/hevc_2016/MSU_HEVC_comparison_2016_free.pdf http://compression.ru/video/codec_comparison/hevc_2016/MSU_H... And whilst AV1 and H.265 are pretty close in PSNR where it is inferior is in vendor support. There are TVs for example available today that natively support H.265 whereas AV1 is still in development. Not to mention all of the terrestrial broadcast providers all using H.265.
- Veratyr 9y ago
- pornel 9y agoIt makes a lot of sense for Apple to use H.265 as a base for the format. It's much more efficient than all the other JPEG-killers (easily beats WebP by a wide margin). Apple already pays for the patents, and has invested in H.265 hardware and software for H.265 video. However, the HEIF wrapper for H.265 strikes me as quite complex. It brings baggage of ISO wrapper formats and tries to support everything ever crammed into any image-ish format. That, in addition to patent licensing, may be another difficulty in widespread support for the format. It will be hard to implement robust, secure and fully-featured players.
- NuDinNou 9y agoLet's wait and see what AOMedia[0] response will be to HEIF [0] https://en.wikipedia.org/wiki/Alliance_for_Open_Media https://en.wikipedia.org/wiki/Alliance_for_Open_Media
- jpap 9y agoWhile the full standard text and machinery can get quite complex, you can still construct relatively simple files housing a single image, thumbnail, and Exif that isn't that much more complex than a typical JPEG file. Compared to JPEG, where you need to define quantization tables, Huffman tables, etc. in marker segments, much of the codec-related complexity in HEIF is layered within the compressed image payload (e.g. HEVC NAL units). Besides the coding efficiency, JPEG really hit the big time because of the readily available libjpeg cross-platform implementation. In this case, HEIF can leverage existing implementations of the ISO Base Media File Format box model; it builds heavily on that standard. Nokia is doing the right thing by releasing their format handling library, though perhaps they could loosen up their license to include commercial use. ;-) Hopefully there are some developers amongst us inspired enough to start a new open project that, like libjpeg, brings HEIF to the masses.
- deavmi 9y agoGood to see Nokia.
- throwasehasdwi 9y agoNokia owns several HEVC patents, in fact I think they're currently suing apple about it. This is a veiled attempt to get everyone to use their patented tech so we can relive the mp3/mp4 clusterfuck. You would be a fool to use this for anything. What's so bad about WebP? It's not the best thing in the universe but it beats JPEG and PNG and has a wide support base. With a shim it has native support in most browsers anyways since WebP is a just a single frame of video. And as a bonus you don't have to worry about someone suing you for royalties down the road.
- pornel 9y ago> What's so bad about WebP? WebP is based on VP8, which is now obsolete. It was meant to be a H.264 competitor, a generation behind H.265 and VP9/VP10. WebP compression efficiency is much closer to JPEG than H.265. This format beats WebP by a margin wider than WebP was smaller than JPEG.
- ika_ 9y ago>This format beats WebP by a margin wider than WebP was smaller than JPEG. I have been looking for this, do you know where to find a comparison between HEIF and WEBP?
- emn13 9y agoAnd people today still use jpeg routinely, so I'm not sure any of that is really a killing blow.
- throwasehasdwi 9y agoMost devices still encode and decode far more H264 and JPEG than newer generations. We shouldn't be stuck with the horse and buggy because we're waiting for cars to be autonomous. WebP is superior to JPEG and the most widely available alternative.
- astrange 9y agoWebP should not have been released. It was created by skal right before Google released VP8, and nobody had any time to work on the actual format or even see it before it was frozen. You could easily beat it in quality by now.
- TD-Linux 9y agoThe license in the provided implementation seems to be a custom, non-commercial, non-OSI-approved license [1]. It also includes a patent grant, but excludes codec patents, making it even more useless (Nokia can't license other people's HEVC patents, of course). It's based on the LGPL-licensed libde265 library. [1] https://github.com/nokiatech/heif/blob/master/LICENSE.TXT https://github.com/nokiatech/heif/blob/master/LICENSE.TXT
- slackingoff2017 9y agoLol Nokia has Codec patents on HEVC too. So they're granting a license that doesn't include their HEVC patents or anyone elses? Is this the library equivalent of a Honeypot? Maybe they're not getting enough royalties.
- bengotow 9y agoLicense notwithstanding, has anyone tried to create a HEIF file? There are a few samples on this site, but even given the code they've provided on GitHub I can't seem to create a HEIF image from another image format. Their code seems to take an H265 HEVC bytestream as input, which is fine, but I can't find anything that will build an HEVC bytestream for an image rather than a video...
- astrange 9y agoYou can easily turn an image into a video, or into HEVC, with ffmpeg.
- bengotow 9y agoHaha thanks - I didn't think it supported H265 but it looks like I just have to build it from source (https://trac.ffmpeg.org/wiki/Encode/H.265 https://trac.ffmpeg.org/wiki/Encode/H.265). Suppose that's still fairly "easily" ;-)
- bengotow 9y agoUpdate: I figured out how to do this and posted my findings here: http://jpgtoheif.com/ http://jpgtoheif.com/
- pcgamestime 9y agoAgree
- williamle8300 9y agoIf I could give unsolicited advice: focus on graceful progressive enhancements. Galleries, tiles, etc should be left to JavaScript.
- intoverflow2 9y ago"Have you got a 'heef' file of that photo please?" Could have been branded better if I'm honest. I know engineers generally don't care about stuff like this but some of us will have to say this term many times a day if takes off.
- PM_ME_YOUR_NDA 9y agoCan somebody please ELI5. Assuming I have iOS 11 on my iPhone and I take a bunch of pictures. Will I have to convert the pictures to JPG/JPEG/etc. myself when I decide to import them to my Mac/Windows machine?