9 ms·
What's Going on with HEIF and Mac OS 10.13
- treve 9y agoIt seems a bit crazy that for a new image format, developers need to rely on an operating system shipping an API, instead of depending on an (open-source?) library.
- unfamiliar 9y agoPresumably both are options, but the OS library has hardware acceleration.
- crooked-v 9y agoIt looks like FFmpeg can do it, albeit in a somewhat roundabout way. http://jpgtoheif.com http://jpgtoheif.com
- dchest 9y agoPatents.
- ComputerGuru 9y agoOne gets you guaranteed immunity if you sign up for three different patent pools and pay a massive arm and a leg. The other claims to be I encumbered but the reality isn’t so clear.. but you’d probably have Google’s lawyers to help you out if it came to that.
- redial 9y agoThey can do it themselves if they want to. Apple is including support with High Sierra for everybody but anyone can add support to any formats on their own apps without involving Apple at all. Many apps, like VLC and HandBrake for example, have been supporting H.265 for a while.
- TheAceOfHearts 9y agoIt looks like this is also true for mpv. Honestly, having to deal with codecs is a waste of time from the user's perspective. I seriously love how players like mpv and VLC can play pretty much every format under the sun without additional configuration.
- ulfw 9y agoWithout any GPU support sadly.
- pornel 9y agoThe format is incredibly complex kitchen-sink of features and legacy ISO specs. There is an open-source library for it, but it's written in C++, and issues filed about it make it look like it's insecure (i.e. a large complex C++ codebase written by less than an expert) and not even entirely compatible with Apple's implementation. So I think it's fine to leave HEIF as basically an internal format of Apple's camera app, and for everything else export to formats more suitable for interchange (e.g. BPG which uses same the compression scheme, but is a simple lightweight container).
- jacobolus 9y agoI dunno, I think the container format sounds pretty reasonable, at least at a high level (I haven’t tried to implement it or anything). Many of those features [choice of lossy or lossless compression; extensible embedded metadata; multiple images per file; animation; transparency; embedded thumbnail; non-destructive indication of preferred crop/rotation; choice of color spaces and bit depth; ...] are extremely useful, but currently are only supported (in conjunction) by completely proprietary app-specific formats like PSD or the like. I can imagine at least a dozen use cases for this format that current formats don’t serve at all.
- TD-Linux 9y agoSee this post implementing it in ffmpeg: https://ffmpeg.org/pipermail/ffmpeg-devel/2017-August/215003.html https://ffmpeg.org/pipermail/ffmpeg-devel/2017-August/215003...
- jacobolus 9y agoI guess the main possibilities are (1) use an existing openly specified format, for which there is already at least some published working code, even if it isn’t perfect, or (2) invent a brand new format, and hope you don’t screw it up somehow by overlooking something important. I’m not entirely convinced #2 would have been any better (does Apple have a good reputation for inventing data serialization formats?), and inre #1 I’m not sure if there’s any extant choice better than this new format with support for all of the uses Apple cares about. Considering that all the widely supported formats throughout the industry have various severe limitations so that it’s impossible to cover the typical use cases (even for just web images, say) using less than 3 or 4 completely different formats, it’s not clear that anyone else has figured out how to really solve this one either. Too bad that the world couldn’t find someone with good taste, extensive experience, and a forward-looking vision to make a more future-compatible image format about 15–20 years ago.
- pmarreck 9y agoHow does WebP compare?
- niftich 9y agoKeep in mind WebP is essentially a VP8 I-frame in a RIFF container [1]. VP8 is the predecessor video codec to VP9; VP9 being the codec targeted by its makers to be in competition with HEVC which is commonly deployed inside HEIF -- whether they succeeded in this promise is an exercise left to the reader, possibly by following sources such as the ones compiled on Wikipedia [2]. But not all compression techniques that are applicable to video are relevant with still images. WebP's performance has been evaluated and some have found it lacking [3], while some have found it good [4]. Discounting the fact that the HEIF container has some useful features on its own, the picture inside is often an HEVC I-frame, which is the same idea as Fabrice Bellard's BPG. Therefore, comparisons between VP8 and BPG may also be indicative of typical HEIF-as-it-will-be-typically-deployed performance. [1] https://developers.google.com/speed/webp/docs/riff_container https://developers.google.com/speed/webp/docs/riff_container [2] https://en.wikipedia.org/wiki/VP9 https://en.wikipedia.org/wiki/VP9 [3] https://en.wikipedia.org/wiki/WebP#Criticism https://en.wikipedia.org/wiki/WebP#Criticism [4] https://blog.cloudflare.com/a-very-webp-new-year-from-cloudflare/ https://blog.cloudflare.com/a-very-webp-new-year-from-cloudf... [5] https://bellard.org/bpg/ https://bellard.org/bpg/
- gribbly 9y agoPersonally I'm not impressed with the quality of the lossy encoding, it's not enough of a improvement on JPEG to make it worthwhile, for lossless however it's the best format I know of right now, compresses MUCH better than PNG, and both compression and decompression is MUCH faster. FLIF compresses better than webp, but it also compresses/decompresses more slowly, and the difference in size is quite small so overall for lossless compression, WebP is my choice.
- JyrkiAlakuijala 9y agoOh, thank you!! Music to my ears! I designed the WebP lossless format. Our most recent effort PIK tries to achieve the same in the lossy world that WebP lossless has done in the lossless coding -- a practical, efficient, fast and simple compromise.
- microcolonel 9y agoWell, that's a two-pronged issue: one is that it is a patent minefield unlike ever before seen, the other is that it is incredibly slow in software at any quality level. Also, it is legally terrifying to build anything on HEVC, I'm frankly surprised Apple is even trying. The basic license fees are easily ten times as much for HEVC as they were for H.264, and there are known applicable patents which aren't even in the pools (both pools! there are two!). There are also potentially license fees that apply to individual performances/transfers of content, so if you're Netflix or something you'll end up paying for every unit of content shipped. For the lot of these reasons, the Alliance for Open Media (including the Daala team (Xiph and Mozilla), The Thor team (Cisco), Microsoft, Nvidia, AMD, Intel, Mediatek, Netflix, Google, and a good couple dozen other major players) has formed and will probably be freezing a standard bitstream format this year on a format that outperforms HEVC for a given bitrate, a given CPU time target, and on energy consumption (and implementation complexity) in hardware.
- gribbly 9y agoYeah, while I can understand Apple picking up HEVC, this image format makes very little sense to me. Outside of Apple I can't see this have any uptake due to being patent/royalty laden, which means that images you encode using this format need to be reencoded again (and degraded in case of lossy, which is the most common type) in order to be shown on the web or on other operating systems/devices.
- jws 9y agoMeanwhile, every Apple user’s photo library will be half the size compared to JPEG. For many users photos are the single largest category in their iOS device storage. More importantly for Apple, they (want to) store everyone’s photos in their cloud, so they need half the storage and bandwidth to pull that off. For sending to other people, most photos are going to be downsized anyway so a transcoding isn’t the end of the world.
- ksec 9y agoIt is very likely in the next 12 - 24 months time, every Smartphone shipped will have paid the HEVC patents. So while the PC industry may be a little different, all mobile phone will have HEVC support soon. Now whether the software for HELF support will catch up is another question.
- akaru 9y agoDeveloping a macOS app that targets HEVC/HEIF, I was wondering this myself. Now get this--HEVC encoding is faster on a last-gen iPhone than it is on a top-of-the-line current MBP!
- dmitrygr 9y agoBecause intel does not have HEVC encode hardware and A11 does?
- akaru 9y agoNo, they both do. And I'm talking about the A10 even. Still faster than an i7/Iris Plus 650. Of course, the graphics hw is killer in iPhones, so it's not such a surprise. But it makes you do a double-take.
- brisance 9y agoThat’s interesting; do you have a reference where I can read more about this?
- astrange 9y agoYou can always make an encoder faster than another encoder. What you have to ask is - how good is it for the speed? (This is called rate-distortion optimization.) Better yet, how good is it for the bitrate and speed? (This is the really hard part.)
- jack1243star 9y agoNitpick, but RDO refers to trade-off between bitrate and distortion (quality), IIRC
- astrange 9y agoSometimes you're limited by time more than bits, but they're both rates.
- ulfw 9y ago
- deleted 9y ago[deleted]
- alariccole 9y agoIt was likely due to HEVC licensing. HEIF is just the container, so they still should’ve provided ways to generate .heif files using other codecs like JPEG. They may have assumed that many people would associate HEIF with HEVC (what would technically be a .heic file), and wanted to avoid confusion. But we’re still confused!
- ken 9y ago"Apple says it plans to add HEIF encoding for Macs in a future release" -- https://arstechnica.com/gadgets/2017/09/macos-10-13-high-sierra-the-ars-technica-review/8/ https://arstechnica.com/gadgets/2017/09/macos-10-13-high-sie...
- 0x0 9y agoBold (nay, courageous!) move to launch a JPEG replacement and not follow up with desktop support. What happens if you edit a HEIF in Photos.app, and perhaps try to share a flattened version via imessage, email or upload to wordpress? Do you get a transcoded JPEG out? (What's the quality loss like for that case?) Without industry buy-in and high quality open source support, I'm struggling to see how this will ever expand beyond archiving iPhone camera rolls...?
- redial 9y agoTo avoid compatibility issues every time an image leaves the device it does so as a JPEG. Even if nobody else supports the new format, in their mind it's still a win because of the space saved on the device itself.
- lathiat 9y agoFor some reason, discord sharing photos fails with HEIF images.. it uploads as 'unknown' with no extension and doesn't display. Not sure if they somehow accidentally opted in to getting HEIF from the chooser or what.. but there you go.
- dilippkumar 9y ago> Bold (nay, courageous!) move to... Something about this sequence of words reminds me of Kruppe (From the Malazan Book of the Fallen series).
- MBCook 9y ago> Without industry buy-in and high quality open source support, I'm struggling to see how this will ever expand beyond archiving iPhone camera rolls...? It wouldn’t surprise me to see Microsoft support this, or perhaps camera makers. After all it’s 50% smaller than an equivalent JPEG, thats a nice feature. There’s nothing stopping google from supporting it other than the fact that they seem to like to force their own standards on everyone else (locking 1080p+ YouTube behind VP9). Even if it’s only available in the Mac/iPhone ecosystem… that’s a HUGE ecosystem. Anytime something is being sent over iMessage you can be pretty sure that the receiving device can decode that image format so you don’t have to send a JPEG or an H264 file. The ability to effectively double the capacity of iPhones when it comes to taking pictures and/or video is a really big deal. Combined with the fact they finally got rid of the stupid 16 GB tier and even the cheapest iPhone can now hold a LOT more pictures.
- taf2 9y agoHow does it compare to WebP?
- astrange 9y agoIt's at least a generation improvement. WebP is not very good; skal standardized it way early, before Google let anyone else review VP8.
- ComputerGuru 9y agoThey should have held off until VP9. Everything about VP8 (and, by extension, webp) was such a bungled mess. You come out with a format that barely competes with something released 5 years earlier then release an image format based on it while its superior replacement is being developed and close to release? AVC was released 10 years prior to HEVC; it had a good run and that encouraged adoption. Google barely got people to support their inferior (though mostly free) replacement before they obviated it not five years later, thus guaranteeing no one would ever bother with hardware encoding for an ever changing always outdated on arrival family of codecs. A real shame.
- alphabettsy 9y agoYou can generate HEIF with FFMPEG right now, but that’s hardly a solution.
- NamTaf 9y agoI'm confused; I have an 8 and I have High Efficiency selected, but it appears I can take a photo and see it and open it on my work Win7 PC. It's a .jpg file format and seems to work just fine. The mystery deepends with video: when I do a 4k60 video, MPC-HC is listing it as still using H264 but I thought 4k60 required H265: Video: MPEG4 Video (H264) 3840x2160 60fps 90829kbps [V: Core Media Data Handler (h264 high L5.1, yuv420p, 3840x2160, 90829 kb/s)] Audio: AAC 44100Hz mono 92kbps [A: Core Media Data Handler (aac lc, 44100 Hz, mono, 92 kb/s)] What am I missing?
- jmreid 9y agoI believe it converts those files to compatible formats when you transfer them off the device. The exception seems to be when you have macOS 10.13 and use Photos, the images and videos stay in HEIF and HEVC.
- brigade 9y agoIn Settings / Photos, there's an option to keep the original hevc/heif when transferring to PC/Mac.
- NamTaf 9y agoPerfect, found that now thank you! A bit weird to have it separate to the option to take in that format, though sort of understandable in a twisted logic since it's to do with the photos app not the camera app.
- Sephr 9y ago4K60 does not require H265, it's just that most H264 chips can't handle realtime 4K60 encoding.
- vim_wannabe 9y agoSo when AV1 happening? Not that mac would support it but was just wondering.
- ulfw 9y agoWithout write libraries it's a dead product. What surprises me is how fast HEIF files are on my iPhone 7 Plus (instantaneous browsing through photo library on iPhone. No different to JPEGs), yet how sluggish it is on my 2017 MBP 13". Select a .heic on your Mac and spacebar-preview it. Then select another one in the same folder and you'll see it takes a good few seconds (depending on size of image).
- JyrkiAlakuijala 9y agohttps://www.github.com/google/pik https://www.github.com/google/pik could be an opensource alternative, more dense for photographs than AV1 or HEIF, less complex, and decodes 100x faster https://encode.ru/threads/2814-Psychovisual-analysis-on-modern-lossy-image-codecs?p=54189&viewfull=1#post54189 https://encode.ru/threads/2814-Psychovisual-analysis-on-mode...