9 ms·
Apple Backs AV1: What Does This Mean for the Future of Video Codecs
- Cknight70 9y agoWhy has Apple still not added the ability to view webms on iOS?
- robert_foss 9y agoBecause Google is a direct competitor to them.
- vetinari 9y agoApple traditionally ignored non MPEG codecs[1], VPx is ignored for the same reason, and not due to being developed by Google. [1] i.e. Vorbis, Theora, Flac, matroska containers, etc. Opus is supported in Safari for WebRTC because they have to, but is completely unavailable in other contexts.
- ZeroGravitas 9y agoThey didn't ignore Flac, they made an Apple-only knock-off of it.
- DaiPlusPlus 9y agoI speculate they needed a DRM-capable container in case the rightsholders demanded DRM when iTunes started selling lossless audio files.
- savoytruffle 9y agoALAC did show up at the same time as the first AirPort Express in 2004, and it is the format used to transport audio over AirPlay (first called AirTunes). That may have been the original impetus.
- dwaite 9y agoI was under the impression it was an AAC project that didn't get adoption outside the Apple ecosystem, not that it was Apple created. Do you know whether Apple was the submitter to ISO?
- niftich 9y agoIt sounds like when you say "an AAC project", you're thinking of MPEG-4 ALS [1], which was the codec chosen by MPEG for lossless audio in the mid-2000s and last updated in 2009. Despite ALS being blessed by MPEG, it never achieved much acceptance in the market. Its most direct ancestor was LPAC [2], developed at the Technical University of Berlin; but ALS included improvements from NTT Communication Science Laboratories and RealNetworks [3]. Apple developed Apple Lossless ("ALAC"), a different, unrelated [4] format, which was fairly similar to FLAC, but different in some minor ways (paraphrasing the FLAC dev's own words [4]). [1] https://en.wikipedia.org/wiki/Audio_Lossless_Coding https://en.wikipedia.org/wiki/Audio_Lossless_Coding [2] https://en.wikipedia.org/wiki/Lossless_predictive_audio_compression https://en.wikipedia.org/wiki/Lossless_predictive_audio_comp... [3] http://elvera.nue.tu-berlin.de/files/0737Liebchen2005.pdf http://elvera.nue.tu-berlin.de/files/0737Liebchen2005.pdf [4] https://hydrogenaud.io/index.php/topic,32111.msg279843.html#msg279843 https://hydrogenaud.io/index.php/topic,32111.msg279843.html#...
- tzahola 9y agoActually FLAC and Opus support was added to the Core Audio / AudioToolbox / AVFoundation APIs in iOS 11. Though the Opus decoder only supports raw packet decoding, i.e. you still need something to decode the container (usually ogg). Also, FLAC file decoding is horribly slow - compared to the FOSS reference decoder -, because for some reason the whole file is scanned through on the first read when using the AudioFile APIs (AudioFileReadPacketData). (I’m gonna make a demonstration project for this and send them a bugreport tomorrow.)
- microcolonel 9y agoSo bizarre to write their own decoder, especially given libFLAC is BSD licensed.
- tzahola 9y agoIt actually makes sense, because libFLAC has no public API for getting the compressed audio packets as is. Rice/huffmann/whatever decompression always happens inside libFLAC. With Apple’s AudioFile API you can decide whether you want the decompressed PCM samples, or the underlying compressed packets. You can get better energy efficiency if you offload the decompression step to the dedicated coprocessor. The difference is smaller than with hardware vs software h.264 decoding, but there is a difference. (For lossy codecs this also makes sense when you want to forward the compressed packets to a wireless speaker in order to avoid lossy recompression on Bluetooth transmission.)
- tzahola 9y agoFyi, here’s the demonstration project I talked about: https://github.com/tzahola/AudioFileFLACBug https://github.com/tzahola/AudioFileFLACBug
- jsjohnst 9y agoActually, VPx was technically an acquisition[0] by Google. Not sure it matters as they’ve evolved since, but it is somewhat notable none the less. [0] https://en.m.wikipedia.org/wiki/On2_Technologies https://en.m.wikipedia.org/wiki/On2_Technologies
- acdha 9y agoWhat user benefit would there have been? Any format added has a maintenance cost (ever notice how many security updates are found in AV code) so there would have had to be some compelling benefit versus the MPEG standards they already supported. That's especially important for battery-powered devices since the primary WebM codec (VP8) didn't offer a reason to switch from H.264 and would had to be implemented in software rather than using the optimized hardware on the device (this is why YouTube playback performance is so much worse on most systems unless you disable WebM support in your browser).
- gsich 9y agoUser benefit? They can play videos, vs. they can't. Huge benefit for a vastly used codec.
- untog 9y agoVastly used? I'm not so sure about that. Vastly used in tandem with H264 when required? Absolutely. If iOS can hardware decode H264 but not WebM it actually isn't in the user's interest to support WebM, because it'll reduce incentive to provide H264 versions of content, which means iOS users will experience battery drain faster. It might not be particularly fair, but it does make sense.
- wlesieutre 9y agoNative apps like VLC can play WebM on iOS with software decoding, but as you say it murders the battery. If Apple supported that in browsers I'd bet that Google or other companies would switch their video hosting over, much to the detriment of anyone who watches videos on an iPhone. It's an open standard so clearly they're doing the pro-consumer good thing!
- gsich 9y agoIt would give an incentive to Apple to support hardware decoding.
- pornel 9y agoThe boring answer is that they've already invested in H.264, including hardware decoding support, and it works well enough for them, so they don't need to add support for VPx.
- deleted 9y ago[deleted]
- abritinthebay 9y agoBecause it’s not as widely used as the hardware accelerated alternative?
- deleted 9y ago[deleted]
- shmerl 9y agoBecause in general they are jerks when it comes to free codecs support. The fact that they joined AOM is a major shift from their previous behavior and it most likely happened because they realized they have little choice and not because they wanted to do the right thing.
- zackkatz 9y agoThis article failed to mention that Apple is a founding member of the Alliance for Open Media, the organization that released AV1. > Founding members are Amazon, Apple, ARM, Cisco, Facebook, Google, IBM, Intel, Microsoft, Mozilla, Netflix and NVIDIA. > [aomedia.org](https://aomedia.org https://aomedia.org)
- cornstalks 9y agoAOM terminology sucks here. “Founding member” just means they have a seat on the board of directors (and it’s awesome to see Apple join that), not that they were truly part of the initial founding (Apple joined in late 2017/early 2018, which is well after AOM was publicly announced in 2015).
- wmf 9y agoThat's some kind of retroactive founding thing; Apple just recently joined after AV1 was completed. http://web.archive.org/web/20171226163021/https://www.aomedia.org/ http://web.archive.org/web/20171226163021/https://www.aomedi...
- shmerl 9y agoFounding there apparently doesn't mean those who joined when alliance was created, but one of the members which participates in decision making.
- adamnemecek 9y agoSounds like Apple all right.
- vetinari 9y agoI'm not sure, whether this propaganda piece should come with some more warnings. The paragraphs starting with 'AV1 mythbusters' are unbelievable: 1. The MSU report ("performance mentioned by AOM sources") with known methodology has to be verified, but that one demo with hand picked scenes and undocumented codec settings are totally the definitive answer. 2. On the performance side, we don't know the complexity of the codec yet. And on the cost side, HEVC is going to be more expensive by definition, due to being an rent extraction scheme. 3. You can't compare reference WIP codec with optimized one. The premise of the entire paragraph is just wrong.
- madeofpalk 9y agoAs soon as I saw the word 'mythbusters' I knew they had a dog in the game and wasn't going to give a fair and even summary, either way.
- biggc 9y ago> 2. On the performance side, we don't know the complexity of the codec yet. And on the cost side, HEVC is going to be more expensive by definition, due to being an rent extraction scheme. The codec is still years away from being widely adopted. Its encoder is still highly unoptimized, to the point of it taking tens of seconds to encode a single frame.
- vetinari 9y ago> The codec is still years away from being widely adopted. Its encoder is still highly unoptimized, to the point of it taking tens of seconds to encode a single frame. Sure, but that statement is valid for every single codec in development phase. AV1 is not an exception.
- Ace17 9y ago> You can't compare reference WIP codec with optimized one. The premise of the entire paragraph is just wrong. AOM dev here. Currently, aomenc takes ~100s to encode a single 4K frame. Which means to encode live 4K 60 FPS, you need to multiply the encoding speed by 6000 (!). Indeed, the codec isn't fully optimized yet, and the required performance improvement is huge. Of course this freaks people out! On the other hand, reference video encoders are famous for being ridiculously slow. The JM (H.264 ref encoder) had similar encoding times, and this didn't stop H.264 from being encoded live.
- mtgx 9y agoMaybe Apple will stop the nonsense and support the Vulkan API, too.
- tzahola 9y agoHaha, dream on!
- anfilt 9y agoMade think of this. It's from the Vulkan Programming guide. https://imgur.com/gallery/1pxYT https://imgur.com/gallery/1pxYT
- zanny 9y agoAn AV codec isn't a weapon to lock developers into your platform while wasting millions of dollars and thousands of man hours on pointless api ports. There are too many implementations of encoders / decoders for the various formats lying around to get away with that. Its not like they didn't try with Quicktime in the past (ish? .mov was just a crippled .mp4), but that never really made them any money I don't think.
- CharlesW 9y ago> .mov was just a crippled .mp4 That's backwards — Apple contributed the Movie file format as the basis of the MPEG-4 file format.[1] [1] https://en.wikipedia.org/wiki/MPEG-4_Part_14#History_of_MP4 https://en.wikipedia.org/wiki/MPEG-4_Part_14#History_of_MP4
- ComputerGuru 9y agomov significantly predated MP4
- m_eiman 9y agoMetal release date: 2014 Vulkan release date: 2016 (announced 2015) So since somebody made a non-compatible clone a couple of years later, Apple should have ditched their work and lost the power to make decisions as they see fit in the process? That doesn't sound like something that would appeal to them.
- shmerl 9y ago> As an industry, we hope that the HEVC licensors come up with a reasonable and fair solution to resolving the patent licensing issues. I don't think anyone cares about that (besides said patent holders). As an industry, we should hope that HEVC and its approach to licensing will go bust for good, and everyone will be using free codecs that will unshackle them from patent protection racket.
- becauseiam 9y agoIt's not just patent holders who care. Harmonic sells products where a large portion of the cost is derived from licensing codecs like AVC and HEVC. One of their contribution encoders[1] has separate software licensing for a variety of subsampling ratios. 1: https://www.harmonicinc.com/media/2017/06/Harmonic_DS_ViBE-CP9000.pdf https://www.harmonicinc.com/media/2017/06/Harmonic_DS_ViBE-C...
- shmerl 9y ago> Harmonic sells products where a large portion of the cost is derived from licensing codecs like AVC and HEVC. So, wouldn't they directly benefit from switching to free codecs? It will increase their profits by removing their licensing costs.
- Digital-Citizen 9y agoNot among the powerful, no. The public gets great benefit from FLOSS codecs (Opus, VPx, AV1, etc.) and containers anyone is free to implement without patent hassles (such as Matroska and the smaller variant used in WebM). Patent holders usually try to wrangle control of everyone (Apple being a part of a media codec patent-holding group meant Apple pushed those codecs and back when people cared what Apple picked that meant more power for Apple). In this respect, it's a class issue. In 1990 IBM (in "Think" magazine, #5) said they got considerably more value from cross-licensing than fees (in other words, looking at this in terms of money is wrongheaded) because cross-licensing gives them access to ideas). So, it's the power to control one's competition at play as well. This arrangement means (as Richard Stallman pointed out in essays and talks years ago found online in https://audio-video.gnu.org/video/rms-vub-2011-02-22.ogv https://audio-video.gnu.org/video/rms-vub-2011-02-22.ogv ) that the entire temporary monopoly one ostensibly gets from the patent system is a sham, a myth. Organizations that hold a lot of patents can always find ways to economically compel smaller patent holders into cross-licensing, thus eliminating any real value of patenting ideas for most.
- kalleboo 9y agoJoining the board doesn't mean anything about adoption. They were a part of the Blu-ray group as well, and never shipped a Blu-ray drive