9 ms·

It’s Opus, it rocks and now it’s an audio codec standard

167 points by nullc 14y ago

nvmc 14y ago
MP3 is getting very long in the tooth, not to mention the fact that it's encumbered by patents. A new codec would be nice, but hardware adoption of the codec will take a while. Long enough, dare I say, to kill it dead in the water.
- randallu 14y ago
  Why would you say that? Most "hardware" decoders are DSPs that run software loaded onto them at runtime. This goes for anything with an OMAP, Tegra or Samsung (Samsung's own decode block) SoC in (that I know for sure) and probably almost everything else. Tegra3 and OMAP4 can decode WebM in hardware, for example.
  - wmf 14y ago
    The problem is that they never develop firmware updates so if those DSPs don't support yet another open codec on release day, they never will.
    - gillianseed 14y ago
      perhaps, but even so that does not mean that it won't get supported in later generations. I'm sure people said the same thing about FLAC but it is now supported in lots of devices, like those running Android.
    - randallu 14y ago
      I'm pretty sure Tegra2 got firmware updates during its lifetime, and I know that TI do updates for OMAP. Those updates might not make it through to consumer devices, but that used to be the case for things like GPU drivers too. Google pushed an updated SGX driver for the Galaxy Nexus with 4.1, and Apple pushes updates to their own drivers every release.
  - nvmc 14y ago
    I should have worded my comment better. I mean that adoption by hardware manufacturers will be slow. What's the incentive? Even for a firmware patch? MP3 is `good enough', and everyone already uses it.
    - iSnow 14y ago
      Opus is royalty-free, in contrast to mp3. And it will be used in WebRTC. I can see some devices picking up Opus to support WebRTC and possibly some (not portable music players, obviously) dropping mp3. Obviously neither you nor I have a crystal ball, but I see Opus as the PNG of audio codecs - not displacing the lousy but entrenched de-facto standard (GIF/mp3), but slowly taking over at the technological edge.
  - pslam 14y ago
    Almost no hardware audio decoders use DSPs, or at least actually need to, since about a decade or more now. The manufacturers you mention use DSPs for bad reasons - usually the inertia of software already using a coprocessor and not being willing to license a new codec for the general purpose core. Audio decoding and encoding is an extremely low percentage of available CPU, and it's just not worth off-loading it. Video is another matter, at least when talking about HD. It's still very much worth using special purpose DSPs for that. For audio, the existence of hardware DSPs is simply not a barrier to entry.
kibwen 14y ago
Before someone chimes in with that XKCD strip: How does Opus compare to other codecs? Opus is distinguished from most formats for high quality audio (AAC, Vorbis, MP3) by having low delay and it is distinguished from most low delay formats (G.711, GSM, Speex) by supporting high audio quality. It meets or exceeds existing codecs' quality across a wide range of bitrates, and it operates at lower delay than virtually any existing compressed format. Further, the Opus format itself and the reference implementation are available under liberal royalty-free licenses, making it easy to adopt, compatible with free software, and suitable for usage as part of the basic infrastructure of the Internet. See the Opus comparison page for more details. http://wiki.xiph.org/OpusFAQ#How_does_Opus_compare_to_other_codecs.3F http://wiki.xiph.org/OpusFAQ#How_does_Opus_compare_to_other_... Which is to say, this isn't (okay, isn't just) a case of having "too many standards"; Opus is legitimately impressive from a technical standpoint. Oh, and it's unencumbered by patents. That's cool. EDIT: And here's the coolest part of the article that completely took me by surprise: "The IETF recently decided with “strong consensus” to adopt Opus as a mandatory-to-implement (MTI) codec for WebRTC, an upcoming standard for real-time communication on the web."
- wissler 14y ago
  There is no such thing as "unencumbered by patents". Unless you have a technology that has already been tested thoroughly in the courts, which is the only way you can really determine whether something would be deemed to violate a patent or not, you simply can't know.
  - azakai 14y ago
    Even if it was tested by the courts, you can appeal. And even after the appeal, you can appeal to a higher court, all the way to the Supreme. And even if was tested there, congress can pass a law that allows it. And even if that happens the Supreme court can strike down the law. And even if that happens, the public can mount a revolution and tear down the legal system. But, practically speaking, Opus has extremely good guarantees for being unencumbered, as much as is possible - the people working on it have given licenses to use it, and experts in the field have had a while to go over this stuff. Nothing is 100%, but this is as good as it gets.
AceJohnny2 14y ago
I'm not familiar with Opus. Can someone tell me if it is able to dynamically adapt quality based on available bandwidth? If it loses packets in the stream, do we get a 'skip' in the audio or does the quality go down?
- tqs 14y ago
  It is absolutely able to dynamically adapt based on bandwidth, and is robust to packet loss. It was designed for that :) http://www.opus-codec.org/ http://www.opus-codec.org/ https://wiki.xiph.org/OpusFAQ https://wiki.xiph.org/OpusFAQ
- tveita 14y ago
  > Can someone tell me if it is able to dynamically adapt quality based on available bandwidth? Depends on what you mean. It is a variable bitrate codec, and the encoder can easily adjust the bitrate of the stream. But you would need a way to discover the available bandwidth and communicate it to the encoder - Opus is just a codec, not a container or streaming protocol. > If it loses packets in the stream, do we get a 'skip' in the audio or does the quality go down? The standard doesn't mandate a specific way to deal with packet loss, but the reference implementation tries to handle this gracefully. [1] Some properties of the encoding facilitate this: for instance, packets can optionally include a reduced-bitrate encoding of the previous packet to provide redundancy. [2] [1] http://tools.ietf.org/html/rfc6716#section-2.1.8 http://tools.ietf.org/html/rfc6716#section-2.1.8 [2] http://tools.ietf.org/html/rfc6716#section-4.4 http://tools.ietf.org/html/rfc6716#section-4.4
- nullc 14y ago
  Yes, it's absolutely made for that, which is one of the reasons it has extreme bitrate and frame size agility. Though the decoder doesn't have a back channel to the encoder as part of the format (e.g. a duplex pair is not at all coupled by the codec and could have entirely different settings). So if you want to do rate adaptation the application will have to take care of telling the encoder that the decoder is struggling/losing frames/etc. The codec also includes a pretty good packet loss concealer integrated, and can also use forward error correction to reduce the impact of loss.
mtgx 14y ago
I expect Firefox and Chrome to adopt this within months, but IE10 is coming out soon. Does it mean it won't have it? And then we'll have to wait another year or more for IE11 to implement it? Not to mention IE9 and IE8 will probably never get it. I also assume WebM will replace Vorbis with Opus very soon?
- bicknergseng 14y ago
  While MS might not take full advantage of it, IE 10 will have silent updates. I would hope that would mean they drop the monolithic update strategy.
- deleted 14y ago
  [deleted]
- kibwen 14y ago
  The article mentions that Skype was a party to the standardization process, and hence Microsoft by association. So there's a small reason to hope.
- azakai 14y ago
  > I expect Firefox and Chrome to adopt this within months, Firefox already adopted it (it's in nightly builds). Chrome seems to have WontFixed it, http://code.google.com/p/chromium/issues/detail?id=104241 http://code.google.com/p/chromium/issues/detail?id=104241
  - nullc 14y ago
    It's actually in Firefox 15, not just nightly. People on Reddit are saying chrome's position has reversed but I don't have a link. Works in Opera already if you have gstreamer's opus plugin, and they've apparently said they'll ship it.
    - azakai 14y ago
      The chrome bug just changed from WontFix to Available.
mcantor 14y ago
I'm not an audio guy. Can anyone explain like I'm five why this new codec was needed? How does it affect me as a consumer of audio? Who out there was saying "Gosh, I want to do X, but it's really tough because existing codecs force me to choose between high quality and low delay"? What is an example of "X" in that sentence?
- azakai 14y ago
  There was no existing codec that is both good and not encumbered by patents. Opus is good and not encumbered by patents, and it has just been standardized.
- jmillikin 14y ago
  X = {voip, videoconferencing} Say you wanted to write a live audio chat app to talk to your friends. If you'd chosen the Speex codec, then voice would come over fine, but trying to show off your new guitar skills would fail because what they heard would sound horrible. If you'd chosen the Vorbis codec, then everything would sound great, but there'd be a weird "lag" between participants, as if one of them were on the moon. Also, adding too many listeners would severely strain your internet connection's capacity. Opus has latencies as good as Speex, and can be high-, medium-, or low-quality depending on bandwidth needs.
- nnethercote 14y ago
  Look at the graph in the article. Opus is as good as or slightly better than all the existing codecs at their strong points, and much better than them at their weak points.
YokoZar 14y ago
How does Opus compare to Celt, its direct predecessor?
- rdtsc 14y ago
  It actually includes code from Celt and merged with code from Silk (the Skype codec). Silk is very good at speech and Celt is very good at music. It is like having gears in your car. You need low gears for one mode of operation and high gears for another. So they combined the two into one and made a single codec. I imagine behind the scenes it has a transition mode from one codec to another depending on the configuration parameters.
- derf_ 14y ago
  The final version of libcelt is bit-for-bit compatible with Opus. Unlike SILK, there was never a "stable" version of CELT (we routinely broke the bitstream on almost every commit), so CELT development continued after the two codecs were merged, right up until the Opus bitstream was frozen. That makes Opus a strict superset of CELT, and we've stopped maintaining libcelt, so any existing users should switch to libopus.
mmphosis 14y ago
These comments answer some questions I have concerning FLAC... starwed wrote on September 11th, 2012 at 1:29 pm: Isn’t FLAC already* open and royalty-free?* Dusk wrote on September 11th, 2012 at 1:40 pm: FLAC is a lossless codec — it typically generates output at a bitrate orders of magnitude* higher than lossless codecs such as OPUS, and there’s no way to “dial it down”.* natermer wrote on September 11th, 2012 at 2:30 pm: You want to archive audio with 100% quality using Flac. Opus is for were you are willing to give up some amount of audio quality for the ability to massively increase compression ratio. Using low bit rate Opus may yield a 3MB file for a song, were as FLAC may end up being 150MB for the same audio. What I do for audio is to store it using flac, but encode it on the fly using other codecs for streaming for transfering to devices. That way I always get the best audio quality possible to go along with the best compression possible.
- masklinn 14y ago
  > Using low bit rate Opus may yield a 3MB file for a song, were as FLAC may end up being 150MB for the same audio. 3MB will get you POTS-quality audio though. Lossless codecs usually compress audio to half the original size (so a 150MB FLAC file — for music — will correspond to ~300MB of raw bitstream). Lossy compression to something worth listening to will usually be around 10x compression, so ~30MB for an original 300MB bitstream. A 35mn song compressed in 3MB probably won't be worth listening to.
  - ZoFreX 14y ago
    If you were using AAC (or anything else that's more advanced than MP3) a 3MB file would actually be listenable for a short song. A 150MB FLAC file would be about 20-30 minutes long (based on a typical bitrate of 700-1000 kilobits per second).
    - masklinn 14y ago
      > If you were using AAC (or anything else that's more advanced than MP3) a 3MB file would actually be listenable for a short song. Even mp3 works for "a short song", that's not the context of my comment though, its context is a 150MB FLAC file (~300MB bitstream) recompressed to 3MB with Opus. > A 150MB FLAC file would be about 20-30 minutes long A point I noted in my comment.
sandGorgon 14y ago
Relevant - Hydrogenaudio listening tests (530 tests) scored Opus higher than HE-AAC https://people.xiph.org/~greg/opus/ha2011/ https://people.xiph.org/~greg/opus/ha2011/ http://listening-tests.hydrogenaudio.org/igorc/results.html http://listening-tests.hydrogenaudio.org/igorc/results.html
- deleted 14y ago
  [deleted]

167 points by nullc 14y ago

nvmc 14y ago

MP3 is getting very long in the tooth, not to mention the fact that it's encumbered by patents. A new codec would be nice, but hardware adoption of the codec will take a while. Long enough, dare I say, to kill it dead in the water.

randallu 14y ago
Why would you say that? Most "hardware" decoders are DSPs that run software loaded onto them at runtime. This goes for anything with an OMAP, Tegra or Samsung (Samsung's own decode block) SoC in (that I know for sure) and probably almost everything else. Tegra3 and OMAP4 can decode WebM in hardware, for example.
- wmf 14y ago
  The problem is that they never develop firmware updates so if those DSPs don't support yet another open codec on release day, they never will.
  - gillianseed 14y ago
    perhaps, but even so that does not mean that it won't get supported in later generations. I'm sure people said the same thing about FLAC but it is now supported in lots of devices, like those running Android.
  - randallu 14y ago
    I'm pretty sure Tegra2 got firmware updates during its lifetime, and I know that TI do updates for OMAP. Those updates might not make it through to consumer devices, but that used to be the case for things like GPU drivers too. Google pushed an updated SGX driver for the Galaxy Nexus with 4.1, and Apple pushes updates to their own drivers every release.
- nvmc 14y ago
  I should have worded my comment better. I mean that adoption by hardware manufacturers will be slow. What's the incentive? Even for a firmware patch? MP3 is `good enough', and everyone already uses it.
  - iSnow 14y ago
    Opus is royalty-free, in contrast to mp3. And it will be used in WebRTC. I can see some devices picking up Opus to support WebRTC and possibly some (not portable music players, obviously) dropping mp3. Obviously neither you nor I have a crystal ball, but I see Opus as the PNG of audio codecs - not displacing the lousy but entrenched de-facto standard (GIF/mp3), but slowly taking over at the technological edge.
- pslam 14y ago
  Almost no hardware audio decoders use DSPs, or at least actually need to, since about a decade or more now. The manufacturers you mention use DSPs for bad reasons - usually the inertia of software already using a coprocessor and not being willing to license a new codec for the general purpose core. Audio decoding and encoding is an extremely low percentage of available CPU, and it's just not worth off-loading it. Video is another matter, at least when talking about HD. It's still very much worth using special purpose DSPs for that. For audio, the existence of hardware DSPs is simply not a barrier to entry.

kibwen 14y ago

Before someone chimes in with that XKCD strip: How does Opus compare to other codecs? Opus is distinguished from most formats for high quality audio (AAC, Vorbis, MP3) by having low delay and it is distinguished from most low delay formats (G.711, GSM, Speex) by supporting high audio quality. It meets or exceeds existing codecs' quality across a wide range of bitrates, and it operates at lower delay than virtually any existing compressed format. Further, the Opus format itself and the reference implementation are available under liberal royalty-free licenses, making it easy to adopt, compatible with free software, and suitable for usage as part of the basic infrastructure of the Internet. See the Opus comparison page for more details. http://wiki.xiph.org/OpusFAQ#How_does_Opus_compare_to_other_codecs.3F http://wiki.xiph.org/OpusFAQ#How_does_Opus_compare_to_other_... Which is to say, this isn't (okay, isn't just) a case of having "too many standards"; Opus is legitimately impressive from a technical standpoint. Oh, and it's unencumbered by patents. That's cool. EDIT: And here's the coolest part of the article that completely took me by surprise: "The IETF recently decided with “strong consensus” to adopt Opus as a mandatory-to-implement (MTI) codec for WebRTC, an upcoming standard for real-time communication on the web."

wissler 14y ago
There is no such thing as "unencumbered by patents". Unless you have a technology that has already been tested thoroughly in the courts, which is the only way you can really determine whether something would be deemed to violate a patent or not, you simply can't know.
- azakai 14y ago
  Even if it was tested by the courts, you can appeal. And even after the appeal, you can appeal to a higher court, all the way to the Supreme. And even if was tested there, congress can pass a law that allows it. And even if that happens the Supreme court can strike down the law. And even if that happens, the public can mount a revolution and tear down the legal system. But, practically speaking, Opus has extremely good guarantees for being unencumbered, as much as is possible - the people working on it have given licenses to use it, and experts in the field have had a while to go over this stuff. Nothing is 100%, but this is as good as it gets.

AceJohnny2 14y ago

I'm not familiar with Opus. Can someone tell me if it is able to dynamically adapt quality based on available bandwidth? If it loses packets in the stream, do we get a 'skip' in the audio or does the quality go down?

tqs 14y ago
It is absolutely able to dynamically adapt based on bandwidth, and is robust to packet loss. It was designed for that :) http://www.opus-codec.org/ http://www.opus-codec.org/ https://wiki.xiph.org/OpusFAQ https://wiki.xiph.org/OpusFAQ
tveita 14y ago
> Can someone tell me if it is able to dynamically adapt quality based on available bandwidth? Depends on what you mean. It is a variable bitrate codec, and the encoder can easily adjust the bitrate of the stream. But you would need a way to discover the available bandwidth and communicate it to the encoder - Opus is just a codec, not a container or streaming protocol. > If it loses packets in the stream, do we get a 'skip' in the audio or does the quality go down? The standard doesn't mandate a specific way to deal with packet loss, but the reference implementation tries to handle this gracefully. [1] Some properties of the encoding facilitate this: for instance, packets can optionally include a reduced-bitrate encoding of the previous packet to provide redundancy. [2] [1] http://tools.ietf.org/html/rfc6716#section-2.1.8 http://tools.ietf.org/html/rfc6716#section-2.1.8 [2] http://tools.ietf.org/html/rfc6716#section-4.4 http://tools.ietf.org/html/rfc6716#section-4.4
nullc 14y ago
Yes, it's absolutely made for that, which is one of the reasons it has extreme bitrate and frame size agility. Though the decoder doesn't have a back channel to the encoder as part of the format (e.g. a duplex pair is not at all coupled by the codec and could have entirely different settings). So if you want to do rate adaptation the application will have to take care of telling the encoder that the decoder is struggling/losing frames/etc. The codec also includes a pretty good packet loss concealer integrated, and can also use forward error correction to reduce the impact of loss.

mtgx 14y ago

I expect Firefox and Chrome to adopt this within months, but IE10 is coming out soon. Does it mean it won't have it? And then we'll have to wait another year or more for IE11 to implement it? Not to mention IE9 and IE8 will probably never get it. I also assume WebM will replace Vorbis with Opus very soon?

bicknergseng 14y ago
While MS might not take full advantage of it, IE 10 will have silent updates. I would hope that would mean they drop the monolithic update strategy.
deleted 14y ago
[deleted]
kibwen 14y ago
The article mentions that Skype was a party to the standardization process, and hence Microsoft by association. So there's a small reason to hope.
azakai 14y ago
> I expect Firefox and Chrome to adopt this within months, Firefox already adopted it (it's in nightly builds). Chrome seems to have WontFixed it, http://code.google.com/p/chromium/issues/detail?id=104241 http://code.google.com/p/chromium/issues/detail?id=104241
- nullc 14y ago
  It's actually in Firefox 15, not just nightly. People on Reddit are saying chrome's position has reversed but I don't have a link. Works in Opera already if you have gstreamer's opus plugin, and they've apparently said they'll ship it.
  - azakai 14y ago
    The chrome bug just changed from WontFix to Available.

mcantor 14y ago

I'm not an audio guy. Can anyone explain like I'm five why this new codec was needed? How does it affect me as a consumer of audio? Who out there was saying "Gosh, I want to do X, but it's really tough because existing codecs force me to choose between high quality and low delay"? What is an example of "X" in that sentence?

azakai 14y ago
There was no existing codec that is both good and not encumbered by patents. Opus is good and not encumbered by patents, and it has just been standardized.
jmillikin 14y ago
X = {voip, videoconferencing} Say you wanted to write a live audio chat app to talk to your friends. If you'd chosen the Speex codec, then voice would come over fine, but trying to show off your new guitar skills would fail because what they heard would sound horrible. If you'd chosen the Vorbis codec, then everything would sound great, but there'd be a weird "lag" between participants, as if one of them were on the moon. Also, adding too many listeners would severely strain your internet connection's capacity. Opus has latencies as good as Speex, and can be high-, medium-, or low-quality depending on bandwidth needs.
nnethercote 14y ago
Look at the graph in the article. Opus is as good as or slightly better than all the existing codecs at their strong points, and much better than them at their weak points.

YokoZar 14y ago

How does Opus compare to Celt, its direct predecessor?

rdtsc 14y ago
It actually includes code from Celt and merged with code from Silk (the Skype codec). Silk is very good at speech and Celt is very good at music. It is like having gears in your car. You need low gears for one mode of operation and high gears for another. So they combined the two into one and made a single codec. I imagine behind the scenes it has a transition mode from one codec to another depending on the configuration parameters.
derf_ 14y ago
The final version of libcelt is bit-for-bit compatible with Opus. Unlike SILK, there was never a "stable" version of CELT (we routinely broke the bitstream on almost every commit), so CELT development continued after the two codecs were merged, right up until the Opus bitstream was frozen. That makes Opus a strict superset of CELT, and we've stopped maintaining libcelt, so any existing users should switch to libopus.

mmphosis 14y ago

These comments answer some questions I have concerning FLAC... starwed wrote on September 11th, 2012 at 1:29 pm: Isn’t FLAC already* open and royalty-free?* Dusk wrote on September 11th, 2012 at 1:40 pm: FLAC is a lossless codec — it typically generates output at a bitrate orders of magnitude* higher than lossless codecs such as OPUS, and there’s no way to “dial it down”.* natermer wrote on September 11th, 2012 at 2:30 pm: You want to archive audio with 100% quality using Flac. Opus is for were you are willing to give up some amount of audio quality for the ability to massively increase compression ratio. Using low bit rate Opus may yield a 3MB file for a song, were as FLAC may end up being 150MB for the same audio. What I do for audio is to store it using flac, but encode it on the fly using other codecs for streaming for transfering to devices. That way I always get the best audio quality possible to go along with the best compression possible.

masklinn 14y ago
> Using low bit rate Opus may yield a 3MB file for a song, were as FLAC may end up being 150MB for the same audio. 3MB will get you POTS-quality audio though. Lossless codecs usually compress audio to half the original size (so a 150MB FLAC file — for music — will correspond to ~300MB of raw bitstream). Lossy compression to something worth listening to will usually be around 10x compression, so ~30MB for an original 300MB bitstream. A 35mn song compressed in 3MB probably won't be worth listening to.
- ZoFreX 14y ago
  If you were using AAC (or anything else that's more advanced than MP3) a 3MB file would actually be listenable for a short song. A 150MB FLAC file would be about 20-30 minutes long (based on a typical bitrate of 700-1000 kilobits per second).
  - masklinn 14y ago
    > If you were using AAC (or anything else that's more advanced than MP3) a 3MB file would actually be listenable for a short song. Even mp3 works for "a short song", that's not the context of my comment though, its context is a 150MB FLAC file (~300MB bitstream) recompressed to 3MB with Opus. > A 150MB FLAC file would be about 20-30 minutes long A point I noted in my comment.

sandGorgon 14y ago

Relevant - Hydrogenaudio listening tests (530 tests) scored Opus higher than HE-AAC https://people.xiph.org/~greg/opus/ha2011/ https://people.xiph.org/~greg/opus/ha2011/ http://listening-tests.hydrogenaudio.org/igorc/results.html http://listening-tests.hydrogenaudio.org/igorc/results.html

deleted 14y ago
[deleted]