4 ms·

I found an use for lossless streaming: A weakness in the open source landscape I found is that there's a lack of fast codecs. If you want to encode in real tim

I found an use for lossless streaming: A weakness in the open source landscape I found is that there's a lack of fast codecs. If you want to encode in real time (eg, streaming speech or an artist playing in the moment), and encode per client for positional audio in a 3D world, then Opus becomes a bottleneck. FLAC according to some preliminary tests helps by being faster than Opus to encode.

magicalhippo 5y ago
What are the use-cases for pre-baked positional audio? Ie where the client can't do the positional audio processing itself.
- dale_glass 5y ago
  Not pre-baked. I work on https://vircadia.com/ https://vircadia.com/ -- it has to give every user their own audio mix to account for their 3D position. This means codec costs add up fast. You can't just encode once and stream the same thing to a dozen people.
  - magicalhippo 5y ago
    Oh right, I read that wrong. Thought you did per-voice 3D positional encoding on server but mixing on client. Interesting use-case, got my brain racing off to find silly solutions.
boolemancer 5y ago
By "fast to encode" do you mean that in terms of low latency or low CPU usage? I ask because I know Opus is used quite commonly in real-time applications (e.g. voip), and I remember when researching in the past that it is actually capable of lower inherent latency because of support for smaller frame sizes than some of its competition. I haven't looked into how expensive it is to encode in terms of CPU time, so I assume maybe you're taking about a bottleneck in terms of the number of simultaneous streams you can support on a single CPU?
- dale_glass 5y ago
  Both, primarily the first. Low latency is definitely a requirement, but no issues with Opus on that account. > I haven't looked into how expensive it is to encode in terms of CPU time, so I assume maybe you're taking about a bottleneck in terms of the number of simultaneous streams you can support on a single CPU? Yup! I work on https://vircadia.com/ https://vircadia.com/ -- we have to compress audio in real time and every user gets their own mix since it depends on their location in 3D space. It turns out to add up pretty fast, and you can't fit that many people into a cheap VPS. That's why I'm working on FLAC support for it. If bandwidth is plentiful but CPU resources are lacking it's a good alternative to have. This code originally came from the High Fidelity company, which made their own codec. It's some piece of black magic that cuts down audio by exactly 1/4th and is amazingly fast. But it's closed source.
  - astrange 5y ago
    There's no way their proprietary codec is less black magic than Opus is. I've rarely if ever seen a closed source codec designed by people who really know what they're doing; they're usually just whatever the cowboy they hired thought would be cool. It's not like management is going to be able to tell if it's state of the art or not.
    - dale_glass 5y ago
      Well, I say "black magic" because at least to me it's a quite mysterious creation. But I'm no audio engineer. It's a lossy codec that's for some reason fixed rate, and shrinks audio by exactly 1/4th. The resulting quality is very good (works perfectly fine for music), and it's much faster than Opus. It retains stereo and high frequencies and sounds just fine. I'm sure there's a tradeoff somewhere, but it's certainly good enough to hold music events without people complaining. I think it might be possibly related to codecs like AptX. So far I've not found anything competitive that we could use -- stuff is either closed, or far more CPU intensive.
      - astrange 5y ago
namibj 5y ago
Both FLAC and Opus[0][1] have a dial to trade efficiency against CPU time. Have you even tried? I doubt FLAC remains a superior choice at low-complexity Opus settings. The library also has a complexity setting for the encoder. [0] https://opus-codec.org/docs/opus-tools/opusenc.html https://opus-codec.org/docs/opus-tools/opusenc.html [1] > Set encoding computational complexity (0–10, default: 10). Zero gives the fastest encodes but lower quality, while 10 gives the highest quality but slower encoding.
- dale_glass 5y ago
  I have tried it. It works, but the difference in my experience isn't a huge one. Complexity 0 is about twice as fast as 10. Which isn't bad, but if you want to go really fast it still leaves something to be desired.
  - namibj 5y ago
    I hope this has been measured with the binary optimized for the CPU architecture in question, in oder to sqeeze out reasonable gains from likely more portability-focused opus code, particularly regarding vectorization opportunities.

magicalhippo 5y ago

What are the use-cases for pre-baked positional audio? Ie where the client can't do the positional audio processing itself.

dale_glass 5y ago
Not pre-baked. I work on https://vircadia.com/ https://vircadia.com/ -- it has to give every user their own audio mix to account for their 3D position. This means codec costs add up fast. You can't just encode once and stream the same thing to a dozen people.
- magicalhippo 5y ago
  Oh right, I read that wrong. Thought you did per-voice 3D positional encoding on server but mixing on client. Interesting use-case, got my brain racing off to find silly solutions.

boolemancer 5y ago

By "fast to encode" do you mean that in terms of low latency or low CPU usage? I ask because I know Opus is used quite commonly in real-time applications (e.g. voip), and I remember when researching in the past that it is actually capable of lower inherent latency because of support for smaller frame sizes than some of its competition. I haven't looked into how expensive it is to encode in terms of CPU time, so I assume maybe you're taking about a bottleneck in terms of the number of simultaneous streams you can support on a single CPU?

dale_glass 5y ago
Both, primarily the first. Low latency is definitely a requirement, but no issues with Opus on that account. > I haven't looked into how expensive it is to encode in terms of CPU time, so I assume maybe you're taking about a bottleneck in terms of the number of simultaneous streams you can support on a single CPU? Yup! I work on https://vircadia.com/ https://vircadia.com/ -- we have to compress audio in real time and every user gets their own mix since it depends on their location in 3D space. It turns out to add up pretty fast, and you can't fit that many people into a cheap VPS. That's why I'm working on FLAC support for it. If bandwidth is plentiful but CPU resources are lacking it's a good alternative to have. This code originally came from the High Fidelity company, which made their own codec. It's some piece of black magic that cuts down audio by exactly 1/4th and is amazingly fast. But it's closed source.
- astrange 5y ago
  There's no way their proprietary codec is less black magic than Opus is. I've rarely if ever seen a closed source codec designed by people who really know what they're doing; they're usually just whatever the cowboy they hired thought would be cool. It's not like management is going to be able to tell if it's state of the art or not.
  - dale_glass 5y ago
    Well, I say "black magic" because at least to me it's a quite mysterious creation. But I'm no audio engineer. It's a lossy codec that's for some reason fixed rate, and shrinks audio by exactly 1/4th. The resulting quality is very good (works perfectly fine for music), and it's much faster than Opus. It retains stereo and high frequencies and sounds just fine. I'm sure there's a tradeoff somewhere, but it's certainly good enough to hold music events without people complaining. I think it might be possibly related to codecs like AptX. So far I've not found anything competitive that we could use -- stuff is either closed, or far more CPU intensive.
    - astrange 5y ago

namibj 5y ago

Both FLAC and Opus[0][1] have a dial to trade efficiency against CPU time. Have you even tried? I doubt FLAC remains a superior choice at low-complexity Opus settings. The library also has a complexity setting for the encoder. [0] https://opus-codec.org/docs/opus-tools/opusenc.html https://opus-codec.org/docs/opus-tools/opusenc.html [1] > Set encoding computational complexity (0–10, default: 10). Zero gives the fastest encodes but lower quality, while 10 gives the highest quality but slower encoding.

dale_glass 5y ago
I have tried it. It works, but the difference in my experience isn't a huge one. Complexity 0 is about twice as fast as 10. Which isn't bad, but if you want to go really fast it still leaves something to be desired.
- namibj 5y ago
  I hope this has been measured with the binary optimized for the CPU architecture in question, in oder to sqeeze out reasonable gains from likely more portability-focused opus code, particularly regarding vectorization opportunities.