5 ms·
Is there an open codec that concentrates on low CPU usage? I'm fine with it not being very bandwidth efficient. Opus is a very good codec, but it's not amazing
by dale_glass 4y ago
Is there an open codec that concentrates on low CPU usage? I'm fine with it not being very bandwidth efficient.
Opus is a very good codec, but it's not amazing CPU-wise. I work on a VR world, and audio encoding is usually our most limiting factor when running on an VPS. We have the capability to negotiate codecs, so the high cpu/low bandwidth use case is already covered.
What I'm looking for specifically:
* Low CPU usage
* Support for high bitrate, suitable for music and sounds other than voice
* Low latency
- CharlesW 4y agoI couldn't find data on CPU requirements for encode/decode versus, say, Opus, but Apple uses AAC-LD for similar scenarios.
- rasz 4y agoHow is Audio encoding the most limiting factor in a VR project? :o Afaik Opus encoder eats something like 30-50MHz of one cpu core.
- CodesInChaos 4y agoIt sounds plausible that it's the most expensive thing on the server side, if you have cheap simulation/behaviour and many concurrent users. But unless it's a non commercial project, the cost shouldn't be a big deal, so it's still a bit strange.
- dale_glass 4y agoWe work on a community-led fork of the dead commercial High Fidelity project. The server requirements are indeed very light except for audio. Physics are actually farmed out to the clients themselves, it's a bit of a quirky idea, but it actually works if one isn't concerned with accuracy.
- rasz 4y agoVentrilo/teampspeak servers run great on shared hosting. https://www.myteamspeak.com/addons/9ddfa0b2-25c2-4302-8a43-07f8819af9a9 https://www.myteamspeak.com/addons/9ddfa0b2-25c2-4302-8a43-0... gives you positional audio support on teamspeak server
- jimmySixDOF 4y agoApparently RP1 team handle bigger crowd loads through muxing on the server but not sure exactly how that works out for spatial audio there is a Kent Bye Voices of VR podcast discussing how they got 4k users in the same shard.
- muziq 4y agoI did a prototype of a 3D low-latency server side mixing system, based on a hypothetical 4k clients, @48k each being mixed with the 64 loudest clients.. Using Opus, forced to Celt mode only and running 256 stereo sample frames at 128kbps.. Worked well, using only 6 cores for that workload.. The mixing was trivial, but the decode and encode of 4k streams was entirely doable.. This issue at that rate was 1.5M network packets a second.. If I was to revisit it, I’d look at using a simple MDCT based codec, with a simple Psychacoustic model based on MPC (minus CVD) and modified for shorter frames + Mdct behaviour versus PQMF behaviour, without any Huffman coding or entropy coding.. And put that codec on the GPU.. Small tests I did using a 1080ti indicated ~1M clients could be decoded, mixed and encoded (same specs as above) problem is then how to handle ~370M network packets a second :) Edit: Had high hopes for High Fidelity, and came very close to asking for a job there ;) Shame it’s kaput, didn’t know that :(
- dale_glass 4y agoThose are interesting ideas, thanks! I'll have to try and play with that. High Fidelity the company is still around, but they pivoted multiple times radically. Initially their plan was social VR of sorts. Then they tried to make a corporate product for meetings and such, and gave up on that right before COVID19 hit! And after that they ripped out all the 3D and VR and scaled down to a 2D, overhead spatial audio web thing. Think something like Zoom, only you have an icon that you can move around to get closer or further to other people. The original code still lives on, we picked it up and are working on improvements. Feel free to visit out Discord (see my profile).
- bjt2n3904 4y agoI believe codec2 is pretty easy computationally. The M17 project uses it IIRC, and implements it on an STM32.
- CharlesW 4y agoThat'd be a good choice except for the requirement to support non-speech audio.
- douglasheriot 4y agoIt sounds like you’re asking for uncompressed audio? That meets all of your listed requirements. 48kHz * 16bit, single channel = 768kbit/s
- userbinator 4y agoEither that or G.711.
- viraptor 4y agoG711 is neither high bitrate nor usable for music.
- simfree 4y agoThen use G.722, it works fine for music.
- viraptor 4y agoNo, g722 is still a wideband speech codec. Its available frequency goes up to 7 kHz. The uncompressed audio this thread began with goes up to 22 kHz. With g722 you're losing most overtones, or even all overtones from the top of a piano. Please don't use g722 for music apart from on-hold muzak.
- dale_glass 4y agoWe support that already, yup. But it never hurts to see if there's something better than that out there.
- zinekeller 4y agoWhile I also want a low-computation codec that can save space, the historical use cases unfortunately assumes a lot more CPU power to be compensated for a lot less bandwidth, so there's little research in this area, and there's no real incentive to make something like ProRes and DNxHD as if you are editing audio the SSD speeds has been so fast that you'll run into CPU problems first.
- aidenn0 4y agoVorbis might be a good choice there
- ksec 4y agoLC3Plus or AAC-LD. Although they likely don’t fit the definition of Open Codec.
- izacus 4y agoThe Bluetooth codecs are all designed to be very cheap on CPU and low latency - e.g. LC3, AptX or SBC.
- themerone 4y agoI'm skeptical, those are almost always going to be implemented in hardware, so the complexity of a software encoder isn't a design concern. There is some correlation between the cost of a hardware implementation and complexity of a software implementation. SBC is a very simple codec, but AptX and LC3 might not be much better than Opus.
- jchw 4y agoIf you are OK with moderately high bitrates, you might prefer something simpler like an ADPCM scheme. It's pretty damn easy to implement ADPCM, certainly a lot less math heavy than MDCT-based schemes, and they achieve good quality at a somewhat higher bitrate (I have no data, but I'd guess 200-250%~ish.)
- cma 4y agoWhy is your VPS server encoding rather than clients? Are you combining talkers together into one source for doing crowds and avoiding N^2 or something and need to reencode after combining?
- fanick 4y agoI would like to know the answer to this question from dale_glass too.
- dale_glass 4y agoReplied to the parent
- dale_glass 4y agoCorrect, server does spatial audio. It's a community-led continuation of High Fidelity, a dead commercial project. They made their own proprietary codec with excellent performance we can't use and managed to have a couple thousand people in the same server.