5 ms·
C++ audio mixing library design
- juunpp 4y ago> The stream interface is the core of the library, so let’s discuss it first. It is remarkably simple It's interesting (odd?) that the interface doesn't just expose a function to get the next value, similar to a lazy list in Haskell, which also goes well with the original statement in the post that a stream is basically a function. The 'sine_wave' stream for example is 90% filling the array and 10% actual stream logic, which I imagine you're duplicating in other stream implementations.
- lisyarus 4y agoThat's for efficiency reasons: most streams have way more logic than just filling the array, which can hardly be represented in terms of "getting the next value", and loops are good for optimization by compiler, etc. In fact, the actual sine wave implementation looks exactly the way you describe: it handles a generating function to a helper class that just fills the array https://bitbucket.org/lisyarus/psemek/src/master/libs/audio/source/wave/sine.cpp https://bitbucket.org/lisyarus/psemek/src/master/libs/audio/...
- pjmlp 4y agoVery nice article, however in 2022, why using C arrays as parameters in C++ code, when std::span, std::vector exist?
- throwaway17_17 4y agoSDL takes array pointers as arguments for its audio output functions in general, so I would assume they were coding against the SDL Audio API.
- pjmlp 4y agoNo reason to use them on C++ method definitions. void engine::callback(std::span<float> output) { std::size_t samples = 0; if (stream_) { samples = stream_->read(output.data(), output.size()); if (samples == 0) stream_ = nullptr; } auto remaining = output.subspan(samples); std::fill(remaining.begin(), remaining.end(), 0.f); }
- lisyarus 4y agostd::vector would definitely be the wrong choice, since I typically want to pass random subsections of random arrays to be filled by audio samples. std::span would be a good choice, though (I actually have my own span in the engine, from before C++20). Why didn't I use it? No idea, just didn't come to my mind. Guess it might be the influence of the book I was reading, or of SDL_Mixer interface, etc. I may do some refactoring in the future and use spans instead.
- pjmlp 4y agoThanks for the feeback. I only raised the issue, because this is how new programmers keep doing mistakes in C++ when learning the code from others, specially projects that they find are cool to learn from. As you seem to use best practices in the rest of the code that seemed strange to me.
- planede 4y agostd::span is new enough to not be in muscle memory of C++ programmers. Even the standard library commits a similar sin with `from_chars` and not taking a `string_view`.
- gpderetta 4y agostd::span is new, but I have seen some custom form of it in pretty much every codebase I've worked with for a very long time (and implemented some myself). Like it used be the case for std::string in older codebases.
- pencilguin 4y agostring_view would definitely have been wrong. std::span did not exist in the Standard at the time std::from_chars was adopted. An overload that takes std::span would be easy, and harmless. But as std::from_chars is only ever used from within some other abstraction, any benefit would be minimal.
- planede 4y agoWhy would string_view be wrong there?
- pencilguin 4y agoBecause, then, to parse out a number you would first need to get it into a string.
- urban_winter 4y agoI wonder if, in retrospect, the decision to use floats was a good one. The author mentioned issues with the time of each sample for the sine wave, which were float-related. I get that audio effects (compresser, reverb etc) are probably easier using floats but I don't immediately see why it's better to have float as the core data structure and convert to int at the end rather than having int as the core and convert to/from float only when needed.
- dspig 4y agoYes it's a good decision, and 'the standard' for audio in the same way as 32-bit ARGB makes everything easier for graphics. Of course there are some things that are better done with ints or maybe doubles, like keeping track of the time position, but for the sine wave generator it doesn't need to know how long it's been running in total, just the position in the current cycle, so floats are fine for that.
- lisyarus 4y agoThe alternative that my older code used was int16, and it has way bigger precision issues (32-bit float has 23 bits of precision, and in16 has 15). Fundamentally audio samples are just numbers; the fact that they have to be in the [-1..1] range in the end is a hardware technical limitation (the speakers' membranes can only vibrate so much), but there's nothing in the audio itself that says it should be constrained to some range. So, floating-points are a nice fit. Fixed-point math like 16.16 might be a good alternative, but floats have hardware support, so here we go.
- PennRobotics 4y agoThe smallest nitpick: The accepted variable for frequency is f or nu (ν). The formula and units are correct in the article, but lowercase omega is understood as angular frequency and would take the place of 2πf. I also got 3 hours of sleep, so I could be wrong. Also, thanks for the great article and a possible alternative to my own SDL_mixer woes!
- lisyarus 4y ago
- chaosprint 4y ago"All my engine is in C++, and I’m really sick of SDL_DoThatThing(&options, pointer_to_void, size_in_magical_units, &callback, user_data, legacy_value) (sorry for all C admirers, I do respect you, it’s just that I’m not one of you)." How about Rust audio (https://github.com/RustAudio https://github.com/RustAudio) and then FFI? I am developing the audio lib for the music live coding language I design in Rust and I think the experience of Rust is really good, because of `cargo` (the package management), the community, the syntax and compiler, the ergonomics, and so on. For one of my usages, the audio lib in Rust compiles to wasm and runs in browsers smoothly. I can also write VST plugins with the same audio library in Rust: https://github.com/chaosprint/glicol https://github.com/chaosprint/glicol There are also many other Rust audio libs such as dasp, fundsp and hexosynth, all worth checking out. Before I became addicted to Rust audio, I had checked some C++ audio project, such as the Maximillian lib especially the JS bindings, the source code of SuperCollider and the JUCE for VST. But apparently I have made my decision for the reasons abovementioned.
- lisyarus 4y agoAdding a whole new language (meaning compiler, packaging, libs, ecosystem) into my engine just for the audio lib which I _wanted_ to implement myself in the first place? No, thanks.
- secondcoming 4y agoYou insensitive clod!
- UncleEntity 4y agoI think you’re meant to rewrite your whole game engine in rust to make it easier.
- Narew 4y agoDoesn't you will have the same issue, a C-like interface. Maybe I missed something, but Rust and ffi mean you use a C interface from C++ ? The only C++<->Rust I know is either using C intermediate code or using cxx.rs but the support is limited ?
- Jasper_ 4y agoI love audio programming, it's one of those things that's provided me with a lot of joy. I've been working on our in-house audio library for our game engine as well, and designing custom effects for it. In our case, we wanted an 80s DSP sound, so it was fun diving into the old literature. In the end, a simple design won out: https://twitter.com/JasperRLZ/status/1567546255454371842 https://twitter.com/JasperRLZ/status/1567546255454371842
- throwaway17_17 4y agoDo you have any references for the old literature you looked at while trying to capture that sound. I would love to add some of it to my reading list. Also, the processing and track itself in your video are fantastic.
- Jasper_ 4y agoMy code has these links as comments for reading material: https://www.soundonsound.com/techniques/springs-plates-buckets-physical-modelling https://www.soundonsound.com/techniques/springs-plates-bucke... https://www.dsprelated.com/freebooks/pasp/Schroeder_Reverberators.html https://www.dsprelated.com/freebooks/pasp/Schroeder_Reverber... http://www.music.mcgill.ca/~gary/courses/papers/Moorer-Reverb-CMJ-1979.pdf http://www.music.mcgill.ca/~gary/courses/papers/Moorer-Rever... https://www.dsprelated.com/freebooks/pasp/Freeverb_Main_Loop.html https://www.dsprelated.com/freebooks/pasp/Freeverb_Main_Loop... Most reverbs these days are convolutional. I looked at those too! But they require an expensive FFT (usually run in a thread), and the quality sounded "too good" for the audio target I wanted to hit. After trying a lot of stuff, my main primitive ended up being a delay line hooked up to a low-pass. The output sample is a delay line going through a low-pass. That output is mixed with the incoming sample, and shoved back into the delay. Effectively, this makes a comb filter. I run eight of these in parallel at slightly different delay lengths to comb different frequencies. I was trying to shoot for "digital blur" that had a bit of crunch and aliasing that I wanted, while still sounding like a reverb. And the track was done by Jake Kaufmann, our composer at Yacht Club Games! I can't take any credit for it.
- emsy 4y ago
- stodor89 4y ago> All my engine is in C++, and I’m really sick of SDL_DoThatThing(&options, pointer_to_void, size_in_magical_units, &callback, user_data, legacy_value) (sorry for all C admirers, I do respect you, it’s just that I’m not one of you). I started as a C++ programmer (back when C++0x and boost were all the rage), but the more time passes, the more of a C admirer I seem to become.
- stoeckley 4y agoSame. I've written probably more C++ than any other language, and I have a lot of respect for the language overall, but I recently had to use only C for something, and I found it rather refreshing. My use case didn't hit a lot of the pain points of C, but it did feel refreshing as there are fewer ways of doing things, fewer options, compared to C++, and my code ended up feeling more straightforward and readable as a result.
- pjmlp 4y agoC23 says hello. C is becoming into everything C++ has, minus OOP and the additional type safety.
- jstimpfle 4y agoExamples? Because the list of accepted proposals seems conservative. https://en.wikipedia.org/wiki/C2x https://en.wikipedia.org/wiki/C2x https://thephd.dev/c23-is-coming-here-is-what-is-on-the-menu https://thephd.dev/c23-is-coming-here-is-what-is-on-the-menu
- pjmlp 4y agoYou have to look forward to what in the plate post C23 like lambdas, improved constexpr, more _Generic support, and so forth. https://www.open-std.org/jtc1/sc22/wg14/www/wg14_document_log.htm https://www.open-std.org/jtc1/sc22/wg14/www/wg14_document_lo...
- 4y ago
- qalmakka 4y ago- A stream is something you can play as a sound. It may be loaded from a file or generated on the fly. It may be finite or infinite. One important thing is that a stream is single-shot: it doesn’t have a restart method or anything like that. - A channel is something where a stream is actually played. You can request the channel to stop, or you can replace the stream this channel is playing. Am I mistaken, or isn't this very close to what GStreamer already does with sinks and sources?
- lisyarus 4y agoIt pretty much may be!
- tonyarkles 4y agoYeah, very similar model for sure. I say this having used GStreamer in anger though: the complexity of Glib and GObject makes OP’s API quite a bit nicer to use. There are tradeoffs though; while GStreamer is complex and error-prone, it does have two big pieces that OP’s approach doesn’t: - metadata in the streams. GStreamer streams have rich data attached to them that allows both sides of the chain to automatically negotiate the specific format and properties of the data in the stream. - introspection. Given an opaque/genetic GObject pointer to an element in the processing graph, you can query its configuration and sources/sinks without knowing what the object is up front. This ultimately can allow you to walk the entire graph if you want and inspect all of the elements.
- qalmakka 4y ago> the complexity of Glib and GObject makes OP’s API quite a bit nicer to use Yeah, Glib and GObject are a massive PITA, thankfully both have C++ bindings and can be used from Vala, so it's not that bad in practice. The C API is a nightmare though.
- lisyarus 4y agoTrue! But also my use case it quite different compared to GStreamer, and I don't really need all that metadata & introspection.