4 ms·

I just need to say: I've spent some (the happiest part!) of my career (and ~2/3rd of my life, hobby wise) involved with sound/music technology, designing and w

by loxias 5y ago

I just need to say: I've spent some (the happiest part!) of my career (and ~2/3rd of my life, hobby wise) involved with sound/music technology, designing and writing audio (and some video) plugins (VST, AU, PD, and others), audio middle-ware, plugin host frameworks, low level DSP algorithms optimized for performance, authoring music/sound protocols (MIDI-over-BLE, and tons of unfinished network sync things...), translation layers for seamless porting of effects between APIs (ie: auto-generation of an iOS app, or embedded microcontroller code, from VST source code), algorithms/processes for "JITish" C/ASM code generation from a concise higher level mathematical definitions (think: something you'd type in a CAS), so you can edit the math at runtime, and see the result within seconds. 8 years ago I made a PoC of something similar to this, using dark C++ magic (policy based design, lots of TMP) to enable compile-time optimization of filter chains, where each filter is encapsulated at a source level then connected in a dataflow graph, to produce an optimized binary of the whole chain, suitable for running as an application or on bare metal. Apparently, much of what I was going for eons ago (with c++11) is possible now with c++20! I've just begun to skim the code, with an arrogant "This is in my domain of expertise!" hat on. So far, I really, really like what I see, an uncommon experience when I look at Other People's Audio Code. All sorts of little clues that point towards "good code smell". ("Oh good, they used <such and such> library..", "Hah! Correct usage of memory fences!", &c) I tip my hat to you, author! I am embarrassed I hadn't heard of this project before! As soon as I can afford to focus on anything non-paying, I look forward to contributing, competing, or both, with this project and author. :)

udbhavs 5y ago
How would you recommend someone wanting to get into creating audio plugins (with very basic c++ and close to no experience in DSP) start? Also another thing I was wondering, does the math used in audio processing also come in useful for other broader applications like video?
- fit2rule 5y ago
  Learn JUCE, watch some tutorials... http://juce.org http://juce.org
- loxias 5y ago
  > Also another thing I was wondering, does the math used in audio processing also come in useful for other broader applications like video? Dear lord yes. YES. They're all signals (audio, images, video, radio waves, EEG readings, &c). Just in different dimensions and timebases. The math used to tease meaning from the signals is pretty universal. Understanding fundamentals like convolutions, filters, frequency domain, bandwidth, sampling limits, &c are just as critical if you want to do video processing as audio. A Wiener Filter will clean up audio, as well as an image. To a very crude approximation, the math that lets us compress audio to MP3s is the same that lets us compress images to JPEGs. That first question though, is hard. I've been doing it for too long to have any useful advice.. (sorry) I would say, the best way to get into it is to... well... just start. Don't worry about the programming language, try things out in a high level tool, like MATLAB. With a foundation of simple diffeq and some stats, you'll be able to do stuff :)
  - udbhavs 5y ago
    Thanks, usually questions on the internet about learning VST programming are in the context of producers wanting finer control over their sound so the answer is always "it's not worth it, just use Max/PD/Reaktor". But as a CS student who's into making music I was curious about how these tools are made and also how much the theory overlaps with other fields (which by your answer it does to a large extent). So its exciting to know the knowledge will come in handy in other areas and that its not a futile time sink :)
    - gavinray 5y ago
      There is a massive online community (~10,000 people) of audio devs that will help you here: https://theaudioprogrammer.com/discord/ https://theaudioprogrammer.com/discord/ There are some devs from some well-known companies, as well as maintainers of a few popular frameworks there. They're (probably) going to tell you to use JUCE, use "Projucer" as a build tool, and C++ as a language. That's what most people do. I don't particularly love C++, and I prefer the CMake build, instead of their custom build tool, but potato-puhtahto. There's also a Rust Audio development community: https://github.com/RustAudio https://github.com/RustAudio https://discord.gg/b3hjnGw https://discord.gg/b3hjnGw And it may be a bit niche, but there is an (in my opinion) fantastic framework for writing audio plugins in D: https://github.com/AuburnSounds/Dplug https://github.com/AuburnSounds/Dplug It uses Physically-Based Rendering for the UI (PBR), which is generally used for 3D game stuff I think? It's a neat approach and the UI's it produces can look very realistic -- only framework that does this AFAIK. The author Guillaume is a nice person and will answer questions in the #audio-dev channel of the D-lang Discord that is linked in the Dplug repo.
jcelerier 5y ago
Had forgot to answer... thanks for your kind remarks :) > I look forward to contributing, competing, or both, with this project and author. :) yes ! please do :)

by loxias 5y ago

udbhavs 5y ago

How would you recommend someone wanting to get into creating audio plugins (with very basic c++ and close to no experience in DSP) start? Also another thing I was wondering, does the math used in audio processing also come in useful for other broader applications like video?

fit2rule 5y ago
Learn JUCE, watch some tutorials... http://juce.org http://juce.org
loxias 5y ago
> Also another thing I was wondering, does the math used in audio processing also come in useful for other broader applications like video? Dear lord yes. YES. They're all signals (audio, images, video, radio waves, EEG readings, &c). Just in different dimensions and timebases. The math used to tease meaning from the signals is pretty universal. Understanding fundamentals like convolutions, filters, frequency domain, bandwidth, sampling limits, &c are just as critical if you want to do video processing as audio. A Wiener Filter will clean up audio, as well as an image. To a very crude approximation, the math that lets us compress audio to MP3s is the same that lets us compress images to JPEGs. That first question though, is hard. I've been doing it for too long to have any useful advice.. (sorry) I would say, the best way to get into it is to... well... just start. Don't worry about the programming language, try things out in a high level tool, like MATLAB. With a foundation of simple diffeq and some stats, you'll be able to do stuff :)
- udbhavs 5y ago
  Thanks, usually questions on the internet about learning VST programming are in the context of producers wanting finer control over their sound so the answer is always "it's not worth it, just use Max/PD/Reaktor". But as a CS student who's into making music I was curious about how these tools are made and also how much the theory overlaps with other fields (which by your answer it does to a large extent). So its exciting to know the knowledge will come in handy in other areas and that its not a futile time sink :)
  - gavinray 5y ago
    There is a massive online community (~10,000 people) of audio devs that will help you here: https://theaudioprogrammer.com/discord/ https://theaudioprogrammer.com/discord/ There are some devs from some well-known companies, as well as maintainers of a few popular frameworks there. They're (probably) going to tell you to use JUCE, use "Projucer" as a build tool, and C++ as a language. That's what most people do. I don't particularly love C++, and I prefer the CMake build, instead of their custom build tool, but potato-puhtahto. There's also a Rust Audio development community: https://github.com/RustAudio https://github.com/RustAudio https://discord.gg/b3hjnGw https://discord.gg/b3hjnGw And it may be a bit niche, but there is an (in my opinion) fantastic framework for writing audio plugins in D: https://github.com/AuburnSounds/Dplug https://github.com/AuburnSounds/Dplug It uses Physically-Based Rendering for the UI (PBR), which is generally used for 3D game stuff I think? It's a neat approach and the UI's it produces can look very realistic -- only framework that does this AFAIK. The author Guillaume is a nice person and will answer questions in the #audio-dev channel of the D-lang Discord that is linked in the Dplug repo.

jcelerier 5y ago

Had forgot to answer... thanks for your kind remarks :) > I look forward to contributing, competing, or both, with this project and author. :) yes ! please do :)