12 ms·
Clap: The New Audio Plug-In Standard
- adamrezich 4y ago[flagged]
- oidar 4y agoI am hoping for a plugin standard that will take advantage of unused GPU cycles. Haven't see that yet.
- digitalslr 4y agoGPU Audio is doing it with VST 3. And it's probably possible with CLAP too? Processing audio on a GPU is a lower level issue than a plugin standard which describes the interface between the DAW and the plugin (how audio, parameters, notes, modulation data, etc. are exchanged).
- deleted 4y ago[deleted]
- duped 4y agoPro audio plugins have lower latency and harder constraints than graphics processing. You cannot drop a frame, and roundtrip latency is ideally below 5ms. GPU Audio (the company) has a lot of hype but it's not really clear if they gain a ton. The existing analogs to GPUs are expensive pieces of special purpose DSPs like Waves Grid, Universal Audio's UAD, and the biggest which are Avid PT HDX accelerators. It turns out modern CPUs are plenty fast for heavy audio DSP. The point of friction is synchronization and the memory wall, most of the time. It turns out that audio DSP algorithms don't generally benefit from the same kind of parallelism you get on a GPU, since even the basic stuff you'd like to do (filtering, algorithmic reverb, etc) is a bunch of recursive/sequential algorithms that have a non-trivial cost to convert to a parallel form for SIMD evaluation, and the gains are not linear due to repeated computation (classic example, converting an exponential moving average which is used everywhere to N lanes of SIMD is a lot less than an N times speed up). That's not to say there isn't exciting work being done (GPU audio, for example). It's just really hard and not a magic bullet. It's not an Audio Processing Unit, after all. It also has video output, but not high fidelity audio i/o.
- PaulDavisThe1st 4y agoWow. A low level, technical, detailed post about audio processing on HN that leaves me without feeling the need to correct any of it. Congratulations, and thanks!
- colinsane 4y agoGPUs have some use for physical modeling (e.g. convolutional reverb — which at least 5 years ago was not practical to do in realtime on a CPU). i toyed around with such a plugin a while back but i think it takes more expertise to program any reverb to sound obviously superior to the CPU-optimized algorithms we’re more practically familiar with. also additive synthesis greatly benefited from GPU acceleration at one point (particularly if you’re making full use of panning or detuning individual harmonics), but with SIMD/CPU improvements it’s much less relevant now. latency as a barrier to GPU use is sometimes overstated. you can reach sub-ms latency overhead on a GPU roundtrip. maybe relevant if you’re really targeting 5 ms through the whole pipeline, but not for amateur audio. of course, one issue is that these latencies stack if your host architecture uses something like VST/LV2 which only knows how to pass frames via the CPU. it’s one reason why i’d love for some declarative approach to plugins instead of shipping binaries which assume so much about their environment (even just shipping LLVM IR could let you batch GPU calls). but it seems that approach may forever be niche.
- duped 4y agoI haven't delved into it too deep. Some things are embarrassingly parallel, like additive synthesis. Some things can be made quite parallel like convolution. But a lot of stuff is not. The distinction I think that needs to be made between GPUs for graphics vs audio is that all GPU algorithms are built on the assumption that there exists a reasonable compute pipeline that every kind of processing can use or exploit. That is not true for audio processing (your compute kernels for convolution and IIR filtering are completely different, for example, even static IIRs and IIRs that undergo real time modulation have very different topologies). The thing I don't yet know about GPU Audio (the company) is how they deal with routing and process chains. Sub millisecond latency is fine, but if you have two dozen plugins in a chain (or god help you, a real DAG) then how do they handle the delay compensated I/O without adding extra latency for each sequential edge in the graph without the host being aware? It's not big things that hurt latency, it's more a death by a thousand cuts in practice.
- pier25 4y agohttps://www.gpu.audio/ https://www.gpu.audio/ They did this demo at NAMM recently: https://www.youtube.com/watch?v=IFdgymosszA https://www.youtube.com/watch?v=IFdgymosszA
- ushakov 4y agowhat’s the point of a new standard when no DAW besides Bitwig supports it?
- smoldesu 4y agoOld standards suck. VST is owned by Steinberg, and has caused all sorts of trouble in the quest for the "perfect" plugin format. Apple had a couple different solutions to this, but all of them were obviously proprietary to their ecosystem (and only a sidegrade to being shacked to Steinberg). It would be nice to see "one standard to rule them all" for Windows, MacOS and Linux, and u-he has enough weight behind their name to influence a change. There's a lot of hope behind this, and unless Apple decides to open-source their plugin format, there's nobody to compete. VSTs are so bad that most DAWs would adopt an alternative with zero additional features, simply because Steinberg's positioning is a pain for the community. I reckon this little format has a chance, but it remains to be seen whether it's successful or not.
- deleted 4y ago[deleted]
- Optimal_Persona 4y agoSince CLAP was announced last year, I've seen lot of online interest in CLAP from electronic musicians, plus independent audio developers who are sick of dealing with bureaucratic BS around Apple's AU Avid's AAX, and Steinberg's VST plugin standards. I would put Urs Heckman from u-he at the tip-top of not just audio DSP developers, but small business owners who truly care about advancing the state of the art. If u-he and Bitwig can get other Berlin-based audio shops on board this has a good chance of succeeding.
- happyhardcore 4y agoLooking at Ableton 11's feature set it seems they're running out of ideas; CLAP support would definitely be the kinda thing that could compel people to upgrade to a future version. It being MIT licensed also gives it a much better chance of getting integrated into all sorts of smaller DAWs and cheaper ones too that might not have wanted to deal with all the nonsense that comes with VST.
- toma_caliente 4y agoI'm not big in the audio community but I thought LV2 was the open standard for plugins that people are pushing for? That being said if Cockos adopts it into Reaper I'm absolutely willing to try it.
- collegeburner 4y agoLV2 has uhh a lot of warts. This talks about some: https://drobilla.net/2019/11/11/lv2-the-good-bad-and-ugly.html https://drobilla.net/2019/11/11/lv2-the-good-bad-and-ugly.ht... It also never got enough support from big guys. Clap looks like it fixes a lot od LV2 stuff as well, and it can replace VST, which is good because VST is a crap technology.
- lqr 4y agoWhat is so bad about VST?
- mkr-hn 4y agoApparently it's a nightmare to develop due to the quality of the SDK.
- PaulDavisThe1st 4y agoYeah, the API for VST3 is actually pretty good, but the SDK that Steinberg ships ... not so much.
- bexsella 4y agoI would go further than this, I'm interested in audio programming and decided to give it a shot using the Steinberg's SDK after reading the VST3 docs. The way the SDK is organized put me off VST3 entirely.
- Earw0rm 4y agoSort of. The breaking of MIDI is not, by any means, a good API. In a few other aspects it seems needlessly complex - feels like it was designed to serve one company's strategies and interests, not that of an industry.
- deleted 4y ago[deleted]
- jcpst 4y agoGood. This is a space that could use a new, open standard. With Bitwig behind it, I’m paying attention now.
- pixelatedindex 4y agoFor someone who has no background on audio plugins, but have a surface level understanding of DAWs, what would be a good starting point to learn?
- yots 4y agoWhile not perfect, I really liked Will Pirkle’s book[1]. It describes the popular ABIs and has great intro to DSP. I would say the downsides are that the description of various interfaces takes up too many pages and that the book uses his own framework ASPiK for cross-compatability, but it’s easy enough to convert to a more popular framework like JUCE if needed. [1] https://www.routledge.com/Designing-Audio-Effect-Plugins-in-C-For-AAX-AU-and-VST3-with-DSP-Theory/Pirkle/p/book/9781138591936 https://www.routledge.com/Designing-Audio-Effect-Plugins-in-...
- PaulDavisThe1st 4y agoWhat do you want to learn? How plugins do DSP? The basics of most plugin APIs? How plugin APIs differ from each other? How the host(s) see plugin APIs? What the difference is for user between the different APIs? How do people actually create audio plugins?
- jeanlucas 4y agoNot the original person that asked, but my questions are more elemental, google-level maybe (I'm googling it now): What even are audio plugins? How people use it? What are some examples?
- sebdufbeau 4y agoCaution: This explanation might not use 100% accurate terminology, but should get the point across. Most modern music is made via software named DAW (Digital Audio Workstation), which is kind of like an IDE but where music is made. These allow you to play with audio (ex: import audio files, arrange them across a timeline, record sound from external sources like a microphone). This is great if you already have all the audio files you need (eg: you're a live band and recorded a song in a studio), but sometimes you want to create new audio without having access to a recording studio. Audio plugins allow you to create new sounds. You've got two main types of plugins: ones that generate sound (eg: instruments, like a synth) and ones that modify sounds (takes an input sound, modifies it and then outputs it again, like an equalizer/EQ). Audio plugins are external (or bundled) software that you can use inside a DAW, you can download a lot of them for free but they can also get quite expensive (eg: thousands of dollars). The most popular format for these plugins for a long time (and still today) was VST, but Clap aims to be a new standard to compete with VST.
- glowcoil 4y agoI and some other folks in the Rust audio community have put together some low-level bindings for the CLAP API: https://github.com/glowcoil/clap-sys https://github.com/glowcoil/clap-sys They're relatively straightforward due to the fact that CLAP is a simple, pure-C ABI, and there are already some fully functional plugins making use of them (e.g. https://github.com/robbert-vdh/nih-plug https://github.com/robbert-vdh/nih-plug).
- PaulDavisThe1st 4y agoA more even-handed write up at librearts: https://librearts.org/2022/06/introducing-clap/ https://librearts.org/2022/06/introducing-clap/ and from the same author, just last week, an LWN overview of CLAP: https://lwn.net/Articles/893048/ https://lwn.net/Articles/893048/ and from the same author a few weeks before that, and overview of the state of audio plugin APIs with a particular focus on the Linux situation: https://lwn.net/Articles/890272/ https://lwn.net/Articles/890272/
- PaulDavisThe1st 4y agoHN Zeitgeist. Two different comments from this very comment section: > what’s the point of a new standard when no DAW besides Bitwig supports it? > Good. This is a space that could use a new, open standard. With Bitwig behind it, I’m paying attention now. Never believe in the idea of the HN "hive mind" :)
- nkozyra 4y agoI don't think those are incompatible comments. One commenter is just looking at it from the perspective of the market at large and the other uses a niche DAW, looking at its value for personal use. Even so, if it doesn't catch on more broadly there's a good chance it's not going to last. Hoping it makes it.
- cageface 4y agou-he is one of the most well known and well respected plugin vendors. Having them behind it gives it a lot of extra momentum. I think it's a fantastic and extremely welcome even if the name is somewhat unfortunate.
- nkozyra 4y agoI'm aware of them and their rep, but it isn't enough to move the whole industry and it's not like they're going to stop making VST/AU options.
- deleted 4y ago[deleted]
- Shared404 4y agoThe article states that there's contributions from other vendors as well, I suspect that this may actually catch. I've not really followed the industry, but my understanding is the controlling interests behind VST are unpleasant to work with for everyone involved - so this is a chance to build something better and in the open.
- stevehiehn 4y ago"Musicians everywhere are exposing themselves to the CLAP" .. Sorry, I'll see myself out
- somishere 4y agoHonestly, it is an unfortunate naming choice ... no doubt it's going to lead to all kinds of double-take usages in surrounding lit. I certainly read the title to the post twice.
- stereocodes 4y agoI can't wait to get the CLAP.... for my setup.
- mobiuscog 4y agoAre you saying you've never clapped in your lifetime, whether musically or not ?
- agrover 4y agoI'm a big Bitwig user. This makes me want to get u-he and other CLAP-supporting plugins.
- shams93 4y agoI'm about to starting writing some, the mpe support is exciting. Before this Juce have owned most of the plugin sector.
- coldtea 4y agoThe problem is: Logic has Apple's own AU, wont accept others. Cubase has Steinberg's own VST, wont accept others. And Live (which supports both AU and VST) has a "feud" with BitWig (which was started by ex-Ableton staff), so unlikely to accept Clap. So, what does this leaves us with? BitWig, maybe Studio One, and some even smaller players?
- cageface 4y agoI don't think Ableton is that petty. They'll back this if they think they stand to gain something from it. And given the mess that the other plugin standards are I think the value proposition is clear.
- KerrAvon 4y agoWhy would they want to support a third standard? More support headaches for little gain is not generally what software developers desire.
- ratww 4y agoActually 3 standards, AU, plus VST2 and VST3 which have different APIs. One of the advantages of CLAP would be Midi 2.0 support. But given that VST3 recently released support for Midi 2.0, that's a fair question.
- drcongo 4y agoPossibly 4 now, Live recently announced beta support for AUv3
- Earw0rm 4y agoBecause AU is non-existent on Windows, and Steinberg these days controls VST with only its own interests in mind, not those of the wider industry.
- batmansmk 4y agoIt's not just a new standard. CLAP plugins work better than their VST counterpart. They do more. Polyphonic CC modulation is a very fundamental upgrade Live could really have that would still fit its modular strip UI paradigm and justify even more its Push hardware. It would even reduce the gap with Bitwig. Project portability is also extremely desirable for Ableton, who is looking at mobile as a new platform. All the actors are Berlin-based companies located about 30min from each others, knowing each other very well. There is hope.
- eating555 4y agoHere is a more detailed technical post on CLAP, and the comparison between existed open standard LV2: https://cdm.link/2022/06/clap-is-a-new-open-source-plug-in-format-from-bitwig-u-he-do-we-need-it-and-who-will-use-it/ https://cdm.link/2022/06/clap-is-a-new-open-source-plug-in-f...
- justinclift 4y agoHopefully some people create a new "Atmos" style audio standard (eg multiple height surround sound), but OSS. It's a pita with the current situation, where everything past ~7.1 audio channels is proprietary and needs extremely expensive hardware (eg decoders) just to direct the audio to the correct speakers. :(
- PaulDavisThe1st 4y agoAmbisonics predates Atmos by many decades, is license free and has both gratis and libre toolchains available. It can handle any number of speakers, in any configuration.
- justinclift 4y agoThanks, that's the first time I've heard of it. :) Now, there "just" needs to be easily usable Atmos to Ambisonics conversion software so it can be widely adopted too.
- PaulDavisThe1st 4y agoAmbisonics will never be widely adopted. Wide adoption occurs when a significant corporation in consumer technology decides to push something, and in general that occurs when they are in a position to license it. Ambisonics has always been freely available, there is no possible license structure for it, ergo no consumer tech company will ever push it. There are also some important technological differences between Atmos (and similar schemes) and Ambisonics - the former does better with specific speaker configurations, the latter does better with (most) arbitrary ones.
- radiKal07 4y agoGreat move from BitWig to support this. Now if they could also hire a UI designer...
- markofthew 4y agoThis highlights a big problem in the world of music and audio software: everyone likes the idea of standards, just so long as they control the standard. And while this problem no doubt exists in other domains, it's painfully apparent in music and audio applications because the market is too small. Clap won't succeed longterm because of many of the factors outlined in so many of these posts. Notably, Apple aren't going to support it in Logic because Audio Units work just fine, and AUv3 spans both iDevice and Mac markets. Steinberg almost certainly won't support it; they aren't even allowing VST 2.4 plug-ins to run in the Apple Silicon version of Cubase -- and this isn't for technical reasons. Ironically, Steinberg have been trying to push VST 3 since the release of Cubase v4.1 back in 2006, and it's still a giant pain 16 years later. If you look back at VST 1/2, part of the allure was the simplicity of the API; VST 3 tried to change the game too much towards MVC given that it looked at the time you might want to run the processing part of the plug-in on a different system than the user interface part. Pro Tools doesn't get a mention in this thread, for the most part, but for better or worse it remains something of standard in audio post-production in the US. AAX isn't a bad re-imagining of what what RTAS/TDM-based plug-ins pre v10, and Avid aren't going to relinquish any control to third parties. Not least because the need for AAX (such as it is) resolves around support for the company's hardware control surfaces. EuCon was originally envisaged as an agnostic hardware controller protocol by Euphonix, but Avid purchased Euphonix, released the S6 and associated controllers and made EuControl into something of a tolerated illegitimate child. Plug-in developers will have to follow the money, and I would to see the results of a survey correlating money spent on plug-ins based on the main host software used. My guess is that Pro Tools AAX on the Mac probably is the most lucrative, followed by Cubase and Logic. Depressing, but for all the moaning musicians tend to indulge in regarding fair compensation against any company they feel exploited by, there's a reason why music and audio applications and plug-ins remain the most heavily copy-protected software. I don't know of anyone who's ever ditched their host application because of a plug-in format, and I highly doubt the CLAP is going to harm Apple, Steinberg, and Avid in the longterm.
- softfalcon 4y agoI feel like I know 10x as much about audio standards and the major players from this comment alone. This is literally the “it’ll takes me hours to Google it, but them 5 minutes to explain cause they’re an expert” in action. Clearly you know your stuff and the amount of information you conveyed in such a little time is honestly really impressive.
- jcelerier 4y agoFor anyone using c++, my declarative system has some amount of support for clap: https://github.com/celtera/avendish https://github.com/celtera/avendish / https://celtera.github.io/avendish/ https://celtera.github.io/avendish/ But unlike clap, targetting this also gives direct access to a few other environments, namely Max, Pd, ossia score, with the list hopefully growing. Here is an example minimal plugin : https://github.com/celtera/avendish/blob/main/examples/Raw/Minimal.hpp https://github.com/celtera/avendish/blob/main/examples/Raw/M... Note that unlike pretty much every other c/c++ plugin API, the plugin code does not need to include any header, everything is done through reflection of struct members at compile-time. This means that it's dead easy to port plugins to e.g. microcontrollers. Here's a per-sample noise generator which uses a small library of pre-made ports: https://github.com/celtera/avendish/blob/main/examples/Helpers/Noise.hpp https://github.com/celtera/avendish/blob/main/examples/Helpe... And a very naive buffer-based audio filter : https://github.com/celtera/avendish/blob/main/examples/Helpers/Lowpass.hpp https://github.com/celtera/avendish/blob/main/examples/Helpe... UI is supported without relying on a specific UI library, only on a canvas painter concept which can then target Qt, NanoVG, and others to come: https://github.com/celtera/avendish/blob/main/examples/Helpers/FFTDisplay.hpp https://github.com/celtera/avendish/blob/main/examples/Helpe... It does not only support audio but also has an "API-free" gpu abstraction since it's tailored for media arts in general: https://github.com/celtera/avendish/blob/main/examples/Gpu/DrawRaw.hpp https://github.com/celtera/avendish/blob/main/examples/Gpu/D... since it binds directly to audio APIs at compile time, it has pretty much zero code size in itself, the smallest plugin it generates for VST2 is around 7kb IIRC
- NonNefarious 4y agoI'll give them credit right off the bat for spelling "plug-in" right. It's not a "plugin." Doh, but then there's this on the page: "the transparent client-server model..." Nope: It's "client/server." But in non-nitpicking news: I hope this succeeds. OpenFX seems to be sticking around in the video realm (being supported in Resolve and Nuke and a couple of other applications), and I hope CLAP enjoys greater adoption. From this project's existence I presume that VST has some burdensome encumbrances.
- yellowapple 4y agoA "pure C ABI" doesn't really mean much, last I checked, seeing as how just about every combination of OS and CPU has its own quirks re: calling conventions and struct packing and such. Ain't sure what's meant here.
- atoav 4y agoI read it as "we actually thought about the ABI" which is always better than libraries where the ABI is an afterthought.
- ZoomZoomZoom 4y agoIs a MIT license a good fit for an open standard? What prevents any big nasty player adopting and sneakily subverting it with additions/changes in a truly EEE way? > As we proceed beyond the initial set of extensions, we are committed to establishing a transparent process to govern the standard which allows participation from the entire audio software community. We welcome participation from the development community today, and we will share details of these processes and governing models over the second half of 2022. Very vague and doesn't really instil confidence: "Join us now, and months later we'll share how we're going to protect your work."
- kookamamie 4y ago> CLever Audio Plug-in API A terrible name that'll just look stupid in a couple of years from now.
- switch007 4y agoEspecially as the clap is a common term for gonorrhoea where I live.
- jmtucu 4y agoSorry - https://xkcd.com/927/ https://xkcd.com/927/
- atoav 4y agoAs a professional freelance mixer I must say if the programming behind CLAP is anywhere close to the quality they achieved in Bitwig, I am sold on the idea. Bitwig does up to audio rate modulations of nearly every parameter, the stock plugins are all really top notch, the way they implemented voice stacking or polyphonic pitchbends for MPE synths etc. are all done in the way I dreamt it should have been done. It still lacks a bit on the hardcore audio editing and mixing front (e.g. in ProTools you can edit extremely fast, just using the keyboard). Their pricing and licensing model is fair. The price is actually quite okay given what you get in addition to just the DAW (a really good sample library with all the famous drum machines, a reaktor-like modular stock-plugin for synth, effects and midi manipulation, a powerful live-performance tool, etc.)
- deleted 4y ago[deleted]
- jay-anderson 4y agohttps://www.youtube.com/watch?v=H_03JJ7Ca5g https://www.youtube.com/watch?v=H_03JJ7Ca5g - This is a discussion on The Audio Programmer which give some examples and some motivations behind it.