6 ms·
Show HN: Render audio waveforms to HTML canvas using WebGPU
Hey HN. I built this quick and dirty component to render audio waveforms using WebGPU. I just published it to NPM.
It's the first time I use WebGPU and it's been a while since I write shaders. Feedback is very welcome!
GitHub: https://github.com/mrkev/webgpu-waveform https://github.com/mrkev/webgpu-waveform
Examples: https://aykev.dev/webgpu-waveform https://aykev.dev/webgpu-waveform
- robtest123 3y ago[dead]
- serial_dev 3y agoFeedback: it would be great if on the example site interactive demo, the waveform offset could be increased by scrolling / swiping left and right, and the scale should also change by zooming in and out.
- aylmao 3y agoI think you're right. That's the way I use it in the app I built this for: https://twitter.com/aykev/status/1780000204122460484 https://twitter.com/aykev/status/1780000204122460484 It's more code, so I said "later". I also wanted the first example to be easy to read so people wouldn't get lost in the source. But since it's such a common use-case I think I'll add it too. Thanks for the feedback!
- spankalee 3y agoFor cross-framework usage it'd be great if instead of being a render callback, this was distributed as an HTML custom element. Then you could easily use it in any declarative template system, even React/JSX. I tried to make a web components here: https://lit.dev/playground/#gist=8cf935c3869bf4790653cd6fadfd2508 https://lit.dev/playground/#gist=8cf935c3869bf4790653cd6fadf... But the webgpu-waveform module isn't loading. Looks like you have a hard-coded import of "../../react@18.2.0/index.js", which isn't going to be a portable import even for the React users. For the non-React users, they shouldn't have to install React to get the module loading. Could you make two entrypoints, one for React, the other for plain JS?
- llamaLord 3y agoI mean, in a pinch you can just run a build to compile it down to a single JS bundle with zero external dependencies and then mount it into your app. Not ideal, but works.
- spankalee 3y agoBuild systems are not available everywhere, like in many online playgrounds. There's no reason not to have a browser-loadable package by default these days.
- aylmao 3y agoah, I made the dependency optional, since it all goes through an index.ts it still attemps to load React, true. I'll find a way around this, or maybe split the package into a react version and a vanilla version. Thanks for the feedback! I think you're right, a custom element would be nice to have
- jacksonhacker 3y ago[dead]
- pininja 3y agoVery cool! How do you like WebGPU compared to WebGL?
- aylmao 3y agoI prefer it! It's definitely more verbose, but as someone who doesn't do graphics usually, I find it fits my mental model a little better. I tried to build this using WebGL before WebGPU was publicly available, but WebGL is very tailored to certain workflows and I didn't make much progress. It really felt like I was using the wrong tool for the job. Maybe I'm just too much of a noob... though I did manage to build it with WebGPU after all haha.
- miniBill 3y agoThis is how I do it using WebGL: https://github.com/miniBill/audio-editor/blob/main/src/View/Waveform.elm#L92-L130 https://github.com/miniBill/audio-editor/blob/main/src/View/...
- lovegrenoble 3y agoNo appropriate GPUAdapter found
- knowaveragejoe 3y agoI get the same on an M2 macbook air. Wonder if it's platform-specific.
- dhosek 3y agoAlso on M3 MBP. I thought it might be a Safari limitation, but I try to avoid running Chrome as much as possible so I don’t know.
- aylmao 3y agoOn my M1 MBP, Safari 17.4.1, it straight up doesn't work. Can I Use does say Safari only support WebGPU on TP and behind a flag: https://caniuse.com/?search=webgpu https://caniuse.com/?search=webgpu Perhaps a Safari TP bug? I'd appreciate some browser version info so I can dig deeper.
- teki_one 3y agoWorks for me on Safari 17.4 (Sonoma 14.4.1, M2)
- aylmao 3y agoInteresting. I'm assuming Apple is doing some sort of A/B test then, and WebGPU must be enabled for some people but not others. huh.
- dhosek 3y agoIt’s behind a flag. Enable the Develop menu if you haven’t, go to Develop|Feature Flags and you’ll find the setting to enable WebGPU (which is listed as “testing”).
- PaulDavisThe1st 3y ago> WebGPU not supported in this browser. Pretty current firefox on Linux. Oh well.
- PaulDavisThe1st 3y agoAs a side note: Ardour doesn't use a GPU for waveform rendering and so along the way we discovered that the "find max + min values for this chunk of samples" is more or less the most expensive operation in the prepare-to-render step (so much so that we cache these values on disk). We got a notable performance improvement from using SIMD instruction sets for this computation.
- armadsen 3y agoI’ve seen the same in my own waveform rendering systems over the years. SIMD comes in handy for a lot of audio-related things, of course.
- emadda 3y agoI did something similar with regular DOM nodes and css flex here: https://bigwav.app https://bigwav.app Every vertical line is a dom element. I was considering moving to GPU rendering as the dom approach can take a long time to re render on changes.
- aylmao 3y agoThe resulting style looks really cool. This would definitely be doable with the GPU. A halfway-step could be to draw rects on canvas too— it should be faster than the DOM approach, but less complicated than writing a GPU shader.
- SarahC_ 3y agoNo appropriate GPUAdapter found. I've turned on in chrome://flags >WebGPU Developer Features Enables web applications to access WebGPU features intended only for use during development. – Mac, Windows, Linux, ChromeOS, Android, Fuchsia, Lacros #enable-webgpu-developer-features Chrome is up to date Version 123.0.6312.123 (Official Build) (64-bit) RTX2060
- Lockal 3y agoThis needs a screenshot in GitHub, otherwise all I see is "No appropriate GPUAdapter found". Was WebGPU actually justified?
- aylmao 3y agoThis and browser support info. Currently WebGPU is WIP in Safari and FF. https://caniuse.com/webgpu https://caniuse.com/webgpu If you're using Chrome and getting this error, do share more info. As for justification, it was justified. Rendering waveforms is O(n) on pixels, and each can be rendered independently. I'm sure one can get creative with caching images at different resolutions to efficiently implement zooming, but otherwise it's noticeably slow.
- Lockal 3y agoI used latest Chrome on Android smartphone. It was kind of expected for me, that WebGPU does not work there. As I understand, WebGPU actually requires compute-capable GPU. Meanwhile WebGL fallbacks to software rendering via SwiftShader.
- pjmlp 3y agoIt should work perfectly fine on Android 12 and later devices.
- webprofusion 3y agoCool, but the example wave form looks wrong? It doesn't appear to oscillate evenly as audio normally would.
- aylmao 3y agoIt's an audio file linked from the description of this video: https://www.youtube.com/watch?v=qeUAHHPt-LY https://www.youtube.com/watch?v=qeUAHHPt-LY Its a waveform shaped like a cow that sounds like a cow haha
- xipix 3y agoWhy is it that every audio waveform renderer looks super aliased? I also recently made a fast audio waveform renderer using the best bits of Wasm and WebGL to create an animatable waveform that looks a bit less 1990s: https://play.google.com/store/apps/details?id=com.parabolaresearch.bungeetimewarp https://play.google.com/store/apps/details?id=com.parabolare... Happy to open/share the code, just haven't had time. PS (yours didn't work for me in Safari)
- tgv 3y agoIn Safari, you can enable it under Develop > Feature flags.
- aylmao 3y agoOh this looks great, I'd love to have a look if you have time to share it. In my case, I'm just not very experienced with graphics. I'm sure with some tweaking it could look better, but I thought this was good enough for a v1. Re Safari: Unfortunately WebGPU support is still WIP. I'll add a notice on the site haha. https://caniuse.com/webgpu https://caniuse.com/webgpu
- xipix 3y agoThanks. I picked a fixed number of horizontal pixels per second for an internally cached image. I think it was 30 pixels/second so that scrolling at 1x playback rate would be smooth on 60/90/120Hz screens. So it can't zoom into the waveform like yours, but I don't need it to zoom. There are two parts: first the C++/wasm that analyses the audio and generates a bitmap that has width 30*duration and not very high. It effectively draws a line from each audio sample to the next, with alpha, but this can be optimised like crazy to a point where the time it takes is unnoticeable. The second part is the WebGL that the bitmap (aka texture) and renders to the canvas. In principle this could be rotated, spun in 3d, mapped to a sphere or any other crazy GPU manipulations. WebGL isn't too painful and works everywhere I tried including the Android webview in my Bungee Timewarp app.
- MadDemon 3y agoSince the number of pixels on the x-axis of the canvas is typically much smaller than the number of samples, you have to reduce the number of samples. This is very prone to aliasing if not done right, especially if you end up just drawing a line between the points. I found a good way to avoid aliasing by taking the min/max values for each chunk of samples and then filling the area in between rather than drawing lines between points. If you zoom in to a point where the window is only a few values, this will converge to the same result as just drawing lines between the samples. You can test it out by uploading audio files to our audio-to-midi web demo: https://samplab.com/audio-to-midi https://samplab.com/audio-to-midi
- chaosprint 3y agoNice! Great project website styling and demo. I had a WebGPU scope demo using vanila JS here, but it's not connected to AudioContext at all. https://stackblitz.com/edit/vitejs-vite-cuc9vs https://stackblitz.com/edit/vitejs-vite-cuc9vs Still, I had to use the old WebGL solution for https://glicol.org https://glicol.org since the WebGPU support seems to be pretty slow at the moment. Perhaps you can add some kind of detection and downgrade mechanism?
- chaosprint 3y agohttps://caniuse.com/?search=webgpu https://caniuse.com/?search=webgpu
- A-You 3y ago[dead]
- emursebrian 3y agoIt looks nice. If WebGPU is adopted by more browsers, it will probably be pretty useful for doing real-time visualizations of audio. In Emurse, we render waveform, and also pitch plots of audio recordings using canvas. It's pretty fast, but ours are a bit lower-resolution. The most resource-intensive part of the operation is actually processing the processing the audio.
- 8mobile 3y agoHi, please can you show an image on your github repo? Thank you