5 ms·

Why is it that every audio waveform renderer looks super aliased? I also recently made a fast audio waveform renderer using the best bits of Wasm and WebGL to c

by xipix 2y ago

Why is it that every audio waveform renderer looks super aliased? I also recently made a fast audio waveform renderer using the best bits of Wasm and WebGL to create an animatable waveform that looks a bit less 1990s: https://play.google.com/store/apps/details?id=com.parabolaresearch.bungeetimewarp https://play.google.com/store/apps/details?id=com.parabolare... Happy to open/share the code, just haven't had time. PS (yours didn't work for me in Safari)

tgv 2y ago
In Safari, you can enable it under Develop > Feature flags.
aylmao 2y ago
Oh this looks great, I'd love to have a look if you have time to share it. In my case, I'm just not very experienced with graphics. I'm sure with some tweaking it could look better, but I thought this was good enough for a v1. Re Safari: Unfortunately WebGPU support is still WIP. I'll add a notice on the site haha. https://caniuse.com/webgpu https://caniuse.com/webgpu
- xipix 2y ago
  Thanks. I picked a fixed number of horizontal pixels per second for an internally cached image. I think it was 30 pixels/second so that scrolling at 1x playback rate would be smooth on 60/90/120Hz screens. So it can't zoom into the waveform like yours, but I don't need it to zoom. There are two parts: first the C++/wasm that analyses the audio and generates a bitmap that has width 30*duration and not very high. It effectively draws a line from each audio sample to the next, with alpha, but this can be optimised like crazy to a point where the time it takes is unnoticeable. The second part is the WebGL that the bitmap (aka texture) and renders to the canvas. In principle this could be rotated, spun in 3d, mapped to a sphere or any other crazy GPU manipulations. WebGL isn't too painful and works everywhere I tried including the Android webview in my Bungee Timewarp app.
MadDemon 2y ago
Since the number of pixels on the x-axis of the canvas is typically much smaller than the number of samples, you have to reduce the number of samples. This is very prone to aliasing if not done right, especially if you end up just drawing a line between the points. I found a good way to avoid aliasing by taking the min/max values for each chunk of samples and then filling the area in between rather than drawing lines between points. If you zoom in to a point where the window is only a few values, this will converge to the same result as just drawing lines between the samples. You can test it out by uploading audio files to our audio-to-midi web demo: https://samplab.com/audio-to-midi https://samplab.com/audio-to-midi
- xipix 2y ago
  My point is that everyone seems to draw waveforms using only two colours. Inevitably, this results in aliasing. In the extreme we have the "Spotify" visualisations of vertical bars with gaps in between. I believe this is popular because it looks slightly better than a solid waveform lump with an aliased edge. To avoid aliasing you need to use more than two pixel colours.
  - breakfastduck 2y ago
    Its much easier to read the information with the aliasing
  - chrisjj 2y ago
    > To avoid aliasing you need to use more than two pixel colours. The aliasing in question is of the audio not the pixels, so no more colours does not help.
    - xipix 2y ago
      Agreed, there is aliasing of the audio. But also aliasing in the way that the waveform is rendered. Consider: would you draw a line, or a circle using only a foreground and a background pixel colour?
      - 2y ago
animuchan 2y ago
This is beautiful! I'd love to learn from the code.
emursebrian 2y ago
WebGPU only works in Chromium based browsers.

by xipix 2y ago

tgv 2y ago

In Safari, you can enable it under Develop > Feature flags.

aylmao 2y ago

Oh this looks great, I'd love to have a look if you have time to share it. In my case, I'm just not very experienced with graphics. I'm sure with some tweaking it could look better, but I thought this was good enough for a v1. Re Safari: Unfortunately WebGPU support is still WIP. I'll add a notice on the site haha. https://caniuse.com/webgpu https://caniuse.com/webgpu

xipix 2y ago
Thanks. I picked a fixed number of horizontal pixels per second for an internally cached image. I think it was 30 pixels/second so that scrolling at 1x playback rate would be smooth on 60/90/120Hz screens. So it can't zoom into the waveform like yours, but I don't need it to zoom. There are two parts: first the C++/wasm that analyses the audio and generates a bitmap that has width 30*duration and not very high. It effectively draws a line from each audio sample to the next, with alpha, but this can be optimised like crazy to a point where the time it takes is unnoticeable. The second part is the WebGL that the bitmap (aka texture) and renders to the canvas. In principle this could be rotated, spun in 3d, mapped to a sphere or any other crazy GPU manipulations. WebGL isn't too painful and works everywhere I tried including the Android webview in my Bungee Timewarp app.

MadDemon 2y ago

Since the number of pixels on the x-axis of the canvas is typically much smaller than the number of samples, you have to reduce the number of samples. This is very prone to aliasing if not done right, especially if you end up just drawing a line between the points. I found a good way to avoid aliasing by taking the min/max values for each chunk of samples and then filling the area in between rather than drawing lines between points. If you zoom in to a point where the window is only a few values, this will converge to the same result as just drawing lines between the samples. You can test it out by uploading audio files to our audio-to-midi web demo: https://samplab.com/audio-to-midi https://samplab.com/audio-to-midi

xipix 2y ago
My point is that everyone seems to draw waveforms using only two colours. Inevitably, this results in aliasing. In the extreme we have the "Spotify" visualisations of vertical bars with gaps in between. I believe this is popular because it looks slightly better than a solid waveform lump with an aliased edge. To avoid aliasing you need to use more than two pixel colours.
- breakfastduck 2y ago
  Its much easier to read the information with the aliasing
- chrisjj 2y ago
  > To avoid aliasing you need to use more than two pixel colours. The aliasing in question is of the audio not the pixels, so no more colours does not help.
  - xipix 2y ago
    Agreed, there is aliasing of the audio. But also aliasing in the way that the waveform is rendered. Consider: would you draw a line, or a circle using only a foreground and a background pixel colour?
    - 2y ago

animuchan 2y ago

This is beautiful! I'd love to learn from the code.

emursebrian 2y ago

WebGPU only works in Chromium based browsers.