3 ms·

Going to be hard to do real-time audio analysis over postmessage

by hackcasual 5y ago

captbaritone 5y ago
Yeah, this. Iframes or webworkers would have had too much overhead to work inside an animation loop.
- rektide 5y ago
  The article talks about using WebAssembly.Global to avoid message passing overheads / delays. One could almost certainly use SharedArrayBuffer to do a very similar shared mutable memory overhead-lowering optimization with webworkers or iframes, and see the same outcome. I haven't looked in a while but SharedArrayBuffer used to be a core way to communicate with wasm, albeit it seemed super gnarly: if I remember you have to load your wasm program into a SharedArrayBuffer, then run that SharedArrayBuffer to do anything. I still have some hope that some day, some year, some decade, wasm workers might be able to have more than a single Memory at a time, such that we could work with SharedArrayBuffers far more elegantly/slickly, but please no one hold your breath. And of course, Safari doesn't support SharedArrayBuffer nor SharedWebWorker nor like 50% of the other awesome little platform capabilities that just seem so technically excellent. Makes me very very sad they make such great devices so technically maimed. To be fair, the spectre/meltdown global freak out ~3 years ago did cause a huge amount of slamming on the breaks & rapidly withdrawing really good standards. SharedArrayBuffer is only just making it back[1]. Would that we ever have a chance of having high resolution time again too, but that's basically never going to happen. [1] https://developer.chrome.com/blog/enabling-shared-array-buffer/ https://developer.chrome.com/blog/enabling-shared-array-buff...
fulafel 5y ago
The audio sample data is pretty low bandwidth, CD audio is ~150 kB/s. Latency is fine for real time too, but even if it wasn't, this is streaming media playback which tolerates relatively big delays in the pipeline as long as you keep a/v sync.

Going to be hard to do real-time audio analysis over postmessage

captbaritone 5y ago

Yeah, this. Iframes or webworkers would have had too much overhead to work inside an animation loop.

rektide 5y ago
The article talks about using WebAssembly.Global to avoid message passing overheads / delays. One could almost certainly use SharedArrayBuffer to do a very similar shared mutable memory overhead-lowering optimization with webworkers or iframes, and see the same outcome. I haven't looked in a while but SharedArrayBuffer used to be a core way to communicate with wasm, albeit it seemed super gnarly: if I remember you have to load your wasm program into a SharedArrayBuffer, then run that SharedArrayBuffer to do anything. I still have some hope that some day, some year, some decade, wasm workers might be able to have more than a single Memory at a time, such that we could work with SharedArrayBuffers far more elegantly/slickly, but please no one hold your breath. And of course, Safari doesn't support SharedArrayBuffer nor SharedWebWorker nor like 50% of the other awesome little platform capabilities that just seem so technically excellent. Makes me very very sad they make such great devices so technically maimed. To be fair, the spectre/meltdown global freak out ~3 years ago did cause a huge amount of slamming on the breaks & rapidly withdrawing really good standards. SharedArrayBuffer is only just making it back[1]. Would that we ever have a chance of having high resolution time again too, but that's basically never going to happen. [1] https://developer.chrome.com/blog/enabling-shared-array-buffer/ https://developer.chrome.com/blog/enabling-shared-array-buff...

fulafel 5y ago

The audio sample data is pretty low bandwidth, CD audio is ~150 kB/s. Latency is fine for real time too, but even if it wasn't, this is streaming media playback which tolerates relatively big delays in the pipeline as long as you keep a/v sync.