36 ms·

Realtime audio DSP seems like about the last thing that in-browser JavaScript was designed to do well. Janky audio is unnacceptable in a way that janky visuals

Realtime audio DSP seems like about the last thing that in-browser JavaScript was designed to do well. Janky audio is unnacceptable in a way that janky visuals aren't. It totally kills the experience. Without very strong guarantees about GC latency and thread scheduling that are neither available nor on the horizon for in-browser js, it won't work for anything beyond fun hacks.

deleted 12y ago
[deleted]
chrislo 12y ago
Are you aware of the Web Audio API? It's an API designed especially to address those concerns, and achieves low latency and high performance where it is available (Chrome, Firefox and some versions of Safari currently). https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specifica...
- chrisdevereux 12y ago
  Yes, and it works by mostly avoiding realtime DSP in javascript. It provides a high-level API for wiring together prebuilt audio processing nodes. Much like using browser animation APIs vs programatic animation. My point is about the feasibility of JS signal processing, not the feasibility of using JS to glue together signal processors. The WebAudio API does provide a ScriptProcessorNode for JS processing, but it appears to suffer from the problems I described. The spec seems to warn against using this for realtime processing, fwiw.
  - notthetup 12y ago
    "The WebAudio API does provide a ScriptProcessorNode for JS processing, but it appears to suffer from the problems I described." Yup. That is being worked on by W3C and the WebAudio committee as we speak. WebWorker based JS processing is something being considered. https://github.com/WebAudio/web-audio-api/issues/113 https://github.com/WebAudio/web-audio-api/issues/113 http://lists.w3.org/Archives/Public/public-audio/2013OctDec/0073.html http://lists.w3.org/Archives/Public/public-audio/2013OctDec/... I am expecting this to be resolved in a few months. (hopefylly?) If the threading issues are resolved, then real time audio rendering doesn't take that much processing power and can be easily done in JS. Infact if there is nothing else contending for the EventLoop (like in this example of WavPot) the ScriptProcessorNode is able to meet the real-time guarantees pretty well, while still doing decent processing. Also going forward, with Technologies like SIMD.js and asm.js more can be done within a single onprocess callback.
    - chrisdevereux 12y ago
      Meeting real-time guarantees pretty well isn't good enough for a DAW. Not without a lot of buffering (and therefore latency), anyway. The WebWorker proposal is interesting. If the worker has its own heap and realtime scheduling with a correctly configured GC, it would be a big improvement. Getting the different implementations to work consistently enough will be challenging, though. Meeting realtime deadlines is difficult enough when working in C with a known OS and audio stack.
      - chrislo 12y ago
        We're working on a test suite for implementations, which will hopefully help with some of the basics, but it will be challenging to get full coverage. https://github.com/w3c/web-platform-tests/tree/master/webaudio https://github.com/w3c/web-platform-tests/tree/master/webaud...
CamperBob2 12y ago
If you don't care about latency, and in this "DSP sandbox" application you probably don't, JavaScript is fine for real-time DSP. JS under V8 and similar engines can be quite fast. You aren't going to digitize 100 MHz of RF spectrum and build an SDR in JavaScript, but for audio rendering work it'd be fine. Cool hack.
- chrisdevereux 12y ago
  I'm assuming that a full-fledged DAW cares a lot about latency. Live performance becomes difficult when latency creeps much above 5ms, for example. A lot of things that you normally don't think about (like when to allocate a buffer of memory) become critically important with low latency requirements. That isn't an issue for the demoed app, but it is for the hypothetical longer-term goal under discussion.
  - CamperBob2 12y ago
    Yeah, I think they're making a mistake by using the term DAW. That term already means something, and this isn't it.
- montag 12y ago
  "You probably don't care about latency" — famous last words...

deleted 12y ago

[deleted]

chrislo 12y ago

Are you aware of the Web Audio API? It's an API designed especially to address those concerns, and achieves low latency and high performance where it is available (Chrome, Firefox and some versions of Safari currently). https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specifica...

chrisdevereux 12y ago
Yes, and it works by mostly avoiding realtime DSP in javascript. It provides a high-level API for wiring together prebuilt audio processing nodes. Much like using browser animation APIs vs programatic animation. My point is about the feasibility of JS signal processing, not the feasibility of using JS to glue together signal processors. The WebAudio API does provide a ScriptProcessorNode for JS processing, but it appears to suffer from the problems I described. The spec seems to warn against using this for realtime processing, fwiw.
- notthetup 12y ago
  "The WebAudio API does provide a ScriptProcessorNode for JS processing, but it appears to suffer from the problems I described." Yup. That is being worked on by W3C and the WebAudio committee as we speak. WebWorker based JS processing is something being considered. https://github.com/WebAudio/web-audio-api/issues/113 https://github.com/WebAudio/web-audio-api/issues/113 http://lists.w3.org/Archives/Public/public-audio/2013OctDec/0073.html http://lists.w3.org/Archives/Public/public-audio/2013OctDec/... I am expecting this to be resolved in a few months. (hopefylly?) If the threading issues are resolved, then real time audio rendering doesn't take that much processing power and can be easily done in JS. Infact if there is nothing else contending for the EventLoop (like in this example of WavPot) the ScriptProcessorNode is able to meet the real-time guarantees pretty well, while still doing decent processing. Also going forward, with Technologies like SIMD.js and asm.js more can be done within a single onprocess callback.
  - chrisdevereux 12y ago
    Meeting real-time guarantees pretty well isn't good enough for a DAW. Not without a lot of buffering (and therefore latency), anyway. The WebWorker proposal is interesting. If the worker has its own heap and realtime scheduling with a correctly configured GC, it would be a big improvement. Getting the different implementations to work consistently enough will be challenging, though. Meeting realtime deadlines is difficult enough when working in C with a known OS and audio stack.
    - chrislo 12y ago
      We're working on a test suite for implementations, which will hopefully help with some of the basics, but it will be challenging to get full coverage. https://github.com/w3c/web-platform-tests/tree/master/webaudio https://github.com/w3c/web-platform-tests/tree/master/webaud...

CamperBob2 12y ago

If you don't care about latency, and in this "DSP sandbox" application you probably don't, JavaScript is fine for real-time DSP. JS under V8 and similar engines can be quite fast. You aren't going to digitize 100 MHz of RF spectrum and build an SDR in JavaScript, but for audio rendering work it'd be fine. Cool hack.

chrisdevereux 12y ago
I'm assuming that a full-fledged DAW cares a lot about latency. Live performance becomes difficult when latency creeps much above 5ms, for example. A lot of things that you normally don't think about (like when to allocate a buffer of memory) become critically important with low latency requirements. That isn't an issue for the demoed app, but it is for the hypothetical longer-term goal under discussion.
- CamperBob2 12y ago
  Yeah, I think they're making a mistake by using the term DAW. That term already means something, and this isn't it.
montag 12y ago
"You probably don't care about latency" — famous last words...