4 ms·

I'm currently trying to get realtime audio streaming over WebSockets to work. I'm encoding audio using libspeex on the server side (an iOS device) and decoding

I'm currently trying to get realtime audio streaming over WebSockets to work. I'm encoding audio using libspeex on the server side (an iOS device) and decoding it again on the client using speex.js[1]. Transmitting Speex "frames" (where each frame is a 20ms chunk of audio) over WebSockets and decoding in JS works beautifully. However, I have a really hard time to queue up these 20ms AudioBuffer nodes perfectly, without producing any glitches or pops. I'm not sure its even possible with the current API. What I would like to have is an AudioBuffer node where I can dynamically append chunks while its playing. Since all(?) source nodes are for one time only use, the browser could free the data that was already played again. An AudioBuffer that allows modification of the data while it's playing would work too - you could just loop it and use it as a ring buffer. However, my understanding of the spec is, that modifying an AudioBuffer's data is explicitly prohibited once its playing. [1] https://github.com/jpemartins/speex.js/ https://github.com/jpemartins/speex.js/

derf_ 12y ago
You might be interested in looking at what https://github.com/brion/ogv.js https://github.com/brion/ogv.js does. Just curious, if your goal is realtime, why aren't you using WebRTC?
- phoboslab 12y ago
  Thanks, I'll have a look! I can't use WebRTC yet because I want to have support for IE and especially Safari & Mobile Safari. I want to add audio for an iOS App[1] that's using jsmpeg[2] for video streaming. [1] http://instant-webcam.com/ http://instant-webcam.com/ [2] https://github.com/phoboslab/jsmpeg https://github.com/phoboslab/jsmpeg
  - derf_ 12y ago
    IE doesn't support Web Audio either. The project I linked above does contain a Flash shim for IE, which you may find helpful. Any proper realtime audio stack (which ogv.js is not: it targets moderate latency streaming) is going to require a jitter buffer to deal with network/processing latency variation. Writing one is a non-trivial exercise (though not impossibly so). In any case, I'd recommend using Opus (the audio codec used by WebRTC) over Speex.

derf_ 12y ago

You might be interested in looking at what https://github.com/brion/ogv.js https://github.com/brion/ogv.js does. Just curious, if your goal is realtime, why aren't you using WebRTC?

phoboslab 12y ago
Thanks, I'll have a look! I can't use WebRTC yet because I want to have support for IE and especially Safari & Mobile Safari. I want to add audio for an iOS App[1] that's using jsmpeg[2] for video streaming. [1] http://instant-webcam.com/ http://instant-webcam.com/ [2] https://github.com/phoboslab/jsmpeg https://github.com/phoboslab/jsmpeg
- derf_ 12y ago
  IE doesn't support Web Audio either. The project I linked above does contain a Flash shim for IE, which you may find helpful. Any proper realtime audio stack (which ogv.js is not: it targets moderate latency streaming) is going to require a jitter buffer to deal with network/processing latency variation. Writing one is a non-trivial exercise (though not impossibly so). In any case, I'd recommend using Opus (the audio codec used by WebRTC) over Speex.