5 ms·

Web Audio API comes to Firefox

118 points by rnyman 13y ago

comex 13y ago
Excellent. Although Firefox previously had a simple API that could be used to do most of these things manually in JavaScript, the Web Audio API provides convenient high level functionality while retaining the ability to manually process samples if needed, and, of course, now works in both WebKit and Firefox.
- ehsanakhgari 13y ago
  For whatever it's worth, we have deprecated support for the Mozilla Audio Data API in favor of Web Audio, and will remove support for it in the near future.
antocv 13y ago
Good show, but I wonder how far we can really push the web browser, how will this affect desktop/native apps? We have already canvas and webgl and all but I dont see a web photoshop version competing with the real deal, how come? We're here giving access to "web-scripts" to more and more OS functions and closer to hardware, and yet the most productive apps are native, hm?
- wavesounds 13y ago
  How man years has it taken to get photoshop where it is today? Javascript and internet speeds have only recently gotten fast enough to compete. Give it another 5-10 years and then we'll see real competition for large desktop apps.
- kevincennis 13y ago
  I'm not sure that the point is necessarily that the browser should be competing with native pro apps. Web apps tend to be really good at very specialized functions. A silly, but I think still valid example would be a meme generator using <canvas> and JS. If you want a picture of a cat with some white Arial text and a black drop shadow, a web app is going to be great. It'll probably be faster than Photoshop (especially if you count startup time), and way more accessible. Same thing goes for the Web Audio API. Will we see a full-fledged Pro Tools competitor? Probably not. But we will see ringtone generators and voice recorders and (hopefully!) some T-Pain AutoTune apps. Great web apps are all about playing off the web's strengths, not trying to replicate the desktop.
wavesounds 13y ago
Does this mean Firefox will be able to play MP3s now?
- ehsanakhgari 13y ago
  We currently support MP3 playback for Windows Vista+ and Android if I'm not mistaken, and support for other platforms is being worked on. Web Audio will support all of the codecs supported by <audio> depending on the platform.
  - abrowne 13y ago
    And on Linux, if you enable gstreamer and if your gstream has mp3 support installed. (A lot of ifs, but it is the Linux desktop ...)
    - ehsanakhgari 13y ago
      Yes, that should be available in Nightly and Aurora: https://bugzilla.mozilla.org/show_bug.cgi?id=794282 https://bugzilla.mozilla.org/show_bug.cgi?id=794282
  - wavesounds 13y ago
    Thanks! MP3 support for OSX would be great. I was recently working on a music playing web app and was frustrated by this chart http://www.w3schools.com/html/html5_audio.asp http://www.w3schools.com/html/html5_audio.asp
    - ehsanakhgari 13y ago
      You can follow this bug https://bugzilla.mozilla.org/show_bug.cgi?id=851290 https://bugzilla.mozilla.org/show_bug.cgi?id=851290 if you're interested. :)
chipsy 13y ago
After trying to implement it, I've grown frustrated by the Web Audio spec. It has a lot of nice features, but they're ultimately justified by JS being slow. The spec itself is not very tight considering how much it covers - for example, I can easily see parameter changes behaving differently across different implementations. The real tests of the API haven't appeared yet, and if they fail all those features will be a waste because nobody will want to ship on it.
- ehsanakhgari 13y ago
  Do you have specific complaints? We're definitely eager to hear about them at the Audio working group.
  - chipsy 13y ago
    There is no specification for the computation of filter coefficients. Audio Nodes are subject to garbage collection delays. This effectively makes the API no-go for live performance situations. (I know you can't fix it either.) Parameter change specifications contain either an analytic or a mutable form, but in practical implementations, both forms become desirable to implement efficient processing that can also begin at arbitrary times within the envelope. I cancelled my implementation of the API after discovering that the lack of detail in this section made for a fragile, confusing, trial-and-error situation to prevent discrepancies when comparing the results of the analytic form against the mutable one. Rendering does not appear to have any automatic regression test available. Web Audio needs an Acid3-style test.
    - cromwellian 13y ago
      I'm not sure how reverting to a lower level JS-driven API could fix the performance issues. High performance audio, like 2D and 3D graphics, needs the ability to offload to native code or dedicated hardware. Writing a DSP in JS makes things portable, but so does writing an OpenGL rasterizer in CPU code. OpenGL to this day still has portability issues between GPUs, but the benefits outweigh the disadvantages. The only solution is IMHO is to keep finding holes in the spec, driven by feedback from application developers, and plugging the holes. I don't think a spec/standards committee can fix all of these issues, and like with OpenGL, I suspect, we'll need Web Audio 1.1/1.2/1.3/etc
md224 13y ago
I built a fairly basic ambient drone generator with the Chrome implementation of the Web Audio API: http://matt-diamond.com/drone.html http://matt-diamond.com/drone.html Haven't gotten a chance to add Firefox compatibility yet (working on it!).
- ehsanakhgari 13y ago
  That sounds great! Please get in touch with me if you hit any problems, would be happy to help!
dtf 13y ago
Fantastic news. I've used Web Audio under Chrome for a couple of experimental projects now, and it really is a fun API to work with.
paul9290 13y ago
Good stuff! Our mobile & desktop web app, SpeakerBlast, uses the web audio API to play audio in sync across multiple devices. Though up until now it only works in Chrome & in mobile Safari.
- ehsanakhgari 13y ago
  If you hit problems porting that to Firefox, please get in touch with me, would be happy to help you!
jamespitts 13y ago
Fantastic to see some of these things come to reality. In our real-time video entertainment game we've had to code all sorts of tricks in the client to get sound events to occur in sync across clients. Timed events is the most important aspect of creating a game or entertainment experience that sounds real to users. I completely took it for granted before I started on this project. Time is quite a rubbery material right now in browsers!
9999 13y ago
This is fantastic! I have been using the webkitAudioContext for a project and had a very simple script setup to prepare for WebAudio in FF: if (typeof AudioContext !== "undefined") { this.context = new AudioContext(); } It worked without modification, and the audio sounds great so far. In the meantime I'm still falling back to webkitAudioContext for Chrome and Safari until they catch up.
- ehsanakhgari 13y ago
  Great to hear that! Is this project publicly available somewhere by any chance?
  - 9999 13y ago
    Unfortunately it's not at the moment.
pnp 13y ago
This is great news. I've been playing around with audio streams generated by script (DAC emulation) on Chrome/Safari and now I can try Firefox. A key problem (and measure of quality) for me is reducing the latency. I've had trouble getting continuous sound with less than a quarter second of buffer. (Ideally, I'd like to achieve smooth sound with only 17ms of buffer/lag). Another browser to work with will help, and, maybe the IE team will take notice.
- ehsanakhgari 13y ago
  If you hit latency issues in Firefox, please get in touch with me. Note that we currently have a rather high latency on Android, which we have plans to fix, but things should work fairly well on desktop platforms.
andrethegiant 13y ago
For those interested, Apple documents the Web Audio API pretty well: http://developer.apple.com/library/safari/#documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/PlayingandSynthesizingSounds/PlayingandSynthesizingSounds.html http://developer.apple.com/library/safari/#documentation/Aud...
arc_of_descent 13y ago
This is great news! Been waiting for this. Now if only I don't need to use javascript for this. :)
Nekorosu 13y ago
That's huge! I've been waiting for this for quite some time. If Firefox also got some more UI polish it would be my fave browser.
adaline 13y ago
I have made something a little more advanced with Web Audio: http://mixbolt.com http://mixbolt.com Cant wait to have support on all the browsers (especially mobile) :)

118 points by rnyman 13y ago

comex 13y ago

Excellent. Although Firefox previously had a simple API that could be used to do most of these things manually in JavaScript, the Web Audio API provides convenient high level functionality while retaining the ability to manually process samples if needed, and, of course, now works in both WebKit and Firefox.

ehsanakhgari 13y ago
For whatever it's worth, we have deprecated support for the Mozilla Audio Data API in favor of Web Audio, and will remove support for it in the near future.

antocv 13y ago

Good show, but I wonder how far we can really push the web browser, how will this affect desktop/native apps? We have already canvas and webgl and all but I dont see a web photoshop version competing with the real deal, how come? We're here giving access to "web-scripts" to more and more OS functions and closer to hardware, and yet the most productive apps are native, hm?

wavesounds 13y ago
How man years has it taken to get photoshop where it is today? Javascript and internet speeds have only recently gotten fast enough to compete. Give it another 5-10 years and then we'll see real competition for large desktop apps.
kevincennis 13y ago
I'm not sure that the point is necessarily that the browser should be competing with native pro apps. Web apps tend to be really good at very specialized functions. A silly, but I think still valid example would be a meme generator using <canvas> and JS. If you want a picture of a cat with some white Arial text and a black drop shadow, a web app is going to be great. It'll probably be faster than Photoshop (especially if you count startup time), and way more accessible. Same thing goes for the Web Audio API. Will we see a full-fledged Pro Tools competitor? Probably not. But we will see ringtone generators and voice recorders and (hopefully!) some T-Pain AutoTune apps. Great web apps are all about playing off the web's strengths, not trying to replicate the desktop.

wavesounds 13y ago

Does this mean Firefox will be able to play MP3s now?

ehsanakhgari 13y ago
We currently support MP3 playback for Windows Vista+ and Android if I'm not mistaken, and support for other platforms is being worked on. Web Audio will support all of the codecs supported by <audio> depending on the platform.
- abrowne 13y ago
  And on Linux, if you enable gstreamer and if your gstream has mp3 support installed. (A lot of ifs, but it is the Linux desktop ...)
  - ehsanakhgari 13y ago
    Yes, that should be available in Nightly and Aurora: https://bugzilla.mozilla.org/show_bug.cgi?id=794282 https://bugzilla.mozilla.org/show_bug.cgi?id=794282
- wavesounds 13y ago
  Thanks! MP3 support for OSX would be great. I was recently working on a music playing web app and was frustrated by this chart http://www.w3schools.com/html/html5_audio.asp http://www.w3schools.com/html/html5_audio.asp
  - ehsanakhgari 13y ago
    You can follow this bug https://bugzilla.mozilla.org/show_bug.cgi?id=851290 https://bugzilla.mozilla.org/show_bug.cgi?id=851290 if you're interested. :)

chipsy 13y ago

After trying to implement it, I've grown frustrated by the Web Audio spec. It has a lot of nice features, but they're ultimately justified by JS being slow. The spec itself is not very tight considering how much it covers - for example, I can easily see parameter changes behaving differently across different implementations. The real tests of the API haven't appeared yet, and if they fail all those features will be a waste because nobody will want to ship on it.

ehsanakhgari 13y ago
Do you have specific complaints? We're definitely eager to hear about them at the Audio working group.
- chipsy 13y ago
  There is no specification for the computation of filter coefficients. Audio Nodes are subject to garbage collection delays. This effectively makes the API no-go for live performance situations. (I know you can't fix it either.) Parameter change specifications contain either an analytic or a mutable form, but in practical implementations, both forms become desirable to implement efficient processing that can also begin at arbitrary times within the envelope. I cancelled my implementation of the API after discovering that the lack of detail in this section made for a fragile, confusing, trial-and-error situation to prevent discrepancies when comparing the results of the analytic form against the mutable one. Rendering does not appear to have any automatic regression test available. Web Audio needs an Acid3-style test.
  - cromwellian 13y ago
    I'm not sure how reverting to a lower level JS-driven API could fix the performance issues. High performance audio, like 2D and 3D graphics, needs the ability to offload to native code or dedicated hardware. Writing a DSP in JS makes things portable, but so does writing an OpenGL rasterizer in CPU code. OpenGL to this day still has portability issues between GPUs, but the benefits outweigh the disadvantages. The only solution is IMHO is to keep finding holes in the spec, driven by feedback from application developers, and plugging the holes. I don't think a spec/standards committee can fix all of these issues, and like with OpenGL, I suspect, we'll need Web Audio 1.1/1.2/1.3/etc

md224 13y ago

I built a fairly basic ambient drone generator with the Chrome implementation of the Web Audio API: http://matt-diamond.com/drone.html http://matt-diamond.com/drone.html Haven't gotten a chance to add Firefox compatibility yet (working on it!).

ehsanakhgari 13y ago
That sounds great! Please get in touch with me if you hit any problems, would be happy to help!

dtf 13y ago

Fantastic news. I've used Web Audio under Chrome for a couple of experimental projects now, and it really is a fun API to work with.

paul9290 13y ago

Good stuff! Our mobile & desktop web app, SpeakerBlast, uses the web audio API to play audio in sync across multiple devices. Though up until now it only works in Chrome & in mobile Safari.

ehsanakhgari 13y ago
If you hit problems porting that to Firefox, please get in touch with me, would be happy to help you!

jamespitts 13y ago

Fantastic to see some of these things come to reality. In our real-time video entertainment game we've had to code all sorts of tricks in the client to get sound events to occur in sync across clients. Timed events is the most important aspect of creating a game or entertainment experience that sounds real to users. I completely took it for granted before I started on this project. Time is quite a rubbery material right now in browsers!

9999 13y ago

This is fantastic! I have been using the webkitAudioContext for a project and had a very simple script setup to prepare for WebAudio in FF: if (typeof AudioContext !== "undefined") { this.context = new AudioContext(); } It worked without modification, and the audio sounds great so far. In the meantime I'm still falling back to webkitAudioContext for Chrome and Safari until they catch up.

ehsanakhgari 13y ago
Great to hear that! Is this project publicly available somewhere by any chance?
- 9999 13y ago
  Unfortunately it's not at the moment.

pnp 13y ago

This is great news. I've been playing around with audio streams generated by script (DAC emulation) on Chrome/Safari and now I can try Firefox. A key problem (and measure of quality) for me is reducing the latency. I've had trouble getting continuous sound with less than a quarter second of buffer. (Ideally, I'd like to achieve smooth sound with only 17ms of buffer/lag). Another browser to work with will help, and, maybe the IE team will take notice.

ehsanakhgari 13y ago
If you hit latency issues in Firefox, please get in touch with me. Note that we currently have a rather high latency on Android, which we have plans to fix, but things should work fairly well on desktop platforms.

andrethegiant 13y ago

For those interested, Apple documents the Web Audio API pretty well: http://developer.apple.com/library/safari/#documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/PlayingandSynthesizingSounds/PlayingandSynthesizingSounds.html http://developer.apple.com/library/safari/#documentation/Aud...

arc_of_descent 13y ago

This is great news! Been waiting for this. Now if only I don't need to use javascript for this. :)

Nekorosu 13y ago

That's huge! I've been waiting for this for quite some time. If Firefox also got some more UI polish it would be my fave browser.

adaline 13y ago

I have made something a little more advanced with Web Audio: http://mixbolt.com http://mixbolt.com Cant wait to have support on all the browsers (especially mobile) :)