5 ms·

Generation of Music Using Pure JS and HTML5

54 points by game_man 14y ago

bluetidepro 14y ago
This is really neat! I wonder if this would be a smaller file/render size to implement over a normal audio file for HTML5 games. If so, I'm sure with a bit of work, you could make a cool "8 bit sounding" theme song for your game!
chuppo 14y ago
This should make it easier to make a pure js NES emulator. We have sprites/canvas/webgl, and this level of audio fits perfectly.
- mistercow 14y ago
  The problem is that this lets you generate a buffer, and when you're done with that, play it back, but it doesn't let you push blocks of audio continually while previous blocks play. That will be necessary for any real-time audio application like NES emulation. Ultimately, this should be solved with the Web Audio API, but so far that appears to implemented in Firefox/Chrome/Safari, and only partly in those. I can see a Flash shim being useful there. Now, in the mean time, I do have an idea of how this might be implemented on top of Riffwave.js, but it's iffy. The problem is that while the HTML5 audio element does have some events that tell you about playback status, we cannot possibly rely on JS events (or the placement of new audio tags) to give us the temporal granularity needed for audio timing. If you mistime the beginning of one block from the end of the previous block by half a millisecond, you'll hear an audible pop. So one possible solution would be to crossfade blocks, using an overlap period on the same order as your timing uncertainty. In the cleverest implementation, the timing uncertainty would be continually measured and the overlap size would be adapted to be as low as possible. But that's not a perfect solution. It would ameliorate the problem, but it would also introduce its own artifacts. For example, if you have a 440 Hz sine wave playing, and one block is offset by the next by 1.13 ms, then at the midpoint between two blocks (where each block is at 50% volume), the waves will interfere completely, resulting in a point of silence. The artifact would be soft, but it would still be a problem.
nicolasmiller 14y ago
This rules. The other weekend I was feeling curious and started a little project to see what it takes to get up and running doing basic audio synthesis: https://github.com/nicolasmiller/bleeper https://github.com/nicolasmiller/bleeper. I got distracted with some other stuff, but the goal is to compile examples in as many languages/environments as possible.
jlongster 14y ago
I'm at a conference, so I can't listen to it, but I made something kind of similar that responds to touch events: http://jlongster.com/s/touch.html#notes http://jlongster.com/s/touch.html#notes Requires Web Audio API. I talked about it here: http://jlongster.com/2012/09/12/web-apps.html http://jlongster.com/2012/09/12/web-apps.html
AlexC04 14y ago
I looked at the JS over here http://codebase.es/riffwave/riffwave.js http://codebase.es/riffwave/riffwave.js and find this line this.dataURI = 'data:audio/wav;base64,'+FastBase64.Encode(this.wav); (edit... more accurately the whole RIFFWAVE function blows my mind) Really quite revolutionary (in terms of my understanding)... I already knew about making JPEGS etc, so this makes perfect sense but just seeing him build all the headers for a wav file in memory then "spit it out" to the page is stunning. Kudos, Bravo, Standing slow clap, etc... Thanks for sharing.
nsxwolf 14y ago
Fun and educational. Reminds me of those first experiments with music creation on 8-bit BASIC machines.
ocivelek 14y ago
web audio api also does help.. for instance https://github.com/dashersw/pedalboard.js https://github.com/dashersw/pedalboard.js
- game_man 14y ago
  Awesome.. thanks for sharing!
valdiorn 14y ago
Yeah, that's MY CODE, mate. Maybe give some credit, eh? Edit: Looks like OP just borrowed my note array.. which I in turn borrowed from Aphex twin, so no harm done... :) Also, it isn't working 100% in Chrome, the sampling rate is probably wrong. You can check out my example Here: http://valdemarorn.github.com/Files/JSAudio/index.html http://valdemarorn.github.com/Files/JSAudio/index.html Anyway, Riffwave was posted here yesterday, and I posted a similar example I did with WebAudioAPI. Basically, it's that code that OP has adapted to fit into RiffWave. My code can be found here: https://github.com/ValdemarOrn/valdemarorn.github.com/tree/master/Files/JSAudio https://github.com/ValdemarOrn/valdemarorn.github.com/tree/m...
- kushsolitary 14y ago
  Yes the notes sequence and the base frequency algorithm is yours and I notified you about using it here. http://news.ycombinator.com/item?id=4561565 http://news.ycombinator.com/item?id=4561565 I am sorry for not giving credit to you on the main page. Edit: BTW, what is wrong with the sampling frequency? I think 44KHz is sufficient and it's working fine on my Chrome. A fork maybe?
  - valdiorn 14y ago
    No problem, I was just a little surprised when I saw it :)
bkanber 14y ago
That distortion you're hearing is because of sharp changes in your signal's amplitude. To get rid of it, you should apply a little bit of smoothing to the attack and release of individual notes.
- pacaro 14y ago
  A little ADSR envelope and one smidgen of FM synthesis helps... var data = [], sampleRateHz = 44100, lengthInSeconds = 8, sampleCount = sampleRateHz * lengthInSeconds, //Notes sequence, play with it ;) notes = [87, 0, 87, 0, 85, 0, 82, 0, 80, 0, 82, 82, 80, 0, 82, 0, 85, 0, 78, 0, 78, 78], noteLength = sampleCount / notes.length, //Base Frequency based on the notes baseFreq = function(index) { var r = 2 * Math.PI * 220.0 * Math.pow(2,(notes[index]-69)/12.0) / sampleRateHz; return r; }, envelope = function(offset, attack, decay, sustain, release) { if (offset <= attack) { return offset / attack } offset -= attack; if (offset <= decay) { var prop = offset / decay; return 1 + prop * (sustain - 1); } offset -= decay; var susLen = 1 - (attack + decay + release); if (offset <= susLen) { return sustain; } offset -= susLen; return sustain * (1 - offset / release); }; //Fill up the sound data!! for(var j = 0; j < sampleCount; j++) { var currNote = Math.round(j/noteLength); var currFreq = baseFreq(currNote); var sample = currFreq * j + 0.5 * Math.sin(currFreq * j * 1.21); var offset = (j % noteLength) / noteLength; var amp = envelope(offset, 0.05, 0.01, 0.5, 0.75); data[j] = 128 + Math.round(96 * amp * Math.sin(sample)); }
  - kushsolitary 14y ago
    this helped a LOT! I created a fork just to test and the sound was definitely awesome! http://cssdeck.com/labs/5atlvhph/0 http://cssdeck.com/labs/5atlvhph/0
quinnirill 14y ago
If implementing every effect out there by yourself becomes tedious, you might wanna give audiolib.js a shot: https://github.com/jussi-kalliokoski/audiolib.js https://github.com/jussi-kalliokoski/audiolib.js , it's an audio effects library I've been building for 2 years now. :)

54 points by game_man 14y ago

This is really neat! I wonder if this would be a smaller file/render size to implement over a normal audio file for HTML5 games. If so, I'm sure with a bit of work, you could make a cool "8 bit sounding" theme song for your game!

chuppo 14y ago

This should make it easier to make a pure js NES emulator. We have sprites/canvas/webgl, and this level of audio fits perfectly.

mistercow 14y ago
The problem is that this lets you generate a buffer, and when you're done with that, play it back, but it doesn't let you push blocks of audio continually while previous blocks play. That will be necessary for any real-time audio application like NES emulation. Ultimately, this should be solved with the Web Audio API, but so far that appears to implemented in Firefox/Chrome/Safari, and only partly in those. I can see a Flash shim being useful there. Now, in the mean time, I do have an idea of how this might be implemented on top of Riffwave.js, but it's iffy. The problem is that while the HTML5 audio element does have some events that tell you about playback status, we cannot possibly rely on JS events (or the placement of new audio tags) to give us the temporal granularity needed for audio timing. If you mistime the beginning of one block from the end of the previous block by half a millisecond, you'll hear an audible pop. So one possible solution would be to crossfade blocks, using an overlap period on the same order as your timing uncertainty. In the cleverest implementation, the timing uncertainty would be continually measured and the overlap size would be adapted to be as low as possible. But that's not a perfect solution. It would ameliorate the problem, but it would also introduce its own artifacts. For example, if you have a 440 Hz sine wave playing, and one block is offset by the next by 1.13 ms, then at the midpoint between two blocks (where each block is at 50% volume), the waves will interfere completely, resulting in a point of silence. The artifact would be soft, but it would still be a problem.

nicolasmiller 14y ago

This rules. The other weekend I was feeling curious and started a little project to see what it takes to get up and running doing basic audio synthesis: https://github.com/nicolasmiller/bleeper https://github.com/nicolasmiller/bleeper. I got distracted with some other stuff, but the goal is to compile examples in as many languages/environments as possible.

jlongster 14y ago

I'm at a conference, so I can't listen to it, but I made something kind of similar that responds to touch events: http://jlongster.com/s/touch.html#notes http://jlongster.com/s/touch.html#notes Requires Web Audio API. I talked about it here: http://jlongster.com/2012/09/12/web-apps.html http://jlongster.com/2012/09/12/web-apps.html

AlexC04 14y ago

I looked at the JS over here http://codebase.es/riffwave/riffwave.js http://codebase.es/riffwave/riffwave.js and find this line this.dataURI = 'data:audio/wav;base64,'+FastBase64.Encode(this.wav); (edit... more accurately the whole RIFFWAVE function blows my mind) Really quite revolutionary (in terms of my understanding)... I already knew about making JPEGS etc, so this makes perfect sense but just seeing him build all the headers for a wav file in memory then "spit it out" to the page is stunning. Kudos, Bravo, Standing slow clap, etc... Thanks for sharing.

nsxwolf 14y ago

Fun and educational. Reminds me of those first experiments with music creation on 8-bit BASIC machines.

ocivelek 14y ago

web audio api also does help.. for instance https://github.com/dashersw/pedalboard.js https://github.com/dashersw/pedalboard.js

game_man 14y ago
Awesome.. thanks for sharing!

valdiorn 14y ago

Yeah, that's MY CODE, mate. Maybe give some credit, eh? Edit: Looks like OP just borrowed my note array.. which I in turn borrowed from Aphex twin, so no harm done... :) Also, it isn't working 100% in Chrome, the sampling rate is probably wrong. You can check out my example Here: http://valdemarorn.github.com/Files/JSAudio/index.html http://valdemarorn.github.com/Files/JSAudio/index.html Anyway, Riffwave was posted here yesterday, and I posted a similar example I did with WebAudioAPI. Basically, it's that code that OP has adapted to fit into RiffWave. My code can be found here: https://github.com/ValdemarOrn/valdemarorn.github.com/tree/master/Files/JSAudio https://github.com/ValdemarOrn/valdemarorn.github.com/tree/m...

kushsolitary 14y ago
Yes the notes sequence and the base frequency algorithm is yours and I notified you about using it here. http://news.ycombinator.com/item?id=4561565 http://news.ycombinator.com/item?id=4561565 I am sorry for not giving credit to you on the main page. Edit: BTW, what is wrong with the sampling frequency? I think 44KHz is sufficient and it's working fine on my Chrome. A fork maybe?
- valdiorn 14y ago
  No problem, I was just a little surprised when I saw it :)

bkanber 14y ago

That distortion you're hearing is because of sharp changes in your signal's amplitude. To get rid of it, you should apply a little bit of smoothing to the attack and release of individual notes.

pacaro 14y ago
A little ADSR envelope and one smidgen of FM synthesis helps... var data = [], sampleRateHz = 44100, lengthInSeconds = 8, sampleCount = sampleRateHz * lengthInSeconds, //Notes sequence, play with it ;) notes = [87, 0, 87, 0, 85, 0, 82, 0, 80, 0, 82, 82, 80, 0, 82, 0, 85, 0, 78, 0, 78, 78], noteLength = sampleCount / notes.length, //Base Frequency based on the notes baseFreq = function(index) { var r = 2 * Math.PI * 220.0 * Math.pow(2,(notes[index]-69)/12.0) / sampleRateHz; return r; }, envelope = function(offset, attack, decay, sustain, release) { if (offset <= attack) { return offset / attack } offset -= attack; if (offset <= decay) { var prop = offset / decay; return 1 + prop * (sustain - 1); } offset -= decay; var susLen = 1 - (attack + decay + release); if (offset <= susLen) { return sustain; } offset -= susLen; return sustain * (1 - offset / release); }; //Fill up the sound data!! for(var j = 0; j < sampleCount; j++) { var currNote = Math.round(j/noteLength); var currFreq = baseFreq(currNote); var sample = currFreq * j + 0.5 * Math.sin(currFreq * j * 1.21); var offset = (j % noteLength) / noteLength; var amp = envelope(offset, 0.05, 0.01, 0.5, 0.75); data[j] = 128 + Math.round(96 * amp * Math.sin(sample)); }
- kushsolitary 14y ago
  this helped a LOT! I created a fork just to test and the sound was definitely awesome! http://cssdeck.com/labs/5atlvhph/0 http://cssdeck.com/labs/5atlvhph/0

quinnirill 14y ago

If implementing every effect out there by yourself becomes tedious, you might wanna give audiolib.js a shot: https://github.com/jussi-kalliokoski/audiolib.js https://github.com/jussi-kalliokoski/audiolib.js , it's an audio effects library I've been building for 2 years now. :)