4 ms·
Can anybody explain why the researchers are attempting to generate the whole song as a single waveform, as opposed to wiring generated MIDI into some instrument
by grenoire 6y ago
Can anybody explain why the researchers are attempting to generate the whole song as a single waveform, as opposed to wiring generated MIDI into some instruments and separately a singing algorithm (perhaps a bit easier than the whole bulk work)?
- zeroxfe 6y agoIt's very hard to express all the nuances of real music and tonality in MIDI -- so generating raw audio side-steps all the limitations of a MIDI intermediary, and IMO, the results are absolutely phenomenal! (BTW, there are lots of AI music generators that generate MIDI, so it's less interesting either way.)
- TaylorAlexander 6y agoWell it’s not midi but what you’re describing is similar to this approach: https://magenta.tensorflow.org/ddsp https://magenta.tensorflow.org/ddsp
- mcleaveypayne 6y agoWe did work last year on MIDI alone - https://openai.com/blog/musenet/ https://openai.com/blog/musenet/ and some early work now on conditioning the raw audio based on MIDI (early results at the bottom of the Jukebox blog). Agreed though there should be interesting results from modeling different blends of MIDI, stem, and raw audio data. Raw audio alone gives us the most flexibility in terms of the kinds of sounds we can create, but it's also the most challenging to get good long term structure. Still lots more work to be done!
- mycall 6y agoSomething like MOD/XM music comes to mind.