4 ms·

For context, here's some recent computer music, both MIDI and direct audio generation: https://magenta.tensorflow.org/perceiver-ar https://magenta.tensorflow.o

For context, here's some recent computer music, both MIDI and direct audio generation: https://magenta.tensorflow.org/perceiver-ar https://magenta.tensorflow.org/perceiver-ar Here is an older project that generated audio from lyrics: https://openai.com/blog/jukebox/ https://openai.com/blog/jukebox/ Music has two key difficulties: the long, linear context makes it hard for most models to generate coherent works and reason about long-term higher level structure, and the global structure being learned is much more function of human preference, rather than of physical constraints. That said, I don't think the gap is all that sizeable. Jukebox is easily comparable in quality to the image synthesis from arbitrary text of its time. Perceiver AR is not as impressive as DALL-E 2, sure, but DALL-E 2 is new, and it's plenty of evidence that music synthesis will continue to work.

mrandish 4y ago
> I don't think the gap is all that sizeable. As an enthusiastic amateur producer of music using desktop PC MIDI software and instrument plugins, I agree. The samples you linked already seem coherent enough to at least be useful in helping untrained amateurs like myself come up with ideas for subsequent musical phrases by proposing variations based on my existing midi chords, BPM and perhaps a genre selection. It would only need to suggest plausible continuations to be useful because if the suggestions are in MIDI, it's easy for me to riff and revise any that sound interesting. Even a 'hit rate' of one in five suggestions would be compelling for the 'hobbyist-assist' use case.
woojoo666 4y ago
> Music has two key difficulties: the long, linear context makes it hard for most models to generate coherent works and reason about long-term higher level structure It seems like the DeepMind PerceiverAR that you linked generates entire piano pieces all at once, instead of generating token by token like OpenAI GPT or Jukebox. So that should take care of long-term coherence right? Since all parts of the song are processed and generated at once
- Veedrac 4y ago
  Perceiver AR does solve it that way, yes, albeit only for MIDI. Direct audio generation is still too long for it to keep a full piece in context.

mrandish 4y ago

> I don't think the gap is all that sizeable. As an enthusiastic amateur producer of music using desktop PC MIDI software and instrument plugins, I agree. The samples you linked already seem coherent enough to at least be useful in helping untrained amateurs like myself come up with ideas for subsequent musical phrases by proposing variations based on my existing midi chords, BPM and perhaps a genre selection. It would only need to suggest plausible continuations to be useful because if the suggestions are in MIDI, it's easy for me to riff and revise any that sound interesting. Even a 'hit rate' of one in five suggestions would be compelling for the 'hobbyist-assist' use case.

woojoo666 4y ago

> Music has two key difficulties: the long, linear context makes it hard for most models to generate coherent works and reason about long-term higher level structure It seems like the DeepMind PerceiverAR that you linked generates entire piano pieces all at once, instead of generating token by token like OpenAI GPT or Jukebox. So that should take care of long-term coherence right? Since all parts of the song are processed and generated at once

Veedrac 4y ago
Perceiver AR does solve it that way, yes, albeit only for MIDI. Direct audio generation is still too long for it to keep a full piece in context.