3 ms·
It all comes down to generating tokens. The dominant paradigm is to compose the music by generating the next token based on the last few seen tokens, then take
by bjourne 5y ago
It all comes down to generating tokens. The dominant paradigm is to compose the music by generating the next token based on the last few seen tokens, then take that and generate the next token, and so on. No one has been able to significantly improve upon that. All development in the last ten years or so has been about creating ever larger models with more data so that emitted sequences will stay coherent for longer. The problem with this paradigm is that it works amazingly well for short sequences, but often fails to stay coherent for longer sequences. So composing interesting ten seconds long instrumental music is a solved problem - composing full songs three-to-five minutes long is far beyond the reach of current state-of-the-art.
Text-generating models suffer from the same problem. E.g. GPT-3 will generate two to three-hundred tokens or so and then it will begin to spout nonsense. That it still works so well is because it's trained with a massive corpus and because its tokens are richer. It operates on the world level, while most music composition models operate on the note level which are more analogous to characters than words. If someone were to be able to figure out how to represent multi-instrumental compositions using "music vectors" (analogous to word vectors) that would probably lead to a major breakthrough.
- pishpash 5y agoProbably these would be harmonic units. Unfortunately notes aren't just characters in this. There's also voice leading, and so harmony "words" run into each other and can't be mutually exclusively tokenized. The better analogy might be poetry, and I'm not sure if there are good poetry generators, either.
- OneEyedRobot 5y agoBy 'token' do you mean a note, a chord, general structure (verse, bridge, etc.), a well known phrase? Music is so full of cliche I don't doubt that someone will write software (probably already has) to assist a composer, generate drum parts or harmony, etc. I like the idea of a Rhythm changes generator. With enough monkeys the Flintstone Theme eventually pops out.
- snthd 5y ago>(probably already has) https://github.com/Impro-Visor/Impro-Visor https://github.com/Impro-Visor/Impro-Visor
- bjourne 5y agoA token can be almost anything as long as a sequence of tokens represent something meaningful. For music, I and many others have represented music as sequences of notes interleaved with rests of varying length. E.g "A--3 R3 C#-2" Three tokens to be interpreted as "play an A note in the 3rd octave at tic 0, wait 3 tics, play a C# note in the 2nd octave at tic 4". As you can imagine, the number of tokens grow very fast especially for polyphonic music. You might say that that level of detail is unnecessary for most music. For a 12-bar blues in E you might only need five tokens, "blues 12 bars key E", to represent the whole song. But what if you want to add some cool pentatonic riffs to it and maybe a key change in the middle? Either you add token types for "key change in the middle" and "cool pentatonic riffs" or you will need many more tokens to represent your song. One can think of it as compression. English text is trivial to compress because you just match each word to a word index in a lookup table. Compositions are way harder to compress.
- PaulDavisThe1st 5y agoAn important reason why it is so hard to compress them is that errors frequently don't really matter in the same way they do with language. You can really play a melody with many, many variations and it is still (to a point) the same melody. This is much harder to do with language, and has important implications when thinking about what a composition "actually is".
- TomSwirly 5y ago> It all comes down to generating tokens. So counterpoint is just out the window, is it?
- PaulDavisThe1st 5y agoCounterpoint is just a different way to answer the question "what note should follow the one I just played". There's nothing particularly special about it from an ML perspective.
- bjourne 5y agoCounterpoint is a special case of multiple voices and is handled by interleaving. Let A_i be what instrument A does at time t and B_i what instrument B does at the same time. Then with five tokens look back: f(A_1 B_1 A_2 B_2 A_3) = B_3 and f(B_1 A_2 B_2 A_3 B_3) = A_4 and so on. There are variations to this theme. For example, you could generate the previous token at each time step instead of the next one but, essentially, it is the same idea.