3 ms·

Are you planning to describe all that in more detail? I skimmed the 4 blog posts you published, but it's still not clear to me how the music is being generated.

by p1esk 9y ago

Are you planning to describe all that in more detail? I skimmed the 4 blog posts you published, but it's still not clear to me how the music is being generated. Also, why did you decide to separate the "layers"? Technically, an RNN trained on similar music should be able to generate all "layers" simultaneously.

stevehiehn 9y ago
I will hopefully add more detail eventually. When I say layer I am taking about a music layer like a melody not a layer of neurons. Most of the music is midi rendered with vst synths or sound fonts.
- p1esk 9y ago
  Yes, that was what I meant when I said "layers" as well. RNN can generate all layers at once (more than one note per time step for a single instrument, and more than one instrument per time step overall). Basically what I'm saying is if you train it on such complex music, it will learn to generate it.
  - stevehiehn 9y ago
    Ah!, I think I understand what you are saying. I'm not sure if you used a NN before but you will notice that there is an exponential relationship with the number of inputs and the size of the training set required. Because I don't have millions of perfect training data I need to break down the problem and make ML models to deal with specific problems. For example predicting if a drum beat is compatible with a bassline etc.
    - p1esk 9y ago
      What exponential relationship? Can you point me to any papers about it? It's actually quite easy to generate polyphonic (classical style) music with an RNN: just predict the entire note frequency range of one time step as a binary encoded vector. But yes, I can see how for multiple instruments, the sparsity of inputs might require more training samples. I'm wondering if the "layered" approach has some advantages, perhaps more control over some aspects of the music?
      - stevehiehn 9y ago
        Ya, there is no official relationship. From my experiments the more inputs you have the larger the dataset required. I do not know what is best way to do this but the layered approach is how humans compose music so it made sense to try that. I'm not sure I agree that its easy to generate polyphonic music. I would like to be wrong but when you say "easy" I don't think you're referring to music that follows a harmonic structure or factors in timbre. Most music I hear follows structure with repeating themes etc. Not just a stream of conscious series of notes.
        p1esk 9y ago
        Well, I just tried googling "rnn generated music": https://youtu.be/A2gyidoFsoI?t=1m44s https://youtu.be/A2gyidoFsoI?t=1m44s Sounds like a pretty good polyphony, but of course, it could be just overfitting on Bach training samples. Here are a couple of more sophisticated (and original) pieces: https://soundcloud.com/user-95265362/op-1-for-piano-solo-in-d-major https://soundcloud.com/user-95265362/op-1-for-piano-solo-in-... https://soundcloud.com/user-95265362/op-21-for-piano-solo-genesis https://soundcloud.com/user-95265362/op-21-for-piano-solo-ge...
        stevehiehn 9y ago
        Those do sound amazing! My goal is to create interactive music. If you noticed in my UI it can change at any point. In order to do that I think I need to premeditate the harmonic structure. i.e chord progression. I suspect that those RNN are not conforming to a predetermined harmonic structure. I doubt if you ask it say generate a piece following Dmaj9/Amaj/Bmin/Bmin it will understand that but I dont know.
        p1esk 9y ago
        if you ask it say generate a piece following Dmaj9/Amaj/Bmin/Bmin it will understand that Dmaj9/Amaj/Bmin/Bmin will be the seed sequence to the RNN. Of course, the network might ignore the seed, or if it's overfitting it will produce a chunk of the training sample which has this sequence in it, but you can definitely specify the starting point (and you usually do with RNNs).
deleted 9y ago
[deleted]

by p1esk 9y ago

stevehiehn 9y ago

I will hopefully add more detail eventually. When I say layer I am taking about a music layer like a melody not a layer of neurons. Most of the music is midi rendered with vst synths or sound fonts.

p1esk 9y ago
Yes, that was what I meant when I said "layers" as well. RNN can generate all layers at once (more than one note per time step for a single instrument, and more than one instrument per time step overall). Basically what I'm saying is if you train it on such complex music, it will learn to generate it.
- stevehiehn 9y ago
  Ah!, I think I understand what you are saying. I'm not sure if you used a NN before but you will notice that there is an exponential relationship with the number of inputs and the size of the training set required. Because I don't have millions of perfect training data I need to break down the problem and make ML models to deal with specific problems. For example predicting if a drum beat is compatible with a bassline etc.
  - p1esk 9y ago
    What exponential relationship? Can you point me to any papers about it? It's actually quite easy to generate polyphonic (classical style) music with an RNN: just predict the entire note frequency range of one time step as a binary encoded vector. But yes, I can see how for multiple instruments, the sparsity of inputs might require more training samples. I'm wondering if the "layered" approach has some advantages, perhaps more control over some aspects of the music?
    - stevehiehn 9y ago
      Ya, there is no official relationship. From my experiments the more inputs you have the larger the dataset required. I do not know what is best way to do this but the layered approach is how humans compose music so it made sense to try that. I'm not sure I agree that its easy to generate polyphonic music. I would like to be wrong but when you say "easy" I don't think you're referring to music that follows a harmonic structure or factors in timbre. Most music I hear follows structure with repeating themes etc. Not just a stream of conscious series of notes.
      - p1esk 9y ago
        Well, I just tried googling "rnn generated music": https://youtu.be/A2gyidoFsoI?t=1m44s https://youtu.be/A2gyidoFsoI?t=1m44s Sounds like a pretty good polyphony, but of course, it could be just overfitting on Bach training samples. Here are a couple of more sophisticated (and original) pieces: https://soundcloud.com/user-95265362/op-1-for-piano-solo-in-d-major https://soundcloud.com/user-95265362/op-1-for-piano-solo-in-... https://soundcloud.com/user-95265362/op-21-for-piano-solo-genesis https://soundcloud.com/user-95265362/op-21-for-piano-solo-ge...
        stevehiehn 9y ago
        Those do sound amazing! My goal is to create interactive music. If you noticed in my UI it can change at any point. In order to do that I think I need to premeditate the harmonic structure. i.e chord progression. I suspect that those RNN are not conforming to a predetermined harmonic structure. I doubt if you ask it say generate a piece following Dmaj9/Amaj/Bmin/Bmin it will understand that but I dont know.
        p1esk 9y ago
        if you ask it say generate a piece following Dmaj9/Amaj/Bmin/Bmin it will understand that Dmaj9/Amaj/Bmin/Bmin will be the seed sequence to the RNN. Of course, the network might ignore the seed, or if it's overfitting it will produce a chunk of the training sample which has this sequence in it, but you can definitely specify the starting point (and you usually do with RNNs).

deleted 9y ago

[deleted]