5 ms·
So its not just a single neural net. Its a bunch of different ML algorithms chained together. I'm essentially automating a 12(ish) piece electronic band. So f
by stevehiehn 9y ago
So its not just a single neural net. Its a bunch of different ML algorithms chained together. I'm essentially automating a 12(ish) piece electronic band. So first I have a ML algorithm that makes a chord progression. Then another that makes a bassline from the chords. Then another that makes a beat from the bassline & the chords and so on. At each step it uses the selected patterns to predict the next layer to be added. So to answer the question directly you can use RNN for some timeseries stuff. To decide if rhythm pattern are compatible I found success with CNN, random forests, and support vector machines. I even found you can make reasonable chord progressions just using N-grams. What I'm trying to say is I found I had to make an entire audio production pipeline that uses ML at each step of the way.
- p1esk 9y agoAre you planning to describe all that in more detail? I skimmed the 4 blog posts you published, but it's still not clear to me how the music is being generated. Also, why did you decide to separate the "layers"? Technically, an RNN trained on similar music should be able to generate all "layers" simultaneously.
- stevehiehn 9y agoI will hopefully add more detail eventually. When I say layer I am taking about a music layer like a melody not a layer of neurons. Most of the music is midi rendered with vst synths or sound fonts.
- p1esk 9y agoYes, that was what I meant when I said "layers" as well. RNN can generate all layers at once (more than one note per time step for a single instrument, and more than one instrument per time step overall). Basically what I'm saying is if you train it on such complex music, it will learn to generate it.
- stevehiehn 9y agoAh!, I think I understand what you are saying. I'm not sure if you used a NN before but you will notice that there is an exponential relationship with the number of inputs and the size of the training set required. Because I don't have millions of perfect training data I need to break down the problem and make ML models to deal with specific problems. For example predicting if a drum beat is compatible with a bassline etc.
- p1esk 9y agoWhat exponential relationship? Can you point me to any papers about it? It's actually quite easy to generate polyphonic (classical style) music with an RNN: just predict the entire note frequency range of one time step as a binary encoded vector. But yes, I can see how for multiple instruments, the sparsity of inputs might require more training samples. I'm wondering if the "layered" approach has some advantages, perhaps more control over some aspects of the music?
- stevehiehn 9y agoYa, there is no official relationship. From my experiments the more inputs you have the larger the dataset required. I do not know what is best way to do this but the layered approach is how humans compose music so it made sense to try that. I'm not sure I agree that its easy to generate polyphonic music. I would like to be wrong but when you say "easy" I don't think you're referring to music that follows a harmonic structure or factors in timbre. Most music I hear follows structure with repeating themes etc. Not just a stream of conscious series of notes.
- p1esk 9y agoWell, I just tried googling "rnn generated music": https://youtu.be/A2gyidoFsoI?t=1m44s https://youtu.be/A2gyidoFsoI?t=1m44s Sounds like a pretty good polyphony, but of course, it could be just overfitting on Bach training samples. Here are a couple of more sophisticated (and original) pieces: https://soundcloud.com/user-95265362/op-1-for-piano-solo-in-d-major https://soundcloud.com/user-95265362/op-1-for-piano-solo-in-... https://soundcloud.com/user-95265362/op-21-for-piano-solo-genesis https://soundcloud.com/user-95265362/op-21-for-piano-solo-ge...
- deleted 9y ago[deleted]
- p0nce 9y agoFascinating, thanks!