3 ms·
This is really, really awesome. Some other ideas that would be really cool: 1) Do key detection and pitch-shift all the loops to a common key before processing
by highd 9y ago
This is really, really awesome. Some other ideas that would be really cool:
1) Do key detection and pitch-shift all the loops to a common key before processing. That might make more of the melodies come through the eigenvectors.
2) Visualize the loop point cloud - maybe with like 10-50 dimensions of PCA followed by 2 or 3 dimensions T-SNE.
3) Maybe some form of earlier dimensionality reduction? I.e. you could do a short-time fourier transform and then threshold and bin frequencies - then invert the transform to reduce the sounds to their basic characteristics. That make make it so not all of the first 20 or so eigenvectors all have slightly different kick drums. For example, if you have kick drums with fundamentals at 21Hz, 22Hz, 23Hz, 24Hz etc. those will each require two eigenvectors to represent the sine or cosine phases of the signal - but if you could "project" every kick drum sound so they were close to linearly related then PCA could isolate them with fewer eigenvectors.
I would love to play with some sort of live music generation system based on this - really, really interesting ideas. And goes to show what can be built with traditional data analysis techniques and a clever idea!
EDIT: Also if you uploaded your preprocessed data I think that would be really amazing.
- umutisik 9y agoThank you for these great suggestions. I will make the dataset available eventually. It's about 15 gigs so I just need some time to do it right. I will let you know when I have it released.
- sideshowb 9y agoThese are all good suggestions. "Earlier dimensionality reduction" could possibly be used to reduce noise in the eigenvectors: while PCA removes noise as defined by dimensions with low overall variance, the results still sound noisy from a sound engineering perspective. I saw some research years ago about breaking down sound into pitched and unpitched components, using some kind of kernel transformation in fourier space. If I remember rightly this reduced the usual 'artifacts' you get when manipulating fourier transforms of white noise. Perhaps a good candidate for this case. Sorry I can't remember more though I think the talk was by this guy https://www.york.ac.uk/music/staff/academic/jez-wells/ https://www.york.ac.uk/music/staff/academic/jez-wells/ Either way, to the OP who I presume can only be Dr Brady of the Museum of Techno; good work old chap, you deserve a sherry.
- throwaypestban 9y agoHow do you visualize 10-50 dimensions? I'm asking because in Algebra courses I was told more than once to try not to visualize vectors of higher dimensions, and I understood that to mean it would at least be difficult.
- highd 9y agoIt's often a fools errand, but on real-world datasets people make scatter plots after using t-sne to project to 2 or 3 dimensions - in many cases it works out alright.