3 ms·

Would love to see something like this auto-mix song transitions in playlists to be more DJ-like.

by kbouck 3y ago

jareklupinski 3y ago
I started down this road, and have amassed a small collection of files made up of just the first ten and last ten seconds of demixed tracks of every song in my library I hoped to put together a system that compares the backs to the fronts and lists the output to find cool transitions, but I have no idea how to actually "grade" the similarities Beyond basic BPM matching on the drums tracks, nothing I've tried has made for anything really compelling (sounds random... :( )
- sevagh 3y ago
  OK, so, tangentially related: I tried to do something once - I took small chunks of songs generated by SampleRNN in an attempt to stitch together the ones that sounded the most similar to create a much longer track. The script [1] uses Essentia Chromaprint [2] to "grade" the similarity of audio tracks, and combine the ones with the closest chromaprint. No crossfade or BPM matching, just yolo concatenation. I have a track on Soundcloud which uses the above technique (mashing together short generated clips by their chromagram), trained on Cannibal Corpse [3] 1: https://github.com/sevagh/1000sharks.xyz/blob/master/samplernn-scripts/curation.py https://github.com/sevagh/1000sharks.xyz/blob/master/sampler... 2: https://essentia.upf.edu/reference/std_Chromaprinter.html https://essentia.upf.edu/reference/std_Chromaprinter.html 3: https://soundcloud.com/user-167126026/1000sharks-domainal-skeleton?in=user-167126026/sets/1000sharks-ai-generated-metal&si=ab01893e5c864f33bb5cea170877ce09&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing https://soundcloud.com/user-167126026/1000sharks-domainal-sk...
- tristanc 3y ago
  Interesting, I attempted to do the same as you but stopped just shy of BPM matching. However I did get sound similarity working using an audio tagging neural net [1]. I chopped off the first and last 15 seconds of every song in my collection and ran them all through this analysis which produces a ~520 dimensional vector. I then targeted specific endings I wanted to match and used Euclidian distance to find the closest matching song beginning. YMMV but I thought it actually worked pretty well, I just never got to automating the BPM matching. I can try to look for my old script if you're interested :) [1] https://github.com/fschmid56/EfficientAT https://github.com/fschmid56/EfficientAT
- kbouck 3y ago
  my idea was to let an existing app like djay do the beat matching (which it does really well!) and demixing (less well), and then make a custom app that would act like a midi controller, adjusting the vocals, beats, melody while songs are transitioning, perhaps working off of cues bookmarked in the songs.
holoduke 3y ago
In a few years from now DJs will have buttons on their pads to extract voices, beats, melodies etc realtime. Could result into an interesting new style of club music.
- kbouck 3y ago
  this already exists in the djay app (paid add-on feature). even runs on a smartphone! performs about the same as the app this thread is about.
  - kbouck 3y ago
    > re: "realtime" it separates the components of tracks that it can download (and process), not of a live audio feed
- deleted 3y ago
  [deleted]
- crtasm 3y ago
  DJs have this now, the track still needs to be processed but that can happen in advance or be done on-the-fly (VirtualDJ + a GPU takes about 10s to process a track and can do so seamlessly while the track is playing).
- ragazzina 3y ago
  The AI could really just: - select the best next song given a simple input (for example, a microphone or a camera looking into the crowd) - mix it into the current song - repeat
Gordonjcp 3y ago
Well, the naive way to do it would be to detect the onset of each beat by assuming it corresponds to kick drums on fours, and then speeding up or slowing down the track you're mixing into to correspond with it, timing it so the fours line up. This would be very easy to implement and over a longish crossfade (say a couple of bars) would be fairly convincing. Using AI you could probably intuit the "pulse" of the music to find the first beat of the first bar of a measure, and sync tracks up so that they mix in a place that makes musical sense.

Would love to see something like this auto-mix song transitions in playlists to be more DJ-like.

by kbouck 3y ago

Would love to see something like this auto-mix song transitions in playlists to be more DJ-like.

I started down this road, and have amassed a small collection of files made up of just the first ten and last ten seconds of demixed tracks of every song in my library I hoped to put together a system that compares the backs to the fronts and lists the output to find cool transitions, but I have no idea how to actually "grade" the similarities Beyond basic BPM matching on the drums tracks, nothing I've tried has made for anything really compelling (sounds random... :( )

sevagh 3y ago
OK, so, tangentially related: I tried to do something once - I took small chunks of songs generated by SampleRNN in an attempt to stitch together the ones that sounded the most similar to create a much longer track. The script [1] uses Essentia Chromaprint [2] to "grade" the similarity of audio tracks, and combine the ones with the closest chromaprint. No crossfade or BPM matching, just yolo concatenation. I have a track on Soundcloud which uses the above technique (mashing together short generated clips by their chromagram), trained on Cannibal Corpse [3] 1: https://github.com/sevagh/1000sharks.xyz/blob/master/samplernn-scripts/curation.py https://github.com/sevagh/1000sharks.xyz/blob/master/sampler... 2: https://essentia.upf.edu/reference/std_Chromaprinter.html https://essentia.upf.edu/reference/std_Chromaprinter.html 3: https://soundcloud.com/user-167126026/1000sharks-domainal-skeleton?in=user-167126026/sets/1000sharks-ai-generated-metal&si=ab01893e5c864f33bb5cea170877ce09&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing https://soundcloud.com/user-167126026/1000sharks-domainal-sk...
tristanc 3y ago
Interesting, I attempted to do the same as you but stopped just shy of BPM matching. However I did get sound similarity working using an audio tagging neural net [1]. I chopped off the first and last 15 seconds of every song in my collection and ran them all through this analysis which produces a ~520 dimensional vector. I then targeted specific endings I wanted to match and used Euclidian distance to find the closest matching song beginning. YMMV but I thought it actually worked pretty well, I just never got to automating the BPM matching. I can try to look for my old script if you're interested :) [1] https://github.com/fschmid56/EfficientAT https://github.com/fschmid56/EfficientAT
kbouck 3y ago
my idea was to let an existing app like djay do the beat matching (which it does really well!) and demixing (less well), and then make a custom app that would act like a midi controller, adjusting the vocals, beats, melody while songs are transitioning, perhaps working off of cues bookmarked in the songs.

holoduke 3y ago

In a few years from now DJs will have buttons on their pads to extract voices, beats, melodies etc realtime. Could result into an interesting new style of club music.

kbouck 3y ago
this already exists in the djay app (paid add-on feature). even runs on a smartphone! performs about the same as the app this thread is about.
- kbouck 3y ago
  > re: "realtime" it separates the components of tracks that it can download (and process), not of a live audio feed
deleted 3y ago
[deleted]
crtasm 3y ago
DJs have this now, the track still needs to be processed but that can happen in advance or be done on-the-fly (VirtualDJ + a GPU takes about 10s to process a track and can do so seamlessly while the track is playing).
ragazzina 3y ago
The AI could really just: - select the best next song given a simple input (for example, a microphone or a camera looking into the crowd) - mix it into the current song - repeat

Gordonjcp 3y ago

Well, the naive way to do it would be to detect the onset of each beat by assuming it corresponds to kick drums on fours, and then speeding up or slowing down the track you're mixing into to correspond with it, timing it so the fours line up. This would be very easy to implement and over a longish crossfade (say a couple of bars) would be fairly convincing. Using AI you could probably intuit the "pulse" of the music to find the first beat of the first bar of a measure, and sync tracks up so that they mix in a place that makes musical sense.