5 ms·

Machine Learning for Drummers

101 points by psobot 8y ago

bagrow 8y ago
Surprised there's no discussion of FFT, power spectra, etc. Would like to see someone with an electrical engineering/signal processing background work on this problem.
- ssalazar 8y ago
  Stock FFT is a really high-dimension feature vector given the number of training examples used here, and most of the resolution of the FFT would be unneeded anyways. "Average loudness in several frequency ranges" captures spectral information at a granularity much more appropriate to the data and classification task. For analyzing drum samples you don't need a lot of frequency resolution, although other low-dimension spectral features like MFCCs or flux would probably be useful.
zneveu 8y ago
Had an idea to do this a couple months ago, but haven't got around to implementing it yet. I'm curious: did you consider using standard image processing techniques with spectrograms as an alternative to decision trees? I know thats how Izotope does their Neutron instrument detection, but I'm not sure how it would compare performance wise. Also, have you tried classifying percussive sounds that aren't actual drums? I'd love to see how it categorizes various stuff.
- psobot 8y ago
  Hey! In order: - I did consider using image processing techniques as opposed to decision trees, but the point here was not to come up with the most advanced and accurate classifier possible, but rather to build something simple and explainable to folks without an ML (or even a CA) background. - I haven't tried this extensively on non-drum-like percussion, but that'd be a great follow up post.
  - blt 8y ago
    I totally understand your decision, but I bet a 1D deep convolutional network would do really well at this given a larger dataset. You can also do a lot of data augmentation by speed changes, filters, adding reverb, etc.
flashman 8y ago
Could I use something like this to identify which of two or three people is speaking in an audio clip? Assume I can label several samples of each person's speech, then present an unlabeled sample for classification.
- RileyJames 8y ago
  I’m looking for something that can do this as well. Anything out already?
  - flashman 8y ago
    I had a go of it by replacing the drum samples with voice samples (both 1-2 seconds and 3-5 seconds), then removing the features concerned with length and volume. Fiddled with the number of sub-sections per sample, and some of the random forest settings, but never consistently got higher than 77% accuracy between the four speakers. Maybe it would do better with two speakers.
- psobot 8y ago
  Yes! While the technique I used in this post is pretty simplistic, you could use a similar method to solve what's generically known as the [speaker recognition problem](https://en.wikipedia.org/wiki/Speaker_recognition https://en.wikipedia.org/wiki/Speaker_recognition). That said, this is an open problem within audio research, and there are solutions much more complex than what I have here. An example of one open source speaker recognition project is [bob.bio.spear](https://pypi.org/project/bob.bio.spear/ https://pypi.org/project/bob.bio.spear/), which I haven't tried, but looks promising.
TeMPOraL 8y ago
Great! Refreshing to see a ML post using some well-understood methods instead of throwing a random neural net from Kaggle at the problem... Tangential: > Is a given audio file a sample of a kick drum, snare drum, hi-hat, other percussion, or something else? (...) Humans have no trouble classifying these two sounds, as we’ve likely heard them tens of thousands of times before. Are people taught that in schools or something? Because I personally can't classify those sounds, don't know these names, and I'm not sure how I was supposed to learn them, other by playing in a band.
- ludston 8y ago
  > Are people taught that in schools or something? Because I personally can't classify those sounds, don't know these names, and I'm not sure how I was supposed to learn them, other by playing in a band. This is something that is taught at schools with a music program. (Although, not necessarily discretely). If you are someone whom has played music before, it is easy to forget what music sounded like before your ear was trained. (i.e. certain instruments and harmonies can be indistinguishable without training) Is it common to have never played on at a drum kit in your entire life?
  - TeMPOraL 8y ago
    Thanks! > Is it common to have never played on at a drum kit in your entire life? I didn't, not on a real one at least. I know the sounds though, I spent ungodly amount of time playing on an electronic keyboard as a kid, where I could (and often would) change the sounds under keys to drums. However, nowhere (AFAIR) were the names of those sounds mentioned, and I'm not sure where I could encounter them.
    - jbenner-radham 8y ago
      In my anecdotal experience I only know the names of various percussion by sound because I was a drummer in band class at school. My friends weren’t taught that in the general education program though. That could possibly vary by region though.
- 8y ago

101 points by psobot 8y ago

Surprised there's no discussion of FFT, power spectra, etc. Would like to see someone with an electrical engineering/signal processing background work on this problem.

ssalazar 8y ago
Stock FFT is a really high-dimension feature vector given the number of training examples used here, and most of the resolution of the FFT would be unneeded anyways. "Average loudness in several frequency ranges" captures spectral information at a granularity much more appropriate to the data and classification task. For analyzing drum samples you don't need a lot of frequency resolution, although other low-dimension spectral features like MFCCs or flux would probably be useful.

zneveu 8y ago

Had an idea to do this a couple months ago, but haven't got around to implementing it yet. I'm curious: did you consider using standard image processing techniques with spectrograms as an alternative to decision trees? I know thats how Izotope does their Neutron instrument detection, but I'm not sure how it would compare performance wise. Also, have you tried classifying percussive sounds that aren't actual drums? I'd love to see how it categorizes various stuff.

psobot 8y ago
Hey! In order: - I did consider using image processing techniques as opposed to decision trees, but the point here was not to come up with the most advanced and accurate classifier possible, but rather to build something simple and explainable to folks without an ML (or even a CA) background. - I haven't tried this extensively on non-drum-like percussion, but that'd be a great follow up post.
- blt 8y ago
  I totally understand your decision, but I bet a 1D deep convolutional network would do really well at this given a larger dataset. You can also do a lot of data augmentation by speed changes, filters, adding reverb, etc.

flashman 8y ago

Could I use something like this to identify which of two or three people is speaking in an audio clip? Assume I can label several samples of each person's speech, then present an unlabeled sample for classification.

RileyJames 8y ago
I’m looking for something that can do this as well. Anything out already?
- flashman 8y ago
  I had a go of it by replacing the drum samples with voice samples (both 1-2 seconds and 3-5 seconds), then removing the features concerned with length and volume. Fiddled with the number of sub-sections per sample, and some of the random forest settings, but never consistently got higher than 77% accuracy between the four speakers. Maybe it would do better with two speakers.
psobot 8y ago
Yes! While the technique I used in this post is pretty simplistic, you could use a similar method to solve what's generically known as the [speaker recognition problem](https://en.wikipedia.org/wiki/Speaker_recognition https://en.wikipedia.org/wiki/Speaker_recognition). That said, this is an open problem within audio research, and there are solutions much more complex than what I have here. An example of one open source speaker recognition project is [bob.bio.spear](https://pypi.org/project/bob.bio.spear/ https://pypi.org/project/bob.bio.spear/), which I haven't tried, but looks promising.

TeMPOraL 8y ago

Great! Refreshing to see a ML post using some well-understood methods instead of throwing a random neural net from Kaggle at the problem... Tangential: > Is a given audio file a sample of a kick drum, snare drum, hi-hat, other percussion, or something else? (...) Humans have no trouble classifying these two sounds, as we’ve likely heard them tens of thousands of times before. Are people taught that in schools or something? Because I personally can't classify those sounds, don't know these names, and I'm not sure how I was supposed to learn them, other by playing in a band.

ludston 8y ago
> Are people taught that in schools or something? Because I personally can't classify those sounds, don't know these names, and I'm not sure how I was supposed to learn them, other by playing in a band. This is something that is taught at schools with a music program. (Although, not necessarily discretely). If you are someone whom has played music before, it is easy to forget what music sounded like before your ear was trained. (i.e. certain instruments and harmonies can be indistinguishable without training) Is it common to have never played on at a drum kit in your entire life?
- TeMPOraL 8y ago
  Thanks! > Is it common to have never played on at a drum kit in your entire life? I didn't, not on a real one at least. I know the sounds though, I spent ungodly amount of time playing on an electronic keyboard as a kid, where I could (and often would) change the sounds under keys to drums. However, nowhere (AFAIR) were the names of those sounds mentioned, and I'm not sure where I could encounter them.
  - jbenner-radham 8y ago
    In my anecdotal experience I only know the names of various percussion by sound because I was a drummer in band class at school. My friends weren’t taught that in the general education program though. That could possibly vary by region though.
8y ago