I had a go of it by replacing the drum samples with voice samples (both 1-2 seconds and 3-5 seconds), then removing the features concerned with length and volum

I had a go of it by replacing the drum samples with voice samples (both 1-2 seconds and 3-5 seconds), then removing the features concerned with length and volume. Fiddled with the number of sub-sections per sample, and some of the random forest settings, but never consistently got higher than 77% accuracy between the four speakers. Maybe it would do better with two speakers.