3 ms·
Automatically classifying the content of sound files using ML
- mnky9800n 3y agoThis was a bit disappointing because I was hoping this would be an unsupervised learning example. This is not the author's fault. However there is a large need for unsupervised acoustic event detection and classification. Current methods like lta/sta, match filter, etc. Are not good enough anymore. I will be happy to see this kind of technology grow or be pointed in the right direction.
- msephton 3y agoSorry, can you explain what you were hoping for? Do you want to be able to classify sounds in a live audio feed? That should be possible relatively easy, by recording a feed and then every so often cutting off the most recent chunk for processing. If I've misunderstood, please explain further.
- ladberg 3y agoI think they were hoping that the off-the-shelf model the author used was a novel technique trained on unlabeled data, not a dataset labeled by humans.
- mnky9800n 3y agoYes current methods are either too precise or too coarse. They either label everything or they label nothing. I'm not looking for something with high precision. Just something to help me do the analysis.
- jononor 3y agoCan you provide some more details on your use case? In general a powerful approach is to use a neural network to extract vector embeddings and then do simple models on top. The neural network could for example be a pretrained audio classifier (like YAMNet mentioned in article), or a model trained with self-supervised learning on representative (unlabeled) data. A small model for unsupervised event detection / segmentation could be agglomerative clustering. A data-efficient supervised approach could be K Nearest Neighbors.
- mnky9800n 3y agoAs an example, I have 9 months of hydrophone cable data that has a large number of sounds that occur across a large variety of time lengths and frequencies. It would be nice to have a method that highlights certain signals given specific meta data. Tbh, what I think I want is a pipe dream.
- jononor 3y agoIf you are willing to label 100 instances of each event type, then the approaches suggested above should work. And half of the data should be used to quantify how well it works.
- akomtu 3y agoWhat kind of metadata?
- gopher_space 3y agoI think some of the folks working with cetaceans might be able to point you towards interesting areas. I'm aware of work on processing petabytes of highest-res hydrophone data and I'm pretty sure they're tagging things that look like new clicktrains in some manner. Does this sound appropriate? As an aside, I've been kicking around the thought of what a software resonator might look like and where the analogy would work. I feel like there's a really efficient filter you could pull out of the concept.
- gnulinux 3y agoOne problem I wasn't able to solve very satisfyingly is music -> midi. Even with solo instruments, the output data is a hot jumbled mess with all kinds of errors from pitch to rhythm to dynamics. Once anything more than one timbre is involved, or e.g. an orchestra, it seems completely impossible to solve it with current tools since the model needs to be smart enough to tell apart fundamental frequencies with multiple timbres.
- jononor 3y agoPolyphonic / multi-track music transcription is a very challenging tasks and still an open area of research. Plenty of progress made the last 5 years, but still not at a very practical level. Source separation and single/lead instrument transcription has gotten pretty good though. So if the music fits with the 5-stem split of say Spleeter it can do an OK job
- egberts1 3y agoSend MOAR metal opera my way!