4 ms·

A friend of mine is a violin teacher, and he has been asking if it’s possible to build an app that would recognize the notes being played from a microphone, in

by yalok 5y ago

A friend of mine is a violin teacher, and he has been asking if it’s possible to build an app that would recognize the notes being played from a microphone, in realtime, and overlay it over the expected music sheet, to help kids see where might have missed a note… Which of the existing string music transcription libraries would fit the bill?

chrismorgan 5y ago
In the field of speech, you have the concept of forced alignment, aligning an audio recording with a transcript by way of text-to-speech and pattern matching, which can certainly be used to detect some sorts of errors consistently and various other errors with some probability. (I’m not up to the point of doing this, but I’m early in making software for audiobook recording and plan to see what I can manage in error detection eventually. Unfortunately from what I’ve seen just with a little bit of casual playing around, small errors are not generally well identified because there will be too many false positives.) You could probably adapt a lot of that stuff to music without too much trouble, though it’d certainly require changes and integration effort. My vague feeling is that this direction, with MIDI rendering and pattern matching, would probably be surprisingly effective and I suspect better than trying to more directly match the recording to the score with the help of just Fourier transformations, or converting the audio to a score with something like this and comparing them, though even with the forced alignment adaptations I imagine you’d be changing it to use mostly Fourier transformations for feature detection. (I’m a good programmer in general, but still decidedly a layman in audio matters. Take my thoughts here with a grain of salt.)
mushishi 5y ago
This is what Yousician does for singing (and other analog instruments) but unfortunately not for the violin: https://yousician.com/singing https://yousician.com/singing Haven't tried it but maybe the following app could help? > Trala uses signal processing, groundbreaking technology that analyzes the sound of your violin and gives you instant feedback on pitch and rhythm every time you practice. When you play the wrong notes, we’ll help you get back on track. https://www.trala.com/ https://www.trala.com/ (I just remembered it being recommended somewhere.)

by yalok 5y ago

chrismorgan 5y ago

In the field of speech, you have the concept of forced alignment, aligning an audio recording with a transcript by way of text-to-speech and pattern matching, which can certainly be used to detect some sorts of errors consistently and various other errors with some probability. (I’m not up to the point of doing this, but I’m early in making software for audiobook recording and plan to see what I can manage in error detection eventually. Unfortunately from what I’ve seen just with a little bit of casual playing around, small errors are not generally well identified because there will be too many false positives.) You could probably adapt a lot of that stuff to music without too much trouble, though it’d certainly require changes and integration effort. My vague feeling is that this direction, with MIDI rendering and pattern matching, would probably be surprisingly effective and I suspect better than trying to more directly match the recording to the score with the help of just Fourier transformations, or converting the audio to a score with something like this and comparing them, though even with the forced alignment adaptations I imagine you’d be changing it to use mostly Fourier transformations for feature detection. (I’m a good programmer in general, but still decidedly a layman in audio matters. Take my thoughts here with a grain of salt.)

mushishi 5y ago

This is what Yousician does for singing (and other analog instruments) but unfortunately not for the violin: https://yousician.com/singing https://yousician.com/singing Haven't tried it but maybe the following app could help? > Trala uses signal processing, groundbreaking technology that analyzes the sound of your violin and gives you instant feedback on pitch and rhythm every time you practice. When you play the wrong notes, we’ll help you get back on track. https://www.trala.com/ https://www.trala.com/ (I just remembered it being recommended somewhere.)