3 ms·

The assumption that this is possible come from our ability to isolate voices in a crowd by paying attention to one or more of them. However our ability to do so

The assumption that this is possible come from our ability to isolate voices in a crowd by paying attention to one or more of them. However our ability to do so rests on two important factors that don't exist in these datasets. 1. We have two ears to allow for sound localization and 2. The sounds we distinguish are colocated in space allowing us to use ambient information for disambiguation. This means that the problem being solved here is harder than the natural problem we have evolved and learned to solve. This is both impressive and possibly problematic. Some feature of training in a goal directed fashion in naturalistic environments could be essential for higher quality speaker isolation, or it might not matter at all. The multiplicity of models phenomenon tells us there are likely many solutions to this problem.

milesvp 6y ago
It’s porentially even harder than this. Not only do we have two ears, our ears are shaped to help determine where sounds are coming from, and we make constant movements to further help process audio. I remember years ago I played an FPS capture the flag type game with headphones, and, not everyone knows this, but the math works out that the same sound could come from in front or behind, so I developed a strong tendency to make small mouse movements rotating left and right to help me identify where footsteps were coming from. I didn’t even know I was doing it until I talked to an audio engineer who mentioned this problem, and next time I played, I realized how much I relied on it to avoid getting blindsided.