3 ms·

I'm not sure if ML can solve this problem in a decent way. For example, let's say most of your voice falls within a range of 120hz to 160hz and there is a guit

by nickjj 7y ago

I'm not sure if ML can solve this problem in a decent way. For example, let's say most of your voice falls within a range of 120hz to 160hz and there is a guitar playing in the background at various frequencies where a majority of it happens to match your voice's frequency. Now your goal is to cut out the guitar, leaving a pristine copy of your voice. How would ML or any strategy be able to differentiate a voice or guitar at 140hz? I deal with a similar problem when recording my voice for video courses in a non-treated room. I have to set up a noise gate to lower the volume of certain frequencies (background computer fans, etc.) until you can't hear them. They tend to give off a low frequency compared to a human voice, but the more loud the fan is relative to my voice requires setting the noise gate to a higher frequency. If you try to filter too much, then your voice gets cut off for those frequencies and it sounds horrible. It either cuts out entirely or you sound super dirty and metallic since your wave forms are all jaggy instead of smooth lines. I'm no ML expert but I think this problem would be insanely hard to solve in a way that's high quality. You can bend your voice's frequencies and the frequency of various instruments in a similar way. Then on top of that, there's going to be situations where the singer's voice is going to happen while you're talking. That'll be even harder to separate if you criss-cross frequencies. And on top of that, you might have 2-3 guitars, drums, keyboard, a singer and your voice all hitting the same frequencies. So much overlap! I'd love to hear the quality of a tool that claims it can remove only the songs out of any audio footage that may include your voice, background voices of other people talking as well as other ambient sound (water falling, people shuffling around, cars honking, etc.). If I were watching someone's vlog I would want to hear the atmosphere of the place. Making it completely mute except for their voice would be extremely weird and unsettling (and still crazy complicated to do I bet).

davinic 7y ago
Most productions use a recording of "room tone" to create a noise print, which is subtracted from the recorded audio to reduce noise. Software such as Adobe Audition have this built in. A known song could be used as a time-series with a similar effect.
- nickjj 7y ago
  > Most productions use a recording of "room tone" to create a noise print, which is subtracted from the recorded audio to reduce noise. Software such as Adobe Audition have this built in. Yes, this is what I used to do with REAPER (a digital audio workstation tool). It was even able to do this in soft real time (that's how I recorded my more recent courses) with only 10ms of delay. But from my understanding it does this by dampening or removing (depending on how aggressive you have it configured) the frequencies of your room or whatever sound you're trying to remove. A relatively low volume fan hum is pretty easy to remove without distorting the target audio, but if the frequencies start to overlap with varying degrees of volume the whole thing becomes a jumbled mess. To put this into perspective, the video at the top of https://diveintodocker.com/ https://diveintodocker.com/ used REAPER's noise reduction plugin to remove a pretty loud tower and power supply fan. Even with studio grade headphones at max volume you probably won't notice a hint of hissing or white noise when I'm not speaking, but if you have a trained ear there's a subtle clamping down on the lower end of my voice's range. 99.9999% of people won't notice it, but this is also a best case scenario for removing frequencies (a constant low hum at relatively low volume) that rarely interweaves with my voice's frequency. The whole game changes with multiple instruments and volumes that cut across your range.
  - sansnomme 7y ago
    You can use ML to optimize for the vocals e.g. like noise cancellation. The upside of this is that you will almost certainly get crystal clear voice output (since 1) you have the known music piece, may not be the exact copy but the signal is there 2) the voice is clear since the music is background music). The only downside is that clear voice is not the same as original voice and it may sound unnatural, but you won't have crackling or any of the other artefacts of traditional filter. This is purely in theory of course, I have not tested it yet.
    - nickjj 7y ago
      I'm really looking forward to seeing examples of how it performs in real situations. Even outside the scope of removing copyright material there's a lot of interesting use cases, like being able to split vocals or instruments from any track so you can practice layering in your own music as a musician (such as playing your own guitar solo over an existing song).

by nickjj 7y ago

davinic 7y ago

Most productions use a recording of "room tone" to create a noise print, which is subtracted from the recorded audio to reduce noise. Software such as Adobe Audition have this built in. A known song could be used as a time-series with a similar effect.

nickjj 7y ago
> Most productions use a recording of "room tone" to create a noise print, which is subtracted from the recorded audio to reduce noise. Software such as Adobe Audition have this built in. Yes, this is what I used to do with REAPER (a digital audio workstation tool). It was even able to do this in soft real time (that's how I recorded my more recent courses) with only 10ms of delay. But from my understanding it does this by dampening or removing (depending on how aggressive you have it configured) the frequencies of your room or whatever sound you're trying to remove. A relatively low volume fan hum is pretty easy to remove without distorting the target audio, but if the frequencies start to overlap with varying degrees of volume the whole thing becomes a jumbled mess. To put this into perspective, the video at the top of https://diveintodocker.com/ https://diveintodocker.com/ used REAPER's noise reduction plugin to remove a pretty loud tower and power supply fan. Even with studio grade headphones at max volume you probably won't notice a hint of hissing or white noise when I'm not speaking, but if you have a trained ear there's a subtle clamping down on the lower end of my voice's range. 99.9999% of people won't notice it, but this is also a best case scenario for removing frequencies (a constant low hum at relatively low volume) that rarely interweaves with my voice's frequency. The whole game changes with multiple instruments and volumes that cut across your range.
- sansnomme 7y ago
  You can use ML to optimize for the vocals e.g. like noise cancellation. The upside of this is that you will almost certainly get crystal clear voice output (since 1) you have the known music piece, may not be the exact copy but the signal is there 2) the voice is clear since the music is background music). The only downside is that clear voice is not the same as original voice and it may sound unnatural, but you won't have crackling or any of the other artefacts of traditional filter. This is purely in theory of course, I have not tested it yet.
  - nickjj 7y ago
    I'm really looking forward to seeing examples of how it performs in real situations. Even outside the scope of removing copyright material there's a lot of interesting use cases, like being able to split vocals or instruments from any track so you can practice layering in your own music as a musician (such as playing your own guitar solo over an existing song).