4 ms·

Does anyone know of anything similar which can extract human voice out of a video which has other noises including fans, people coughing, electric generator etc

by kr4 8y ago

Does anyone know of anything similar which can extract human voice out of a video which has other noises including fans, people coughing, electric generator etc?

Jarwain 8y ago
I'd imagine one could apply this network to a video of someone speaking
- augbog 8y ago
  Yup I believe they announced this at Google I/O Keynote this year actually -- they mentioned that while it might not be enough with just the audio, looking at mouth gestures can give AI enough to know what might be said by whom. https://www.youtube.com/watch?v=ogfYd705cRs&t=7m0s https://www.youtube.com/watch?v=ogfYd705cRs&t=7m0s
  - pbhjpbhj 8y ago
    Reflecting on your last phrase: I was watching Esports the other day, the player was talking with a very loose mouth and I wondered if he was avoiding being lip-read. I imagine a lip reading AI would get wide usage. Managers will be wearing face masks to hide their lips. (It's probably doable now to listen with a spy-mic, but it's obvious in a way that using a normal video camera isn't).
    - laszlokorte 8y ago
      Julia Probst is a german deaf blogger [1] who is famous [2] for lipreading tactical commands given by soccer coaches to their team during a match and posting them on twitter. She is even hired by sport channels on TV to provide the commentator with insight information. [1] https://twitter.com/einaugenschmaus https://twitter.com/einaugenschmaus [2] http://www.sueddeutsche.de/sport/lippenlesen-im-fussball-die-geheimnisverraeter-1.2215827 http://www.sueddeutsche.de/sport/lippenlesen-im-fussball-die...
8bitsrule 8y ago
The human ear. From what I can hear, most of what's being done is selective filtering. Leaving behind some artifacts and much less fidelity. Similar feats by human engineers have been around for decades. Analog filters/Fourier can be used to put frequency segments into channels (e.g. vocoding), then keeping the useful channels.

by kr4 8y ago

Does anyone know of anything similar which can extract human voice out of a video which has other noises including fans, people coughing, electric generator etc?

Jarwain 8y ago

I'd imagine one could apply this network to a video of someone speaking

augbog 8y ago
Yup I believe they announced this at Google I/O Keynote this year actually -- they mentioned that while it might not be enough with just the audio, looking at mouth gestures can give AI enough to know what might be said by whom. https://www.youtube.com/watch?v=ogfYd705cRs&t=7m0s https://www.youtube.com/watch?v=ogfYd705cRs&t=7m0s
- pbhjpbhj 8y ago
  Reflecting on your last phrase: I was watching Esports the other day, the player was talking with a very loose mouth and I wondered if he was avoiding being lip-read. I imagine a lip reading AI would get wide usage. Managers will be wearing face masks to hide their lips. (It's probably doable now to listen with a spy-mic, but it's obvious in a way that using a normal video camera isn't).
  - laszlokorte 8y ago
    Julia Probst is a german deaf blogger [1] who is famous [2] for lipreading tactical commands given by soccer coaches to their team during a match and posting them on twitter. She is even hired by sport channels on TV to provide the commentator with insight information. [1] https://twitter.com/einaugenschmaus https://twitter.com/einaugenschmaus [2] http://www.sueddeutsche.de/sport/lippenlesen-im-fussball-die-geheimnisverraeter-1.2215827 http://www.sueddeutsche.de/sport/lippenlesen-im-fussball-die...

8bitsrule 8y ago

The human ear. From what I can hear, most of what's being done is selective filtering. Leaving behind some artifacts and much less fidelity. Similar feats by human engineers have been around for decades. Analog filters/Fourier can be used to put frequency segments into channels (e.g. vocoding), then keeping the useful channels.