4 ms·

You don't need a neural network for this - after song identification, you might be able to stream the right song at the right location, then simply play the son

You don't need a neural network for this - after song identification, you might be able to stream the right song at the right location, then simply play the song "inverted" - you'll increase background noise but the song won't be identifiable. It's a very simple DSP operation after song identification. It sounds like a lot of fun actually. The front end location identification might not have an extant solution, but post identification it should be doable with standard tricks. This is the cyberpunk future we were waiting for. Who wants to do some DSP?! If anyone wants to work together on this, we absolutely should. This is a fun problem with a very, very cyberpunk vibe. Plus it's real DSP work!

me_me_me 6y ago
That's what I was thinking too. It doesnt even have to be perfect. All it needs to do is mask/distort music to make the algo matching content id not identify it. Hmmm.... would playing another song at the same time not have similar effect? This might be fun weekend project :)
anaerobicover 6y ago
This won't magically make other audio sources that were present when the recording was made easier to hear, though. Once the recording is made, they're not independent any more. They'll still be masked, maybe even more.
- analognoise 6y ago
  No, but it will preserve them online for later filtering/post processing.
  - anaerobicover 6y ago
    Maybe, depending on the details of your scheme, I guess? If you subtract data, that data is gone, and there's no separate stream of data in the recording for the cops' voices vs. the phone.
    - analognoise 6y ago
      We just "play" the song - so it's a spotify call. Ideally anything not aligned with the song meant to induce takedown and the inverted playing of it will still be maintained. To clarify, we don't want to subtract the data from the feed, we can just nullify it near the microphone. I think it would be harder to filter on an instagram feed directly, not sure how their api looks or if it's possible?
      - anaerobicover 6y ago
        I see, you're saying play the inverse at the recording time. So I misunderstood, I thought you were meaning doing it to the recording. This is also complicated, because you need to factor in the relative locations of the original source, your source, and the pickup to make sure the phase is correct.
- defterGoose 6y ago
  Level-wise they are separate though. It should be easy enough to find the proper gain for the reference recording (how loud does it sound in the video?), and then subtracted, leaving any residual energy in that band from desired sources. Yeah, there will be some gain-matching and spatial artifacts, but probably not enough to make things (voices for instance) unintelligible. Thats actually where any ML component would naturally sit. It would try to basically determine a "close-enough" deconvolution envelope for the reference recording.
bmicraft 6y ago
I don't think this is realistically achievable when you have all kinds of distortion and echoes. You'd also need a way to identify the exact position on the song and correct that if it was wrong because it was in the refrain.
- analognoise 6y ago
  I was thinking about this - a convolution filter with a long enough window would have a pretty good chance of identifying the right refrain location. You're right about echoes, but it doesn't need to be perfect - the echoes are already probably below the detection floor for the takedown filter. Also: If you identify and match up with a refrain, you'd still nullify that refrain. You'd just have to keep on your toes and move to the right location once you realized there was a mismatch. This is a classic cat/mouse "warring filters" problem!
diggan 6y ago
Isn't this how balanced audio cables work effectively? Have two signals, one hot and one cold, invert them against each other and noise would be cancelled out. Replace noise with the song that got identified, and you can cancel it out.
- pixelface 6y ago
  I think if you time your destructive interference to the source of the audio perfectly and cancel out the sound, you haven't gotten the whole picture. As sound travels away from the source it reflects off surfaces and some of those reflections also reach the microphone. Those sounds have traveled a different distance than the original source and are no longer timed with your interfering element. Imagine holding your hand to block your direct view of a person in a house of mirrors but you can still see them in the mirrors. On top of that these reflections won't contain identical audio as the surfaces will absorb different spectral content, so now even if you had infinite perfectly timed anti-signals for each reflection, your anti-signal has spectral content that is missing from the incoming reflected audio and now your anti-signal is a signal unto itself. In order to create a signal that perfectly cancels out all of the copyrighted audio you would have to have knowledge about the form and materials of the physical environment as well as the position of the microphone within that environment.
  - diggan 6y ago
    This was already mentioned and answered to before here: https://news.ycombinator.com/item?id=26115933 https://news.ycombinator.com/item?id=26115933 In the end, the solution does not have to be perfect. Just have to hide the song well enough so the content ID cannot be made. As you say, the sound that will manage to bypass the "inverted balanced-like filter" will be distorted and hopefully content ID is not accurate enough to recognize it.
dahart 6y ago
Yeah it does seem technically within reach and potentially solvable. I was thinking of including song identification when I mentioned an NN as a potential solution. I’m also assuming that there a bunch of other serious complications like resampling and clock skew between the two streams, a changing frequency response over time, etc. You can imagine that if there are audio dropouts or song pauses in the video you’re trying to fix, then naively subtracting the source song will result in accidentally playing the copyrighted song instead of removing it. And it might be trickier if people sing along to the song, etc. I think this situation is quite a bit tougher than the standard trick people use to remove ambient audio by recording with a 2nd mic further away and subtracting it.

me_me_me 6y ago

That's what I was thinking too. It doesnt even have to be perfect. All it needs to do is mask/distort music to make the algo matching content id not identify it. Hmmm.... would playing another song at the same time not have similar effect? This might be fun weekend project :)

anaerobicover 6y ago

This won't magically make other audio sources that were present when the recording was made easier to hear, though. Once the recording is made, they're not independent any more. They'll still be masked, maybe even more.

analognoise 6y ago
No, but it will preserve them online for later filtering/post processing.
- anaerobicover 6y ago
  Maybe, depending on the details of your scheme, I guess? If you subtract data, that data is gone, and there's no separate stream of data in the recording for the cops' voices vs. the phone.
  - analognoise 6y ago
    We just "play" the song - so it's a spotify call. Ideally anything not aligned with the song meant to induce takedown and the inverted playing of it will still be maintained. To clarify, we don't want to subtract the data from the feed, we can just nullify it near the microphone. I think it would be harder to filter on an instagram feed directly, not sure how their api looks or if it's possible?
    - anaerobicover 6y ago
      I see, you're saying play the inverse at the recording time. So I misunderstood, I thought you were meaning doing it to the recording. This is also complicated, because you need to factor in the relative locations of the original source, your source, and the pickup to make sure the phase is correct.
defterGoose 6y ago
Level-wise they are separate though. It should be easy enough to find the proper gain for the reference recording (how loud does it sound in the video?), and then subtracted, leaving any residual energy in that band from desired sources. Yeah, there will be some gain-matching and spatial artifacts, but probably not enough to make things (voices for instance) unintelligible. Thats actually where any ML component would naturally sit. It would try to basically determine a "close-enough" deconvolution envelope for the reference recording.

bmicraft 6y ago

I don't think this is realistically achievable when you have all kinds of distortion and echoes. You'd also need a way to identify the exact position on the song and correct that if it was wrong because it was in the refrain.

analognoise 6y ago
I was thinking about this - a convolution filter with a long enough window would have a pretty good chance of identifying the right refrain location. You're right about echoes, but it doesn't need to be perfect - the echoes are already probably below the detection floor for the takedown filter. Also: If you identify and match up with a refrain, you'd still nullify that refrain. You'd just have to keep on your toes and move to the right location once you realized there was a mismatch. This is a classic cat/mouse "warring filters" problem!

diggan 6y ago

Isn't this how balanced audio cables work effectively? Have two signals, one hot and one cold, invert them against each other and noise would be cancelled out. Replace noise with the song that got identified, and you can cancel it out.

pixelface 6y ago
I think if you time your destructive interference to the source of the audio perfectly and cancel out the sound, you haven't gotten the whole picture. As sound travels away from the source it reflects off surfaces and some of those reflections also reach the microphone. Those sounds have traveled a different distance than the original source and are no longer timed with your interfering element. Imagine holding your hand to block your direct view of a person in a house of mirrors but you can still see them in the mirrors. On top of that these reflections won't contain identical audio as the surfaces will absorb different spectral content, so now even if you had infinite perfectly timed anti-signals for each reflection, your anti-signal has spectral content that is missing from the incoming reflected audio and now your anti-signal is a signal unto itself. In order to create a signal that perfectly cancels out all of the copyrighted audio you would have to have knowledge about the form and materials of the physical environment as well as the position of the microphone within that environment.
- diggan 6y ago
  This was already mentioned and answered to before here: https://news.ycombinator.com/item?id=26115933 https://news.ycombinator.com/item?id=26115933 In the end, the solution does not have to be perfect. Just have to hide the song well enough so the content ID cannot be made. As you say, the sound that will manage to bypass the "inverted balanced-like filter" will be distorted and hopefully content ID is not accurate enough to recognize it.

dahart 6y ago

Yeah it does seem technically within reach and potentially solvable. I was thinking of including song identification when I mentioned an NN as a potential solution. I’m also assuming that there a bunch of other serious complications like resampling and clock skew between the two streams, a changing frequency response over time, etc. You can imagine that if there are audio dropouts or song pauses in the video you’re trying to fix, then naively subtracting the source song will result in accidentally playing the copyrighted song instead of removing it. And it might be trickier if people sing along to the song, etc. I think this situation is quite a bit tougher than the standard trick people use to remove ambient audio by recording with a 2nd mic further away and subtracting it.