5 ms·

This is an awesome project, but it seems it was done without reference to academic literature on source separation. In fact, people have been doing audio source

by btown 8y ago

This is an awesome project, but it seems it was done without reference to academic literature on source separation. In fact, people have been doing audio source separation for years with neural networks. For instance, Eric Humphrey at Spotify Music Understanding Group describes using a U-Net architecture here: https://medium.com/this-week-in-machine-learning-ai/separating-vocals-in-recorded-music-at-spotify-with-eric-humphrey-51c2f85d1451 https://medium.com/this-week-in-machine-learning-ai/separati... - paper at http://openaccess.city.ac.uk/19289/1/7bb8d1600fba70dd79408775cd0c37a4ff62.pdf http://openaccess.city.ac.uk/19289/1/7bb8d1600fba70dd7940877... They compare their performance to the widely-cited state of the art Chimera model (Luo 2017): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5791533/#R24 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5791533/#R24 with examples at http://naplab.ee.columbia.edu/ivs.html http://naplab.ee.columbia.edu/ivs.html - from the examples, there's significantly less distortion than OP. Not to discourage OP from doing first-principles research at all! But it's often useful to engage with the larger community and know what's succeeded and failed in the past. This is a problem domain where progress could change the entire creative landscape around derivative works ("mashups" and the like), and interested researchers could do well to look towards collaboration rather than reinventing each others' wheels. EDIT: The SANE conference has talks by Humphrey and many others available online: https://www.youtube.com/channel/UCsdxfneC1EdPorDUq9_XUJA/videos https://www.youtube.com/channel/UCsdxfneC1EdPorDUq9_XUJA/vid...

deleted 8y ago
[deleted]
musicale 8y ago
People have also been doing audio source separation effectively for years without neural networks.
- calf 8y ago
  It's interesting cause I have a recording of human voices plus a background TV show that was too loud; I've looked around for something that would be able to separate the two but I haven't found a straightforward solution. For example if you Google then FASST is one of the ones that come up, but it's a whole framework and in order to use it you'd have to learn the research yourself; much of these software is not geared for end users.
  - lightedman 8y ago
    Learn how to do waveform inversions - if you have a stereo signal, anything not fully-centered will come through better while the rest is cut out. You can then take that, invert it, and play it back with the original, cutting out that noise and keeping the fully-centered things like vocals present. This is how I play guitar to my favorite songs on my computer.
    - proctor 8y ago
      this sounds interesting. could you elaborate a bit? it is unclear if you are inverting once, or twice. "if you have a stereo signal, anything not fully-centered will come through better while the rest is cut out" -- is this before or after an inversion?
      - thibauts 8y ago
        Of stereo tracks L and R, you invert R and add it to L, effectively canceling anything centered. This usually removes voices. If you subtract (invert then add) the result from the original L and R tracks you get centered sounds only. Results range from perfect to not effective at all depending on the songs you apply it to.
        lightedman 8y ago
        This is correct. It is a shame there are no music players that can do this natively. I have to do every track I wish to play along with by hand in Audacity (or if I'm using my Win2K laptop, Cool Edit Pro 2000.) Alternatively (in theory, anyways,) you can do a spectral analysis, create a selection/range of frequencies to invert, and that can usually handle isolating individual instruments regardless of stereo isolation as long as the other instrument octave ranges do not overlap with the one you're isolating. I'd love a music player that could do that.
  - jdietrich 8y ago
    iZotope RX. https://www.izotope.com/en/products/repair-and-edit/rx.html https://www.izotope.com/en/products/repair-and-edit/rx.html
wodenokoto 8y ago
> but it seems it was done without reference to academic literature on source separation Sums up towards data science pretty well.

by btown 8y ago

deleted 8y ago

[deleted]

musicale 8y ago

People have also been doing audio source separation effectively for years without neural networks.

calf 8y ago
It's interesting cause I have a recording of human voices plus a background TV show that was too loud; I've looked around for something that would be able to separate the two but I haven't found a straightforward solution. For example if you Google then FASST is one of the ones that come up, but it's a whole framework and in order to use it you'd have to learn the research yourself; much of these software is not geared for end users.
- lightedman 8y ago
  Learn how to do waveform inversions - if you have a stereo signal, anything not fully-centered will come through better while the rest is cut out. You can then take that, invert it, and play it back with the original, cutting out that noise and keeping the fully-centered things like vocals present. This is how I play guitar to my favorite songs on my computer.
  - proctor 8y ago
    this sounds interesting. could you elaborate a bit? it is unclear if you are inverting once, or twice. "if you have a stereo signal, anything not fully-centered will come through better while the rest is cut out" -- is this before or after an inversion?
    - thibauts 8y ago
      Of stereo tracks L and R, you invert R and add it to L, effectively canceling anything centered. This usually removes voices. If you subtract (invert then add) the result from the original L and R tracks you get centered sounds only. Results range from perfect to not effective at all depending on the songs you apply it to.
      - lightedman 8y ago
        This is correct. It is a shame there are no music players that can do this natively. I have to do every track I wish to play along with by hand in Audacity (or if I'm using my Win2K laptop, Cool Edit Pro 2000.) Alternatively (in theory, anyways,) you can do a spectral analysis, create a selection/range of frequencies to invert, and that can usually handle isolating individual instruments regardless of stereo isolation as long as the other instrument octave ranges do not overlap with the one you're isolating. I'd love a music player that could do that.
- jdietrich 8y ago
  iZotope RX. https://www.izotope.com/en/products/repair-and-edit/rx.html https://www.izotope.com/en/products/repair-and-edit/rx.html

wodenokoto 8y ago

> but it seems it was done without reference to academic literature on source separation Sums up towards data science pretty well.