6 ms·

ISSE: An Interactive Sound Source Separation Editor

76 points by ashitlerferad 10y ago

braindead_in 10y ago
Amazing demo. Can it be used to separate two voices in a file?
colanderman 10y ago
Nice! I've been wanting a tool to do exactly this for years. Trying it out now. One question, what does the "training brush" do? Everything else is intuitive so far. Edit: Just tried this out on a song with acoustic guitar + vocals. Worked OK. There's lots of bleed between the tracks, but they are separated to a decent degree.
Anechoic 10y ago
Interesting that this has been out for 3 years and this is the first I've heard of it. I'll have to try out the tool to separate out sources from various recordings we make for noise control projects (for example, separating out the wheel/rail contribution and engine/motor contribution from DMU/EMU train recordings, or fan contribution and transformer contribution in electrical substation recordings).
- lighttower 10y ago
  Seeing stuff on sourceforge makes me immediately distrustful of the project. Like it feels neglected, forgotten.... Buggy. Is this generally true for projects you've seen?
  - taspeotis 10y ago
    > Like it feels neglected, forgotten I feel that way about most SourceForge projects, too. If you check out this project's history [1] the last commit was in 2013. Not that working software needs regular change, but as a rule of thumb it's nice to see recent commits. [1] https://sourceforge.net/p/isse/code/ci/master/tree/ https://sourceforge.net/p/isse/code/ci/master/tree/
  - ashitlerferad 10y ago
    For me, neglected stuff seems like an opportunity to contribute, not something to distrust. SourceForge also allows folks to take over abandoned projects.
  - rcthompson 10y ago
    The stronger indicator for me is seeing that the software is written for someone's thesis. Typically that means that development stopped on graduation.
- JorgeGT 10y ago
  What is your current approach if you can answer? Some kind of spatial beamforming or you must use a given recorded measurement and decompose by frequency content as best as you can? I guess this kind of machine learning that not uses total frequency spectrum but the time-domain evolution of short-time Fourier transforms (as in the spectrogram he is painting over in the demo) has a lot of potential for acoustic source separation. For instance training the machine to recognize the particular time-frequency signature of human voice and automatically filter it from a song.
  - Anechoic 10y ago
    Microphone array (beamforming) or sound intensity. But since those approaches typically require expensive hardware, the current approach is basically "we don't" unless the frequency characteristics are clearly distinct and we can use traditional equalization processes. Machine learning has potential (and a lot of people have tried) but it's only as good at it's training.
ashitlerferad 10y ago
Folks interested in this might be interested in ManyEars and the associated hardware projects: http://manyears.sourceforge.net/ http://manyears.sourceforge.net/
yoo1I 10y ago
This is pretty cool! A couple if years ago, I saw a short documentary about some commercial software that was able to take the recording of an orchestra, separate the instruments and allow the user to edit the pitch and volume of an individual note an instrument was playing, but I can't seem to find it anymore. Might anyone here know about this software ?
- charlesism 10y ago
  You're thinking of Melodyne, by Celemony Software. It's amazing. https://www.youtube.com/watch?v=u573PyXo-pY https://www.youtube.com/watch?v=u573PyXo-pY
  - yoo1I 10y ago
    I recognized the beard and glasses in the first frame of the video. Thanks!
  - yxlx 10y ago
    That was an interesting video, thanks for linking it. Two books are mentioned in the video, both of which I would like to read; - "History of Harmonic Pythagoreanism" by Rudolf Haase - "The Harmony of the World" by Johannes Kepler The original title of the book by Haase, I found by jumping to the mark where it was mentioned in the original german language video, was Geschichte des Harmonikalen Pythagoreismus. Unfortunatelly, I have not been successfull in finding an English translation of the book. I could read the German version but would only understand a few select parts so I would much prefer to read it in English. Does anybody know of an English version of the book?
- anonymous123458 10y ago
  It's also similar to Roland's R-MIX software: http://www.rolandus.com/products/r-mix/ http://www.rolandus.com/products/r-mix/
incepted 10y ago
Ugh... sourceforge.net? No, thanks.
- ashitlerferad 10y ago
  sourceforge isn't like that any more: https://sourceforge.net/blog/sourceforge-acquisition-and-future-plans/ https://sourceforge.net/blog/sourceforge-acquisition-and-fut...
riprowan 10y ago
First off - what a great idea! How am I just finding out about this? As an audio engineer, I know the first thing I want to use this on: a drum kit. Separate the cymbals from the skins!! Separate out the room sound from the direct sounds!! Get that pesky hi-hat out of the snare track! Secondly - why only frequencies? Why not dynamics, stereo position, phase, and transient response? If I could switch the screen to show those things instead of frequency, I could isolate all kinds of other phenomena. For example, isolate only the transients and turn them up/down, EQ them, add reverb to them, etc (drool). Non-engineers have no idea how potent this could be. Or isolate only the instruments appearing at 30% off-center left?
- deleted 10y ago
  [deleted]
- Mathnerd314 10y ago
  There seems to be a deafening silence with audio stuff. Probably because there's no money in it, or rather the money is all held by the recording studios / big distributors, who have no incentive to share their findings. Also Google seems to rank music-software search results really low. Other cool projects that I only found after intensive search: http://photosounder.com/ http://photosounder.com/ http://www.soundhelix.com/ http://www.soundhelix.com/ http://www.cerlsoundgroup.org/Loris/ http://www.cerlsoundgroup.org/Loris/ http://www.klingbeil.com/spear/ http://www.klingbeil.com/spear/ The guy who wrote this is at Apple now, probably working on Logic Pro; the latest versions have some note-editing features.

76 points by ashitlerferad 10y ago

Amazing demo. Can it be used to separate two voices in a file?

Nice! I've been wanting a tool to do exactly this for years. Trying it out now. One question, what does the "training brush" do? Everything else is intuitive so far. Edit: Just tried this out on a song with acoustic guitar + vocals. Worked OK. There's lots of bleed between the tracks, but they are separated to a decent degree.

Anechoic 10y ago

Interesting that this has been out for 3 years and this is the first I've heard of it. I'll have to try out the tool to separate out sources from various recordings we make for noise control projects (for example, separating out the wheel/rail contribution and engine/motor contribution from DMU/EMU train recordings, or fan contribution and transformer contribution in electrical substation recordings).

lighttower 10y ago
Seeing stuff on sourceforge makes me immediately distrustful of the project. Like it feels neglected, forgotten.... Buggy. Is this generally true for projects you've seen?
- taspeotis 10y ago
  > Like it feels neglected, forgotten I feel that way about most SourceForge projects, too. If you check out this project's history [1] the last commit was in 2013. Not that working software needs regular change, but as a rule of thumb it's nice to see recent commits. [1] https://sourceforge.net/p/isse/code/ci/master/tree/ https://sourceforge.net/p/isse/code/ci/master/tree/
- ashitlerferad 10y ago
  For me, neglected stuff seems like an opportunity to contribute, not something to distrust. SourceForge also allows folks to take over abandoned projects.
- rcthompson 10y ago
  The stronger indicator for me is seeing that the software is written for someone's thesis. Typically that means that development stopped on graduation.
JorgeGT 10y ago
What is your current approach if you can answer? Some kind of spatial beamforming or you must use a given recorded measurement and decompose by frequency content as best as you can? I guess this kind of machine learning that not uses total frequency spectrum but the time-domain evolution of short-time Fourier transforms (as in the spectrogram he is painting over in the demo) has a lot of potential for acoustic source separation. For instance training the machine to recognize the particular time-frequency signature of human voice and automatically filter it from a song.
- Anechoic 10y ago
  Microphone array (beamforming) or sound intensity. But since those approaches typically require expensive hardware, the current approach is basically "we don't" unless the frequency characteristics are clearly distinct and we can use traditional equalization processes. Machine learning has potential (and a lot of people have tried) but it's only as good at it's training.

ashitlerferad 10y ago

Folks interested in this might be interested in ManyEars and the associated hardware projects: http://manyears.sourceforge.net/ http://manyears.sourceforge.net/

yoo1I 10y ago

This is pretty cool! A couple if years ago, I saw a short documentary about some commercial software that was able to take the recording of an orchestra, separate the instruments and allow the user to edit the pitch and volume of an individual note an instrument was playing, but I can't seem to find it anymore. Might anyone here know about this software ?

charlesism 10y ago
You're thinking of Melodyne, by Celemony Software. It's amazing. https://www.youtube.com/watch?v=u573PyXo-pY https://www.youtube.com/watch?v=u573PyXo-pY
- yoo1I 10y ago
  I recognized the beard and glasses in the first frame of the video. Thanks!
- yxlx 10y ago
  That was an interesting video, thanks for linking it. Two books are mentioned in the video, both of which I would like to read; - "History of Harmonic Pythagoreanism" by Rudolf Haase - "The Harmony of the World" by Johannes Kepler The original title of the book by Haase, I found by jumping to the mark where it was mentioned in the original german language video, was Geschichte des Harmonikalen Pythagoreismus. Unfortunatelly, I have not been successfull in finding an English translation of the book. I could read the German version but would only understand a few select parts so I would much prefer to read it in English. Does anybody know of an English version of the book?
anonymous123458 10y ago
It's also similar to Roland's R-MIX software: http://www.rolandus.com/products/r-mix/ http://www.rolandus.com/products/r-mix/

incepted 10y ago

Ugh... sourceforge.net? No, thanks.

ashitlerferad 10y ago
sourceforge isn't like that any more: https://sourceforge.net/blog/sourceforge-acquisition-and-future-plans/ https://sourceforge.net/blog/sourceforge-acquisition-and-fut...

riprowan 10y ago

First off - what a great idea! How am I just finding out about this? As an audio engineer, I know the first thing I want to use this on: a drum kit. Separate the cymbals from the skins!! Separate out the room sound from the direct sounds!! Get that pesky hi-hat out of the snare track! Secondly - why only frequencies? Why not dynamics, stereo position, phase, and transient response? If I could switch the screen to show those things instead of frequency, I could isolate all kinds of other phenomena. For example, isolate only the transients and turn them up/down, EQ them, add reverb to them, etc (drool). Non-engineers have no idea how potent this could be. Or isolate only the instruments appearing at 30% off-center left?

deleted 10y ago
[deleted]
Mathnerd314 10y ago
There seems to be a deafening silence with audio stuff. Probably because there's no money in it, or rather the money is all held by the recording studios / big distributors, who have no incentive to share their findings. Also Google seems to rank music-software search results really low. Other cool projects that I only found after intensive search: http://photosounder.com/ http://photosounder.com/ http://www.soundhelix.com/ http://www.soundhelix.com/ http://www.cerlsoundgroup.org/Loris/ http://www.cerlsoundgroup.org/Loris/ http://www.klingbeil.com/spear/ http://www.klingbeil.com/spear/ The guy who wrote this is at Apple now, probably working on Logic Pro; the latest versions have some note-editing features.