5 ms·

For your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify h

For your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify his music under its Creative Commons license. First, the original: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-your-brains/original.mp3 https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Now the derived stems: Vocals: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-your-brains/stems/vocals.mp3 https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Accompaniment: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-your-brains/stems/accompaniment.mp3 https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Note: I'm not affiliated with this project or Mr. Coulton. I just think this is a cool project and wanted to share.

M4v3R 7y ago
Wow, I've listened to several attempts at this over the years, but this one is waaay better than anything I've heard. It's almost perfect.
jesuslop 7y ago
Holly cow! The separation is sort of perfect! Thanks for the demo.
- eps 7y ago
  "Indistinguishable from magic."
deleted 7y ago
[deleted]
codedokode 7y ago
While it's a great technology, the result sounds somewhat robotic. On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder, something is missing. The voice contains pieces of strumming sound. Guitar also sounds "blurred", as if someone cut an object from the picture and blurred the cut to make it less visible. Clap sound is distorted, on the original recording it sounds the same, but after separation it sounds different every time, as if it was filtered or compressed with low bitrate. It is amazing how the ear manages to distinguish all the sounds without distortion.
- MattRix 7y ago
  It's easy to pick up on lots of little things, but this is still extremely impressive. It's also more than good enough for people who want to practice singing/playing over the song by themselves. I think the next step would be to train a network that can un-robotify songs and then run it on this.
- uryga 7y ago
  > On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder funny, the original recording seemed kind of robotic to me! maybe not robotic, but like it's been filtered somehow. but that might just be my not-so-great headphones
  - mwcampbell 7y ago
    The original lead vocal is definitely processed. You can hear that processing clearly if you apply the classic vocal removal (really, center removal) effect. (My favorite implementation of that is the Center Cut DSP for foobar2000.)
- nickjj 7y ago
  That roboticness is because there's overlap in frequencies between the voice and the instruments. I have no idea how this tool splits them up at the implementation level but I imagine it tries to split it up based on frequencies and when it lifts out the voice, it's cutting out a ton of frequencies that would normally be in your voice so now it sounds very unnatural, blocky and metallic. With studio quality headphones I can notice a massive difference for the worse between the original and separated vocal track. It reminds me of when I turned on a noise gate too high when recording audio for my courses. That noise gate clamps down on certain frequencies to help eliminate room noise but it also dampens or removes natural frequencies that occur on the low end of an average male voice. It gives that same very jagged sounding audio waveform.
  - codedokode 7y ago
    I don't argue that it's a great technology, but some of the neighbour commenters wrote things like "perfect", and to me it doesn't sound like "perfect" yet. For example, in the beginning, if you listen to the phrase "from the office ...", in original recording it sounds smooth, like a single phrase (and the voice is warm and pleasant), but in the separated track it sounds like it is synthesized from pieces that are not properly connected, like vocaloid songs. It sounds little harsher. Transitions sound unnatural. And the phrase "heya Tom" in the separated track is split into "heya" and "Tom" with some unnatural sound between (or maybe in the beginning of "Tom"). It is like transitions you can hear in vocaloid tracks. Or the kind of artifacts you get if you over-compress an MP3 file. And "it's good to see you" part also sounds robotic. Maybe it's losing some of harmonics, but in a different way for different syllables and they don't sound like a single phrase anymore.
noja 7y ago
So Jonathan Coulton is now the new Suzanne Vega?
- BrentOzar 7y ago
  > So Jonathan Coulton is now the new Suzanne Vega? Bravo. For people who didn't get the sublime reference, Suzanne Vega's song Tom's Diner was a benchmark test during development of the MP3.[1] [1]: https://en.wikipedia.org/wiki/Tom%27s_Diner#The_"Mother_of_the_MP3" https://en.wikipedia.org/wiki/Tom%27s_Diner#The_"Mother_of_t...
  - mwcampbell 7y ago
    The reference is even more apt because the two of them have collaborated. She sang the led vocal on his song "Now I Am an Arsonist" (from the album Artificial Heart).
  - CamperBob2 7y ago
    Tom's Diner wasn't just a benchmark. Brandenburg listened to it obsessively, over and over, to the exclusion of a lot of other content he might have done well to pay more attention to. Which is what tends to happen when you work on audio processing code, for better or worse. So that's why the MP3 format mangles male vocals so badly at all but the highest bitrates. Now you know the Rest Of The Story... or at least you've read it on the Internet.
    - danso 7y ago
      This sounds really interesting. Is there a good writeup/oral history about this?
savrajsingh 7y ago
Did it just work or did you have to supply something beyond the original JC track?
- mwcampbell 7y ago
  Just worked.
ropable 7y ago
They need to link these on the project readme as a demo.
8bitsrule 7y ago
IME, this tool is certainly an order of magnitude closer to the Holy Grail than anything I've ever heard. Kudos to Deezer R&D. https://www.youtube.com/watch?v=KPlmrq_rAzQ https://www.youtube.com/watch?v=KPlmrq_rAzQ

M4v3R 7y ago

Wow, I've listened to several attempts at this over the years, but this one is waaay better than anything I've heard. It's almost perfect.

jesuslop 7y ago

Holly cow! The separation is sort of perfect! Thanks for the demo.

eps 7y ago
"Indistinguishable from magic."

deleted 7y ago

[deleted]

codedokode 7y ago

While it's a great technology, the result sounds somewhat robotic. On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder, something is missing. The voice contains pieces of strumming sound. Guitar also sounds "blurred", as if someone cut an object from the picture and blurred the cut to make it less visible. Clap sound is distorted, on the original recording it sounds the same, but after separation it sounds different every time, as if it was filtered or compressed with low bitrate. It is amazing how the ear manages to distinguish all the sounds without distortion.

MattRix 7y ago
It's easy to pick up on lots of little things, but this is still extremely impressive. It's also more than good enough for people who want to practice singing/playing over the song by themselves. I think the next step would be to train a network that can un-robotify songs and then run it on this.
uryga 7y ago
> On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder funny, the original recording seemed kind of robotic to me! maybe not robotic, but like it's been filtered somehow. but that might just be my not-so-great headphones
- mwcampbell 7y ago
  The original lead vocal is definitely processed. You can hear that processing clearly if you apply the classic vocal removal (really, center removal) effect. (My favorite implementation of that is the Center Cut DSP for foobar2000.)
nickjj 7y ago
That roboticness is because there's overlap in frequencies between the voice and the instruments. I have no idea how this tool splits them up at the implementation level but I imagine it tries to split it up based on frequencies and when it lifts out the voice, it's cutting out a ton of frequencies that would normally be in your voice so now it sounds very unnatural, blocky and metallic. With studio quality headphones I can notice a massive difference for the worse between the original and separated vocal track. It reminds me of when I turned on a noise gate too high when recording audio for my courses. That noise gate clamps down on certain frequencies to help eliminate room noise but it also dampens or removes natural frequencies that occur on the low end of an average male voice. It gives that same very jagged sounding audio waveform.
- codedokode 7y ago
  I don't argue that it's a great technology, but some of the neighbour commenters wrote things like "perfect", and to me it doesn't sound like "perfect" yet. For example, in the beginning, if you listen to the phrase "from the office ...", in original recording it sounds smooth, like a single phrase (and the voice is warm and pleasant), but in the separated track it sounds like it is synthesized from pieces that are not properly connected, like vocaloid songs. It sounds little harsher. Transitions sound unnatural. And the phrase "heya Tom" in the separated track is split into "heya" and "Tom" with some unnatural sound between (or maybe in the beginning of "Tom"). It is like transitions you can hear in vocaloid tracks. Or the kind of artifacts you get if you over-compress an MP3 file. And "it's good to see you" part also sounds robotic. Maybe it's losing some of harmonics, but in a different way for different syllables and they don't sound like a single phrase anymore.

noja 7y ago

So Jonathan Coulton is now the new Suzanne Vega?

BrentOzar 7y ago
> So Jonathan Coulton is now the new Suzanne Vega? Bravo. For people who didn't get the sublime reference, Suzanne Vega's song Tom's Diner was a benchmark test during development of the MP3.[1] [1]: https://en.wikipedia.org/wiki/Tom%27s_Diner#The_"Mother_of_the_MP3" https://en.wikipedia.org/wiki/Tom%27s_Diner#The_"Mother_of_t...
- mwcampbell 7y ago
  The reference is even more apt because the two of them have collaborated. She sang the led vocal on his song "Now I Am an Arsonist" (from the album Artificial Heart).
- CamperBob2 7y ago
  Tom's Diner wasn't just a benchmark. Brandenburg listened to it obsessively, over and over, to the exclusion of a lot of other content he might have done well to pay more attention to. Which is what tends to happen when you work on audio processing code, for better or worse. So that's why the MP3 format mangles male vocals so badly at all but the highest bitrates. Now you know the Rest Of The Story... or at least you've read it on the Internet.
  - danso 7y ago
    This sounds really interesting. Is there a good writeup/oral history about this?

savrajsingh 7y ago

Did it just work or did you have to supply something beyond the original JC track?

mwcampbell 7y ago
Just worked.

ropable 7y ago

They need to link these on the project readme as a demo.

8bitsrule 7y ago

IME, this tool is certainly an order of magnitude closer to the Holy Grail than anything I've ever heard. Kudos to Deezer R&D. https://www.youtube.com/watch?v=KPlmrq_rAzQ https://www.youtube.com/watch?v=KPlmrq_rAzQ