16 ms·
Spleeter: Extract voice, piano, drums, etc. from any music track
- ooobo 7y agoGave this a go, it's an easy install with pip, and results are pretty quick even on an old macbook. Splits into 2stems (vocals/accompaniment) on some random songs I chose actually quite good using the pretrained models provided. Of course, ripping the vocals out of the accompaniment takes out a good chunk of the middle frequencies so some songs sound a bit wonky. Worth a play if you are interested.
- tomrod 7y agoSame thoughts here. I ran Thriller, Alligator by Of Monsters and Men, and In Hell I'll be in Good Company by The Dead South on the 2 / 5 / 4 stems, respectively. Impressive results. Definitely agree that some of the middle frequencies show some error. It would be really cool to create "music mappers"/life sounds tracks like what you can do with pictures & art styles (e.g. https://medium.com/tensorflow/neural-style-transfer-creating-art-with-deep-learning-using-tf-keras-and-eager-execution-7d541ac31398 https://medium.com/tensorflow/neural-style-transfer-creating...)
- colorincorrect 7y agoknown nothing about the results, i suspect that mid-ranges are poorer mainly because human frequency response is most sensitive towards mid-range aka vocal-pitch frequency
- lrobinovitch 7y agoSame, Rage Against The Machine - Killing In the Name came out sounding great. Very cool.
- sehugg 7y agoIt's really good on the 2-stem stuff. On the 4-stem model, it's a bit shy about the bass part, and parts drift in and out. I'd like to try it on a FLAC.
- SemiTom 7y agoInteresting to read Thomas Dolby's thoughts on music/technology interfaces--particularly with VR https://semiengineering.com/thomas-dolbys-very-different-view-of-progress/ https://semiengineering.com/thomas-dolbys-very-different-vie...
- tomrod 7y agoThis is so neat! I went looking a few months back for something like this, and the best I found was Google's Magenta. It would be really cool to use this to feed into Magenta. Think of the mashups!
- jknz 7y agoOn iOS, Chord AI [1] gives pretty good results for the guitar chords of any music surrounding the phone. [1]: https://apps.apple.com/us/app/chord-ai/id1446177109 https://apps.apple.com/us/app/chord-ai/id1446177109
- ohlookabird 7y agoThat's sounds pretty nice! Anyone know an Android version? I just checked Yamaha Chord Tracker and MyChord, but both don't seem to be able to use the microphone.
- dgreensp 7y agoAre there any examples I can listen to?
- tomrod 7y agoI gave it a quick test using their audio sample file: https://soundcloud.com/thomas-roderick-836298141/sets/spleeter_splits https://soundcloud.com/thomas-roderick-836298141/sets/spleet...
- tomo-makes 7y agoI set up a Colab notebook to try spleeter out for myself. You can try picking up your favorite mp3, renaming it to "audio_sample.mp3", uploading it to the Colab, and spining all the cells on the notebook. Enjoy. https://colab.research.google.com/gist/tomo-makes/33b9bc7f22e257468ff9bbc7122a6e82/spleeter-multi-source-separation-demo.ipynb https://colab.research.google.com/gist/tomo-makes/33b9bc7f22...
- sillysaurusx 7y agoThis is great. Thank you! I extended the notebook with an example of downloading a youtube video, extracting the audio, then feeding it through spleeter. Notebook: https://colab.research.google.com/gist/shawwn/0f286f5d4bc22e04d745657ba3ccad26/spleeter-multi-source-separation-demo.ipynb https://colab.research.google.com/gist/shawwn/0f286f5d4bc22e...
- tomo-makes 7y agoCool! That's so handy.
- unlinked_dll 7y agodemo links would be helpful
- voicedYoda 7y agoI gave a talk at pycon this year about dsp [1], specifically some of the complexities surrounding this. I came across a few other ml projects that claimed to do this as well, and the biggest hold up is getting enough properly trained data, tagged appropriately, in order to let the models train correctly. in the git repo of this project they also explicitly state you need to train on your own data set, though you can use their models of your like. YMMV. I will love to try this out, as it's definitely a complex bit of audio engineering. That said, i loved learning everything i did preparing for my talk and need to finish up some other parts of the project to get the jukebox working... Maybe this will help :) 1. https://m.youtube.com/watch?v=fevxy-s0vo0 https://m.youtube.com/watch?v=fevxy-s0vo0
- TheRealSteel 7y agoI'm sure you've thought of this, but could/have the tracks from the Rock Band games be used for training? There are thousands of them and they're separated into different instrument tracks. They even had bands re-record songs sometimes where seperate masters couldn't be found. If I recall correctly, Third Eye Blind did this for Semi-Charmed Life.
- taxidump 7y agoTo add there is a format of music called Stems designed for DJ's and live remixers from Native Instruments which is a disassembly of the song into it's various parts. https://www.native-instruments.com/en/specials/stems/ https://www.native-instruments.com/en/specials/stems/
- Intermernet 7y agoFYI The term "stems" to refer to the individual tracks of a piece of recorded music is a lot older than NI's format. I love NI, but I'm annoyed that they chose to appropriate the industry standard term as a proprietary product. https://en.wikipedia.org/wiki/Stem_mixing_and_mastering https://en.wikipedia.org/wiki/Stem_mixing_and_mastering
- jefftk 7y agoI have a large number of multitrack recordings of contra dances if anyone wants to try training this on them.
- exikyut 7y agoYou train this on them! (And then put the results on SoundCloud or YouTube.)
- toptal 7y agoCan someone provide a demo link of source music vs. output?
- tomrod 7y agoI gave it a test using the project audio sample. Neat stuff. https://soundcloud.com/thomas-roderick-836298141/sets/spleeter_splits https://soundcloud.com/thomas-roderick-836298141/sets/spleet...
- semiotagonal 7y agoHoly shit that works way better than I expected. The github project should link to this or a similar example, the technical description doesn't do it justice.
- jcims 7y agoYeah the fact that it got the reverb in the vocal track is pretty impressive!
- unlinked_dll 7y agoI really disagree. It sounds... awful. On par with other approaches, sure, but the main vocals sounds like a case study in digital artifacts and the accompaniment sounds like there's a filter automated over the track. Far from useful.
- semiotagonal 7y agoDo you know of something superior?
- cma 7y agoIs there anything like this for images? Meaning essentially trying to decompose back into photoshop layers. Wouldn't be feasible for lots of stuff that is completely opaquely covering something, but I'm thinking for things like recoloring a screen print, etc.
- exikyut 7y agoI have no idea how I managed to re-find these, but I did. Two recent moderately-related/relevant posts: https://news.ycombinator.com/item?id=20978055 https://news.ycombinator.com/item?id=20978055 https://news.ycombinator.com/item?id=21220458 https://news.ycombinator.com/item?id=21220458
- bravura 7y agoIs the paper, "Spleeter: A Fast And State-of-the Art Music Source Separation Tool With Pre-trained Models", available yet? What is the methodology?
- czr 7y agothey made an extended abstract for ismir: http://archives.ismir.net/ismir2019/latebreaking/000036.pdf http://archives.ismir.net/ismir2019/latebreaking/000036.pdf methodology is a separate u-net per instrument type to predict a soft mask in spectrogram space (time x frequency), then they apply that mask to the input audio. fairly standard.
- gdsdfe 7y agoWait the implications of this are huge for electronic music DJs
- exikyut 7y agoThe audio (^F soundcloud) sounds a little warbly... if that can be largely mitigated, then yes, remixes will never be the same
- yowlingcat 7y agoWhile not great, the phase smearing is orders of magnitude better than most vocal isolation plugins I've used. I only expect it to get better. Very cool!
- tomrod 7y agoAudio Neural Transfer Learning could be amazing.
- esfandia 7y agoKaraoke with the most obscure songs!
- theLotusGambit 7y agoNot only are the results good, but the music is generated decently rapidly. The implications are clear: whoever wants to make a quick fortune on YouTube should start converting and uploading truckloads of songs as fast as possible. The demand is there. I could easily see that bringing in millions of views.
- deleted 7y ago[deleted]
- ErikAugust 7y agoThey’d still get tagged for copyright.
- xamuel 7y agoI wonder how it would fare on Pink Floyd's "Sheep", where vocals seamlessly transform into instrumentals and it's impossible to tell where one ends and the other begins. https://www.youtube.com/watch?v=3-oJt_5JvV4 https://www.youtube.com/watch?v=3-oJt_5JvV4 (skip to around 1:40)
- deleted 7y ago[deleted]
- Intermernet 7y agoI'd love to see how this compares with Celemony Melodyne. As far as I've been able to determine, Melodyne doesn't use ML, but it's hard to find out exactly what it does use. Either way, an open source competitor to Melodyne is a welcome addition!
- matchagaucho 7y agoI’ve always assumed Melodyne uses FFT bins.
- SyneRyder 7y agoThat's the second time I've seen someone mention Melodyne for separating vocals from a full song source - I don't think that's something it can do? Melodyne is for tuning vocals / instruments & correcting timing on already isolated tracks.
- czr 7y agomelodyne's editing interface lets you remove different notes from a polyphonic track. so if it's just vocals + other tonal sounds, you can manually remove the other tonal sounds. example: https://youtu.be/2ZjdDatxTaQ?t=83 https://youtu.be/2ZjdDatxTaQ?t=83
- vonseel 7y agoHmm, never tried that with melodyne myself and the video you posted isn't a great example of an accurate vocal extraction - those are more like vocal chops and are already pretty dirty to begin with. Based on my experience with Melodyne, I'd be surprised if you could cleanly extract a plain singing vocal without tons and tons of work.
- dspig 7y agoThere is a patent for Melodyne that describes looking for harmonics vs time in FFTs, then heuristics for deciding which belong to one note and where it starts and ends, then assigning some of the resudual energy (e.g. noisy onset) to each note.
- BeeBoBub 7y agoI am working on a product which makes use of this technology. I generate vocal pitch visualizations for karaoke http://pitchperfected.io http://pitchperfected.io
- mh- 7y agoFYI your email confirmation is going straight to spam on gmail. I'd recommend reaching out to Mailchimp.
- noja 7y agoCool - but your website needs some work. It looks like a landing page to gather interest rather than something backed by a real product. Show us some videos and singing, before and after, etc.
- foobaw 7y agoThis is amazing - so much possible learning for aspiring producers.
- murat124 7y agoI gave it a try with Megadeth's Holy wars[1], was expecting something like this[2] but got very deep audio. Not sure why but perhaps it's because bassist David Ellefson uses pick which gives the percussive sound and it suits to Megadeth. Any parameter I could use with spleeter to get a similar output? [1] https://www.youtube.com/watch?v=9d4ui9q7eDM https://www.youtube.com/watch?v=9d4ui9q7eDM [2] https://www.youtube.com/watch?v=uWkykQHsJ-Y https://www.youtube.com/watch?v=uWkykQHsJ-Y
- jMyles 7y agoSomewhat of a tangent, but does anyone have a recommendation of an open source (ideally python) program that can make MIDI from piano audio?
- iamchrisle 7y agoThere is probably something out there, but I know you can do this in Ableton Live by dragging an audio file onto a MIDI track and it will extract the notes into MIDI for you.
- vonseel 7y agoMelodyne or Ableton. I've found Melodyne to be more accurate, but still not perfect.
- quadrangle 7y agoThey asked for open source
- CurryPaste 7y agoMIDI Guitar 2 from Jam Origin (jamorigin.com) works well even for piano.
- jacquesm 7y agoI've been working on this for the last 3 months. Can you send me your audio file? I'll send you back the midi I can generate; it won't be perfect but it might be usable. See comment elsewhere in this thread.
- jMyles 7y agoAwesome! Is your project open source? Can't wait to see it. Here's the first (but by no means only) file I have in mind: http://ocrmirror.org/files/music/albums/ff7/MP3/3-02%20Shnabubula%20-%20Stone%20Eyes%20(The%20Great%20Warrior).mp3 http://ocrmirror.org/files/music/albums/ff7/MP3/3-02%20Shnab... Stone Eyes from the Final Fantasy VII tribute album Voices of the Lifestream.
- 7y ago
- mwcampbell 7y agoFor your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify his music under its Creative Commons license. First, the original: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-your-brains/original.mp3 https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Now the derived stems: Vocals: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-your-brains/stems/vocals.mp3 https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Accompaniment: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-your-brains/stems/accompaniment.mp3 https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Note: I'm not affiliated with this project or Mr. Coulton. I just think this is a cool project and wanted to share.
- M4v3R 7y agoWow, I've listened to several attempts at this over the years, but this one is waaay better than anything I've heard. It's almost perfect.
- jesuslop 7y agoHolly cow! The separation is sort of perfect! Thanks for the demo.
- eps 7y ago"Indistinguishable from magic."
- deleted 7y ago[deleted]
- codedokode 7y agoWhile it's a great technology, the result sounds somewhat robotic. On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder, something is missing. The voice contains pieces of strumming sound. Guitar also sounds "blurred", as if someone cut an object from the picture and blurred the cut to make it less visible. Clap sound is distorted, on the original recording it sounds the same, but after separation it sounds different every time, as if it was filtered or compressed with low bitrate. It is amazing how the ear manages to distinguish all the sounds without distortion.
- lreichold 7y agoAn interesting alternative approach for instrument sound separation is to use a fused audio + video model. So, given that you also have video of the instruments being played, you can perform this separation with higher fidelity. I was fascinated by the work done by “The Sound of Pixels” project at MIT. http://sound-of-pixels.csail.mit.edu/ http://sound-of-pixels.csail.mit.edu/
- renaudg 7y agoThat’s quite clever but not really practical : instruments heard in most music produced today aren’t "played" by humans.
- marsknight 7y agoThank you!! This is really amazing :)
- strags 7y agoThis is awesome. I now have Guns'n'Roses playing in my office, and Axl Rose is a faint voice coming from the garage.
- czr 7y agomessed around with the 2stem model for a bit and it's reasonably good. I think phonicmind is still a bit better - phonicmind tends to err on the side of keeping too much, while the 2stem model tries to isolate aggressively and often damages the vocal as a result (distorting words by losing some harmonics, or losing quiet words entirely) example: https://files.catbox.moe/wjruiv.mp3 https://files.catbox.moe/wjruiv.mp3 (phonicmind) https://files.catbox.moe/uuzot3.mp3 https://files.catbox.moe/uuzot3.mp3 (spleeter 2stem) you can hear spleeter does better at actually taking out the bass drums, but phonicmind never loses or distorts any part of the vocal, while 2stem occasionally sounds like singing is through metal tube (harmonics are missing). will try to read instructions more carefully and see if there's some way to fix.
- roryokane 7y agoFor those who, like me, hadn’t heard of PhonicMind before, it’s an online service at https://phonicmind.com/ https://phonicmind.com/ that charges $4 to $1.5 per song to separate out vocals, drums, bass, and the rest of the sounds. You can upload any audio file to that website and get a 30-second preview of separated parts for it.
- xiphmont 7y agoJust tried it on _Meet The Sniper_. Disappointment :-( [This was an unreasonable test, and it did really well considering the likely training set. I bet it could do much better with better data. Still... man, I was so hoping for magic.]
- iagooar 7y agoDoes it work with spoken word as well? My use case: improve podcast quality by extracting the vocals only, and leaving out all background and accidental noise.
- ssttoo 7y agoNot free nor open source but you can try a plugin called izotope Rx for this purpose
- deleted 7y ago[deleted]
- haywirez 7y agoIs there a known approach that attempts to separate all distinct sounds (timbres rather than pitch) in a track? Specifically targeted at electronic music, not standard acoustic ensembles.
- dancek 7y agoThat's an interesting idea. Many instruments have greatly varying timbres, though. Combining the timbres back to instruments would require another level of processing.
- sheinsheish 7y agoCan we use sample libraries to write, record and simulate desired stems for training? I guess the more naturally played the better?
- antman 7y agoHow could we extract anything but the voice e.g. karaoke?
- aasasd 7y agoI'd guess extract the voice and then subtract it from the rest with something like Audacity. I'm not sure which operation would do that, but I believe that it exists. Also, other comments here speak of “separating into the voice and accompaniment,” so maybe the model/program already do exactly what you need.
- marcan_42 7y agoInvert and mix. When you have an instrumental version of a song (from the same stems as the vocal version) this is already one way to get the vocals out without any fancy machine learning. The main tricks besides what you can do in Audacity like that are properly time-aligning the tracks (even if they drift a bit) and compensating for phase issues and compression. I wrote a dirty tool that does that and I've been meaning to turn it into some kind of nicer GUI version.
- voicedYoda 7y agoIf you don't mind sharing (even if it's cmd line), I'd love to explore.
- navjack27 7y agoI've been doing something like this for a bit in Audition. Center channel extract > invert phase > save as wav > create multi-track project > add original > add modified > up the volume on the vocal extracted modified version until the vocals go away Or you can do the exact opposite and instead of center channel extracting the vocals you can remove the vocals and use this method to better isolate vocals. Although if things do fun stuff with stereo it might not be exact.
- NilsIRL 7y agoI'm pretty sure it does that by default.
- mothsonasloth 7y agoI'm trying to find something to generate tracks without guitar. Then I can cover them with my guitar. Will this software help me?
- deleted 7y ago[deleted]
- ksherlock 7y agodepends? It splits it into 2 parts (vocal, everything else), 4 parts (drum, bass, vocal everything else), or 5 parts (drum, bass, vocal, piano, everything else). piano isolation is the weakest. If you want to play along to drum + bass, then yes.
- odiroot 7y agoWell, can it extract the bass track from "And the justice for all"?
- iamchrisle 7y agoNon-open source products that also separate vocals from music if you need something more "professional". One-click process: Xtrax Stems 2 (https://audionamix.com/technology/xtrax-stems/ https://audionamix.com/technology/xtrax-stems/) Professional: ADX Trax Pro 3 (https://audionamix.com/technology/adx-trax-pro/ https://audionamix.com/technology/adx-trax-pro/) Both products use a server which have a much larger pre-trained models. The professional one has added features such as handling sibilance, GUI to edit note following as a guide for the models, and an editor tool for extracting using harmonics. (Note: I don't work for this company. I do pay for / use their products, and I also happen to know someone who works there.)
- clashmeifyoucan 7y agoI wonder how this compares to Open Unmix (https://github.com/sigsep/open-unmix-pytorch https://github.com/sigsep/open-unmix-pytorch), that one calls itself state-of-the-art as well and is done in collaboration with Sony from what I see of their paper.
- clashmeifyoucan 7y agoOh I just found out their paper, http://archives.ismir.net/ismir2019/latebreaking/000036.pdf http://archives.ismir.net/ismir2019/latebreaking/000036.pdf. It's pretty competitive.
- amelius 7y agoCan this also separate the backing vocals from the lead singer?
- amelius 7y ago> Spleeter is the Deezer source separation library with pretrained models Curious, what is Deezer using this for?
- davidy123 7y agoI look forward to a day I can click a button to watch videos online without any unnecessary and distracting background music (though it would be better if there were an option and precedent to offer unornamented narrative in video players). The next step after this would be to have live 'music cancelling' headphones for the grocery store (if such a thing still exists).
- swagasaurus-rex 7y agoWow. Office background noise mute. The headphones can filter out speech that isn't above a certain threshold. Coworkers nearby can be heard loud and clearly. Music can play at volume then quiet itself when it detects a person speaking directly to you. Maybe even a training button to inform it that it is false-positive-ing background noise, or true negative and silencing a co worker you would like to hear.
- fdej 7y agoHas anyone tested this with Glenn Gould recordings?
- jacquesm 7y agoHehe, you want to split his singing and humming into a separate track?
- nycjobsboard 7y agoCool stuff
- orloffm 7y agoPlayed with it. The quality of the result is mostly dependent on the amount of clipping in the source file. Basically, all post-90s masters produce weird results with orcs singing in the background. And classics from 60s yield fantastic results.
- cantankerous 7y agoMight be the training data?
- jacquesm 7y agoThis is very timely. I've been working for about 3 months now on a utility that transforms mp3's to midi files. It's a hard problem and even though I'm making steady progress the end is nowhere in sight. This will give me something to benchmark against with for instance voice accompanied by piano. Thank you for making/posting this. For an idea how this project is coming along: https://jacquesmattheij.com/toccata.mp3 https://jacquesmattheij.com/toccata.mp3 Yes, it's terrible :) This particular file the result of the following transformations: midi file -> wav file (fluidsynth) wav file -> midi file (my utility) midi file -> wav file (fluidsynth once more) wav file -> mp3 file (using lame) Of course it also works for regular midi files (piano only for now). The reason why I use the workflow above is that it gives me a good idea how well the program works by comparing the original midi file with the output one. But I did not yet have a way to deal with piano/voice which is a very common combination so this might really help me. Possible applications: automatic music transcription, tutoring, giving regular pianos a midi 'out' port, using a regular piano as an arranger keyboard, instrument transformation and many others. Having fun! Edit: I've done a little write-up: https://jacquesmattheij.com/mp3-to-midi/ https://jacquesmattheij.com/mp3-to-midi/
- IAmGraydon 7y agoJust FYI in case you weren't aware - Ableton Live and several other DAWs have this capability built in. It's far from perfect, but great for humming a melody and then quickly turning it into MIDI.
- jacquesm 7y agoHow good is it (% accuracy) for polyphony? I can upload the toccata original if you want.
- viburnum 7y agoHow do you do this in Ableton?
- smcnally 7y agohttps://www.ableton.com/en/manual/converting-audio-to-midi/ https://www.ableton.com/en/manual/converting-audio-to-midi/ There are four options. > Convert Melody to New MIDI Track > This command identifies the pitches in monophonic audio and places them into a clip on a new MIDI track.
- propercoil 7y agoPytorch > TensorFlow
- lebuffon 7y agoHas someone in the Intelligence community approached the author? Oops that's classified. :) This has implications for extracting sounds from noisy recordings or am I off base? Does it only track pitch patterns?
- jononor 7y agoSource separation of speech and speech denoising is a well established field, more researched in general than music source separation. Intelligence officers very likely have access to a range of well-performing ML tools for extracting speech.
- huskyr 7y agoThis is incredible. I made an example using David Bowie's "Changes". A bit robotic, but even the echo is still present in the vocal track. https://www.youtube.com/watch?v=KPlmrq_rAzQ https://www.youtube.com/watch?v=KPlmrq_rAzQ
- hodder 7y agoVery cool. A close friend of mine (and lead singer in our band May years ago) recently died and we have a couple great recordings from 2 decades ago of his vocals. When we recorded the rest of the instruments they were DI into a Boss BR8. The lyrics sound awesome but the guitar and drums are recorded poorly. This may give us a chance to split the vocals out of the final tracks, and re-record the tracks as a tribute. Much appreciated.
- nagateja_1995 7y agoI was just testing it out on Gazal, it seems to work perfectly. But when it seems to fail with Qawwalis. My understanding of how this works. Is it safe to assume that the training data from dreezer lacks enough examples of Qawwalis?
- mettamage 7y agoAh, so this should've been the answer to my ask HN [1]. ;-) Edit: I see someone added it in as an answer 14 hours ago. Well, you have my vote ^^ [1] https://news.ycombinator.com/item?id=21399838 https://news.ycombinator.com/item?id=21399838
- seventhtiger 7y agoAren't all the most popular audio formats lossy? Extracting full data from lossy compression requires reconstruction. Even if they are able to completely extract all tracks they would have gaps and be very low quality.
- ZeikJT 7y agoIf you want lossless audio in a decently popular format then just go looking for FLAC, you'll definitely find some. Even bandcamp uses it [1]. [1] - https://en.wikipedia.org/wiki/FLAC#Adoption_and_implementations https://en.wikipedia.org/wiki/FLAC#Adoption_and_implementati...
- jsilence 7y agoKaraoke everything!
- alez 7y agoTried it on “Halleluwah” by CAN, had to hear those drums: https://soundcloud.com/alezzzz/can-halleluwah-drums-extracted https://soundcloud.com/alezzzz/can-halleluwah-drums-extracte... Finding drum breaks in music is very time consuming. This is gonna be amazing for music production. Think how 90s jungle would’ve been if they had access to every drum take ever
- smrq 7y agoWow, this is the isolated track I never knew I needed to hear.
- yoogidoky 7y agoThe long story about Spleeter : https://deezer.io/releasing-spleeter-deezer-r-d-source-separation-engine-2b88985e797e https://deezer.io/releasing-spleeter-deezer-r-d-source-separ...
- exogen 7y agoThe extracted vocals sound great! But the resulting accompaniment tracks I've heard so far (tried on a handful of songs) aren't of usable quality for most purposes where you'd want an instrumental track – they're too sonically mangled. Since people are often interested in doing this for a handful of specific tracks and not necessarily en masse, I'd be curious about what a human-assisted version of this could look like and whether you really could get near-perfect results... What if you explicitly selected portions of the track you knew had vocals, so it could (1) know to leave the rest alone and (2) know what the backing track for the specific song naturally sounds like when there's no singing happening? It could try to match that sonic profile more carefully in the vocal-removed version. Or what if you could give it even more info, and record yourself/another singing (isolated) over the track? Then it would have information about what phonemes it should expect to find and remove (and whatever effects like reverb are applied to them).