3 ms·

People will definitely rip no-vocal versions out of there (if they are good quality, that is). I see some pretty neat implications: - Young artists can (illega

People will definitely rip no-vocal versions out of there (if they are good quality, that is). I see some pretty neat implications: - Young artists can (illegally) use beats by popular producers. Young producers can listen to various tracks and better understand some techniques that were shadowed by vocals before. - More bootleg remixes of popular tracks. Things that weren't possible by just slicing and sampling the track would suddenly be possible, too. - Better music stemming / demixing ML models. I believe right now the baseline is Spleeter by Deezer [1], which is pretty good but leaves so much room for improvement. [1]: https://github.com/deezer/spleeter https://github.com/deezer/spleeter

hunter2_ 4y ago
Back in the day, we just took a channel differential (L minus R) with one channel high passed so the difference didn't have all the bass eliminated. Tons of reasons it sucked: stereo reverb on vocals wouldn't cancel out, other instruments panned dead center (or at least highly correlated) would cancel out along with the vocal, and early MP3 encoders often sounded better in "joint stereo" or "intensity stereo" mode (assuming low/mid bitrates) which very reasonably gave more bits to the center and fewer bits to the sides, so the differential sounded like hot garbage. Exciting to see that this technique has been replaced with ML.
diebeforei485 4y ago
Perhaps that is why it's not available on Mac. Of course people can still rip it off an iPhone/iPad through the "analog hole" via the Lightning to 3.5mm adapter
- notpushkin 4y ago
  Or just using a jailbreak? (unless Apple Music uses some hardware DRM)
- hrrsn 4y ago
  djay Pro AI has had this feature for a while now - https://www.algoriddim.com/neural-mix https://www.algoriddim.com/neural-mix It works pretty well on Mac
  - diebeforei485 4y ago
    But Djay Pro doesn't work with Apple Music (the streaming service) or Spotify. It does work with music you have purchased on iTunes. So it's an IP issue. It also works with the higher tiers of TIDAL or SoundCloud - https://help.algoriddim.com/hc/en-us/articles/360012493060-What-streaming-service-options-are-available-in-djay- https://help.algoriddim.com/hc/en-us/articles/360012493060-W...
ShamelessC 4y ago
If it's realtime, and you are mixing vocals and other separated tracks back together on-the-fly, the actual separations don't need to be _super_ hiqh quality, as the mixing will mask most deficiencies. The reason I say that is because I suspect they are doing this with a neural net.
- notpushkin 4y ago
  I think a neural net would be an overkill here. If you are doing the mixing on device, I think the algorithm could be as simple as just overlaying two tracks. Splitting tracks on-device would probably use a neural net, but I don't think they're doing that.
boredemployee 4y ago
i tried spleeter already and while it works "ok" in the best case, this sounds like a game changer if true: https://youtu.be/e_SW1AeCQNU https://youtu.be/e_SW1AeCQNU
- notpushkin 4y ago
  Sounds impressive! Would be nice to compare it with what Spleeter does.
smoldesu 4y ago
How does this enable anything that wasn't possible before?
- notpushkin 4y ago
  Perhaps I'm missing something, but without a remix pack, how would you take vocals and overlay those on top of a completely rewritten track for example, or bite a particular sample from the track that only plays with vocals on top?
  - smoldesu 4y ago
    There are a couple ways. The method most musicians have used since the mid-90s is phase cancellation, where you take an instrumental and use it to cancel out any sound that isn't instrumental (a-la noise-cancelling). In more dire scenarios, people would split the stereo tracks and cancel them against one another, which actually worked fairly well if you had a DAW precise enough to pull it off. Then in the mid-2000s there were a few attempts to use algorithms for this (that mostly failed) and circa-2015 we started to see machine-learning based options like Spleeter and Demucs. This is what Apple is implementing here, and it's been freely available for years. It's not really novel for anyone who isn't writing their music on an iPhone. I mostly ask because I've spent the past 2 or 3 weeks working on a remix album, and haven't really had trouble finding stem separation tools that work well.
hoherd 4y ago
Stemroller[1] is another audio track separator, based on demucs[2]. The demucs GH page has an accuracy comparison table with spleeter, demucs and others. [1]: https://github.com/stemrollerapp/stemroller https://github.com/stemrollerapp/stemroller [2]: https://github.com/facebookresearch/demucs https://github.com/facebookresearch/demucs
layer8 4y ago
I wouldn’t expect the ML-based real-time vocals removal to be high-quality enough for those use cases. It’s a really difficult task, and it doesn’t have to be high-quality to be adequate for karaoke. The press release also says "The vocal slider adjusts vocal volume, but does not fully remove vocals".
_whiteCaps_ 4y ago
Your comment reminded me of ...And Justice For Jason, where people fixed the bass guitar volume of the Metallica album by taking the individual channels from Rock Band. https://www.loudersound.com/news/metallica-and-justice-for-all-with-bass https://www.loudersound.com/news/metallica-and-justice-for-a...
bambax 4y ago
There are many services or apps/plugins that can do this. I think the promise Apple is making isn't necessarily that it will be better, but that they will have done it prior for many songs in their catalog, so that the stems are immediately available.

hunter2_ 4y ago

Back in the day, we just took a channel differential (L minus R) with one channel high passed so the difference didn't have all the bass eliminated. Tons of reasons it sucked: stereo reverb on vocals wouldn't cancel out, other instruments panned dead center (or at least highly correlated) would cancel out along with the vocal, and early MP3 encoders often sounded better in "joint stereo" or "intensity stereo" mode (assuming low/mid bitrates) which very reasonably gave more bits to the center and fewer bits to the sides, so the differential sounded like hot garbage. Exciting to see that this technique has been replaced with ML.

diebeforei485 4y ago

Perhaps that is why it's not available on Mac. Of course people can still rip it off an iPhone/iPad through the "analog hole" via the Lightning to 3.5mm adapter

notpushkin 4y ago
Or just using a jailbreak? (unless Apple Music uses some hardware DRM)
hrrsn 4y ago
djay Pro AI has had this feature for a while now - https://www.algoriddim.com/neural-mix https://www.algoriddim.com/neural-mix It works pretty well on Mac
- diebeforei485 4y ago
  But Djay Pro doesn't work with Apple Music (the streaming service) or Spotify. It does work with music you have purchased on iTunes. So it's an IP issue. It also works with the higher tiers of TIDAL or SoundCloud - https://help.algoriddim.com/hc/en-us/articles/360012493060-What-streaming-service-options-are-available-in-djay- https://help.algoriddim.com/hc/en-us/articles/360012493060-W...

ShamelessC 4y ago

If it's realtime, and you are mixing vocals and other separated tracks back together on-the-fly, the actual separations don't need to be _super_ hiqh quality, as the mixing will mask most deficiencies. The reason I say that is because I suspect they are doing this with a neural net.

notpushkin 4y ago
I think a neural net would be an overkill here. If you are doing the mixing on device, I think the algorithm could be as simple as just overlaying two tracks. Splitting tracks on-device would probably use a neural net, but I don't think they're doing that.

boredemployee 4y ago

i tried spleeter already and while it works "ok" in the best case, this sounds like a game changer if true: https://youtu.be/e_SW1AeCQNU https://youtu.be/e_SW1AeCQNU

notpushkin 4y ago
Sounds impressive! Would be nice to compare it with what Spleeter does.

smoldesu 4y ago

How does this enable anything that wasn't possible before?

notpushkin 4y ago
Perhaps I'm missing something, but without a remix pack, how would you take vocals and overlay those on top of a completely rewritten track for example, or bite a particular sample from the track that only plays with vocals on top?
- smoldesu 4y ago
  There are a couple ways. The method most musicians have used since the mid-90s is phase cancellation, where you take an instrumental and use it to cancel out any sound that isn't instrumental (a-la noise-cancelling). In more dire scenarios, people would split the stereo tracks and cancel them against one another, which actually worked fairly well if you had a DAW precise enough to pull it off. Then in the mid-2000s there were a few attempts to use algorithms for this (that mostly failed) and circa-2015 we started to see machine-learning based options like Spleeter and Demucs. This is what Apple is implementing here, and it's been freely available for years. It's not really novel for anyone who isn't writing their music on an iPhone. I mostly ask because I've spent the past 2 or 3 weeks working on a remix album, and haven't really had trouble finding stem separation tools that work well.

hoherd 4y ago

Stemroller[1] is another audio track separator, based on demucs[2]. The demucs GH page has an accuracy comparison table with spleeter, demucs and others. [1]: https://github.com/stemrollerapp/stemroller https://github.com/stemrollerapp/stemroller [2]: https://github.com/facebookresearch/demucs https://github.com/facebookresearch/demucs

layer8 4y ago

I wouldn’t expect the ML-based real-time vocals removal to be high-quality enough for those use cases. It’s a really difficult task, and it doesn’t have to be high-quality to be adequate for karaoke. The press release also says "The vocal slider adjusts vocal volume, but does not fully remove vocals".

_whiteCaps_ 4y ago

Your comment reminded me of ...And Justice For Jason, where people fixed the bass guitar volume of the Metallica album by taking the individual channels from Rock Band. https://www.loudersound.com/news/metallica-and-justice-for-all-with-bass https://www.loudersound.com/news/metallica-and-justice-for-a...

bambax 4y ago

There are many services or apps/plugins that can do this. I think the promise Apple is making isn't necessarily that it will be better, but that they will have done it prior for many songs in their catalog, so that the stems are immediately available.