4 ms·
Not OP, but... Look at the list of what Apple Sing includes: Adjustable vocals: Users now have control over a song’s vocal levels. They can sing with the orig
by imchillyb 4y ago
Not OP, but...
Look at the list of what Apple Sing includes:
Adjustable vocals: Users now have control over a song’s vocal levels. They can sing with the original artist vocals, take the lead, or mix it up on millions of songs in the Apple Music catalog.
Real-time lyrics: Users can sing along to their favorite songs with animated lyrics that dance to the rhythm of the vocals.
Background vocals: Vocal lines sung simultaneously can animate independently from the main vocals to make it easier for users to follow.
Duet view: Multiple vocalists show on opposite sides of the screen to make duets or multi-singer tracks easy to sing along to.
------
The part of the article where they state these things explicitly...
- jaywalk 4y agoThey don't state anything explicitly referring to real-time processing of the songs. As a matter of fact, calling out "millions of songs in the Apple Music catalog" actually makes it seem like the adjustable vocals will only be available on certain songs that they've added support for.
- 9dev 4y agoI’d guess it’s more probable that lots of songs simply have no lyrics at all, so the claim „all songs in the Apple Music catalog“ would be factually wrong.
- brookst 4y agoIt's hard for me to imagine they'd do something special to support "millions of songs" while excluding others, since the entire catalog is ~100m songs. My guess is that it's entirely dynamic. It's hard to imagine the complexity of doing batch processing to render each song in the library, and maintain that as new songs are uploaded, and update the renders for software improvements. Better to just do it realtime. And since classical, instrumentals, esoteric ambient stuff, death metal, etc, will probably not be supported by the algorithms, I think the "millions" refers to those that can be processed in realtime.
- snotrockets 4y agoThey do something special: get the necessary sign offs from legal. Apple‘s license for some content may bar it from being used that way (that’s another reason for doing it on device: could be different situation as far legal & licensing are concerned)
- brookst 4y agoSure, and a good point. I should have said they wouldn't do anything special on a song-by-song basis for millions of songs. It's not like someone's pushing the button for each song, or building a list of songs. Those that meet the criteria will be included, whether that is 10 or 10 million of them.
- GaryPalmer 4y agoWhy would you need a dedicated chip for any of that?
- Closi 4y agoSeems like it is powered by: > [...] an on-device machine learning algorithm that can process music in real time Their latest apple TV includes the A15 which includes a 'neural engine' for ML, and this is also included in their latest iPhone / iPad, so that might be part of it.
- yamtaddle 4y agoThe divide in expectations is funny. Non-Apple-user: "ML stuff? Must be 'in the cloud'." Apple user: "ML stuff? Must use a special chip in the device."
- notpushkin 4y agoAnd then there's me, asking why do we even need ML for that. :D
- notpushkin 4y ago> Adjustable vocals: Users now have control over a song’s vocal levels. They can sing with the original artist vocals, take the lead, or mix it up on millions of songs in the Apple Music catalog. I think this only requires pre-making two audio files per track, and simultaneously streaming these. Real-time lyrics, Background vocals and Duet view are all nice features too, but the hardest part processing-wise is analysing how loud you sing into the microphone. It's just karaoke with a good UI.
- whiddershins 4y agoThey say it supports millions of songs so I doubt that’s how they are doing this. They are likely using a sophisticated ML version of what old karaoke machines did, and removing the vocals in real-time.
- Wowfunhappy 4y agoEven if they're using ML, I don't imagine they'd do it on-device. They don't even do voice recognition on-device by default.
- Closi 4y ago> [Apple says it is] relying on an on-device machine learning algorithm that processes the music in real-time. The tech builds on Apple’s noise-cancellation expertise and other developments it’s made for FaceTime, the company said. Source: https://techcrunch.com/2022/12/06/apple-music-is-getting-a-new-karaoke-like-feature-apple-sing/?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuYmluZy5jb20v&guce_referrer_sig=AQAAAAEDs_9eWipE0_onnDd8TXeBzrsIRN7xJP2nkP0JfAx0_bvOIys4bJO18kyAct_f6BJsTGibaAA_70OYSg-QNgbQjTCurnO20qxLXwnKN3u-XauKPCB4d1MzfpTPzR3_NhaWjcEn883BRWtqJG0y4lTGurpk67gm-jI0_U_V-i0t https://techcrunch.com/2022/12/06/apple-music-is-getting-a-n...
- Wowfunhappy 4y agoAh, I stand corrected! I wonder why they took that approach...