5 ms·

How vertically integrated do you have to be to support spatial audio? Could Spotify just add it? Could whatsapp calls add it?

dagmx 4y ago
Spotify probably could if they could deliver an Atmos stream. But what would you gain from a WhatsApp call? It’s a mono source
- londons_explore 4y ago
  It would be pretty nice for a group call to have each voice from a different direction... It would also be nice for a callers voice and other sounds in the call, like for example music or background sound, to be audibly separated - the fact they aren't separated is part of what makes background sounds on voice calls so annoying compared to say background sounds when talking to someone on the street..
  - dagmx 4y ago
    Ah yeah FaceTime positions everyone in a spatial way in group calls. Would be nice if other group call setups did the same
    - lodovic 4y ago
      That is not so useful for in a car though
      - dagmx 4y ago
        Why not? In a group call audio setting it can still help you separate who’s talking easier
        lodovic 4y ago
        In a car I assume all distractions are possible dangers and I don't want to confuse background noise in the call for actual dangers on the road. Also, I do not want to visualise the call, as that will be a major distraction from driving. To be fair, driving and having a group call is a bad idea in itself.
  - joshspankit 4y ago
    I have an odd but carefully considered opposition to this: We process speech differently depending on where it comes from, and so someone could be given an “edge” by coming from a specific “spot” while others could be downplayed by being placed in a “lesser spot”. I know it sounds completely out there, but I got this awareness years ago from a scientific study (what to even look up to find it again I don’t know. Hopefully a fellow HN user knows what I’m talking about and can cite) and I have noticed the effect is actually quite drastic once you know what to look for. To a lesser extent: Clubhouse brought in spatial audio and it caused more chaos because of the way it was implemented: Some speakers were behind others which made their voices noticeably louder, and because multiple speakers could talk at once and be understood there was less respect for the person who’s turn it was. (Before that if someone talked over it just became an unintelligible clump of sound and moderators would bring things back to the single speaker)
    - londons_explore 4y ago
      For anyone who has ever worked in a conference center, you will know that by adjusting microphone volumes up and down just a little, you can have a big impact on the outcome of any on-stage debate. Specifically, if someone thinks they're being talked over and their voice drowned out, they'll normally stop speaking. By making one person's voice sound louder to the other, you determine which person will stop speaking when both start to say something at the same time.
      - joshspankit 4y ago
        Thank you for that insight. I‘ve never worked a conference center and had no idea that was something known
crazygringo 4y ago
I think there's two questions here -- one for audio processing, and one for communications. In terms of audio, the speaker locations in the car for a specific model are all fixed and can be known. So it seems like Spotify could probably add it if they had not only all of that speaker data, but also acoustic modeling to see how each speaker's frequency sweep was transmitted and reflected to each passenger's ears. So they'd want to get their hands on an actual car and spend a lot of time making a lot of microphone measurements inside of dummy ears. And then the spatial audio algorithms aren't trivial to write, but that kind of thing has been around for a while now so it's definitely doable. But as for communicating that sound to the car -- the problem is that there are 31 speakers requiring 31 sound channels, and you certainly can't play that over Bluetooth. So I assume there has to be some custom audio stream format happening here over CarPlay? That's where the vertical integration is going to come into play. I also can't help but wonder if the Mercedes come with a dedicated audio-processing chip to handle it all. Because while a phone's chip is definitely capable enough to calculate 31 streams, I'm not sure if a Lightning port has enough bandwidth to output that uncompressed, and I'm also not sure if it would slow down the phone enough to affect the performance of other apps like Maps. So you'd probably want the phone to just output the compressed 5.1 stream, and have the car do all the fancy spatial audio processing. In any case, if this works like AirPods, the Spotify stereo audio stream will be "spatialized" so it'll work just fine. It will be inferred stereo spatial rather than 5.1 mastered, but that's still awfully effective. And a group audio call could similarly be spatialized from a stereo stream that merely panned participants left-to-right.
- dagmx 4y ago
  I think you’re over thinking it. The simple description of how this likely works is: - the tracks are encoded with Dolby atmos that allows encoding positional data for sources. These don’t match up to channels on an audio system in anyway because it gives them flexibility - the audio apps simply provide the atmos encoded stream which is pulled into the cars own audio system. The audio app needs no knowledge of the target playback device. - the cars audio system knows where the speakers are so can then playback sounds on each relative to where the atmos sources are positioned. - this isn’t done over Bluetooth. In this case the Apple Music app is running natively on the cars MBUC system. If it weren’t native, it would use CarPlay which isn’t using Bluetooth for either its wired or wireless mode.

dagmx 4y ago

Spotify probably could if they could deliver an Atmos stream. But what would you gain from a WhatsApp call? It’s a mono source

londons_explore 4y ago
It would be pretty nice for a group call to have each voice from a different direction... It would also be nice for a callers voice and other sounds in the call, like for example music or background sound, to be audibly separated - the fact they aren't separated is part of what makes background sounds on voice calls so annoying compared to say background sounds when talking to someone on the street..
- dagmx 4y ago
  Ah yeah FaceTime positions everyone in a spatial way in group calls. Would be nice if other group call setups did the same
  - lodovic 4y ago
    That is not so useful for in a car though
    - dagmx 4y ago
      Why not? In a group call audio setting it can still help you separate who’s talking easier
      - lodovic 4y ago
        In a car I assume all distractions are possible dangers and I don't want to confuse background noise in the call for actual dangers on the road. Also, I do not want to visualise the call, as that will be a major distraction from driving. To be fair, driving and having a group call is a bad idea in itself.
- joshspankit 4y ago
  I have an odd but carefully considered opposition to this: We process speech differently depending on where it comes from, and so someone could be given an “edge” by coming from a specific “spot” while others could be downplayed by being placed in a “lesser spot”. I know it sounds completely out there, but I got this awareness years ago from a scientific study (what to even look up to find it again I don’t know. Hopefully a fellow HN user knows what I’m talking about and can cite) and I have noticed the effect is actually quite drastic once you know what to look for. To a lesser extent: Clubhouse brought in spatial audio and it caused more chaos because of the way it was implemented: Some speakers were behind others which made their voices noticeably louder, and because multiple speakers could talk at once and be understood there was less respect for the person who’s turn it was. (Before that if someone talked over it just became an unintelligible clump of sound and moderators would bring things back to the single speaker)
  - londons_explore 4y ago
    For anyone who has ever worked in a conference center, you will know that by adjusting microphone volumes up and down just a little, you can have a big impact on the outcome of any on-stage debate. Specifically, if someone thinks they're being talked over and their voice drowned out, they'll normally stop speaking. By making one person's voice sound louder to the other, you determine which person will stop speaking when both start to say something at the same time.
    - joshspankit 4y ago
      Thank you for that insight. I‘ve never worked a conference center and had no idea that was something known

crazygringo 4y ago

I think there's two questions here -- one for audio processing, and one for communications. In terms of audio, the speaker locations in the car for a specific model are all fixed and can be known. So it seems like Spotify could probably add it if they had not only all of that speaker data, but also acoustic modeling to see how each speaker's frequency sweep was transmitted and reflected to each passenger's ears. So they'd want to get their hands on an actual car and spend a lot of time making a lot of microphone measurements inside of dummy ears. And then the spatial audio algorithms aren't trivial to write, but that kind of thing has been around for a while now so it's definitely doable. But as for communicating that sound to the car -- the problem is that there are 31 speakers requiring 31 sound channels, and you certainly can't play that over Bluetooth. So I assume there has to be some custom audio stream format happening here over CarPlay? That's where the vertical integration is going to come into play. I also can't help but wonder if the Mercedes come with a dedicated audio-processing chip to handle it all. Because while a phone's chip is definitely capable enough to calculate 31 streams, I'm not sure if a Lightning port has enough bandwidth to output that uncompressed, and I'm also not sure if it would slow down the phone enough to affect the performance of other apps like Maps. So you'd probably want the phone to just output the compressed 5.1 stream, and have the car do all the fancy spatial audio processing. In any case, if this works like AirPods, the Spotify stereo audio stream will be "spatialized" so it'll work just fine. It will be inferred stereo spatial rather than 5.1 mastered, but that's still awfully effective. And a group audio call could similarly be spatialized from a stereo stream that merely panned participants left-to-right.

dagmx 4y ago
I think you’re over thinking it. The simple description of how this likely works is: - the tracks are encoded with Dolby atmos that allows encoding positional data for sources. These don’t match up to channels on an audio system in anyway because it gives them flexibility - the audio apps simply provide the atmos encoded stream which is pulled into the cars own audio system. The audio app needs no knowledge of the target playback device. - the cars audio system knows where the speakers are so can then playback sounds on each relative to where the atmos sources are positioned. - this isn’t done over Bluetooth. In this case the Apple Music app is running natively on the cars MBUC system. If it weren’t native, it would use CarPlay which isn’t using Bluetooth for either its wired or wireless mode.