5 ms·

Is there an audio format that stores the 3D origins of each sound, so that you could theoretically play it through Airpods (or some other spatially-aware headph

by tines 2y ago

Is there an audio format that stores the 3D origins of each sound, so that you could theoretically play it through Airpods (or some other spatially-aware headphones) and hear the sound change as you tilt your head?

justinsaccount 2y ago
Something like https://en.wikipedia.org/wiki/Dolby_Atmos https://en.wikipedia.org/wiki/Dolby_Atmos and https://en.wikipedia.org/wiki/DTS,_Inc.#DTS:X https://en.wikipedia.org/wiki/DTS,_Inc.#DTS:X
hyperific 2y ago
Another way to do this would be to make an app that uses positional information from the headphones and feeds the sound of your choice into a head related transfer function (HRTF) to give the illusion of spatial audio.
bee_rider 2y ago
I sort of expected this sort of thing to be a nice use for VR, since it already needs to do the tracking. But VR generally didn’t seem to catch on, so…
- PaulHoule 2y ago
  I have a home theater which is 5.1 which is great for movies [1] and some music [2]. There are 'virtual surround' systems based on [3] of which Airpods are the most famous. Most of those don't work for me except in the sense that I learn 'sounds like I have a cold' means 'the sound is supposed to be above me'. The exception is my Hololens where if I use the web browser to play music and turn around it really sounds like the music is coming from behind me. I have yet to be impressed by spatial audio on my Meta Quest 3. [1] Oddly the surround sound soundtracks that impress me the most are old VHS tapes encoded with Dolby Pro Logic [2] let's see... Close to the Edge by Yes, Supernature by Goldfrapp, Maximum-Minimum by Kraftwerk [3] https://en.wikipedia.org/wiki/Head-related_transfer_function https://en.wikipedia.org/wiki/Head-related_transfer_function
_flux 2y ago
You could also just emulate what the music would sound like with a stereo system and support head tracking with that. dear reality does something like that with their dearVR digital audio workstation plugins. And then I recall there was this one old Kickstarter project that made headphones with that idea. It had an IR transmitter for head tracking. I think it might be quite difficult to track head movement with just earbuds and not have the tracking drift, though.
noahfk 2y ago
I'm working on a mobile app that lets you playback projects created in Audiocube and gives you full control over where your listening position. It's definitely possible to have head tracking, I think it will just depend on what headphones and hardware offer it. It would be cool to give you that kind of VR listening experience but just on headphones
colanderman 2y ago
Dolby Atmos is literally that, I think there are several physical formats that can support it. Ambisonics, especially higher-order ambisonics, also is capable of spatial panning, though it encodes the 3D soundfield rather than individual sound sources. Again, there are a few physical formats.
deleted 2y ago
[deleted]
mrandish 2y ago
In addition to Dolby Atmos there is MPEG-H 3D Audio. https://en.wikipedia.org/wiki/MPEG-H_3D_Audio https://en.wikipedia.org/wiki/MPEG-H_3D_Audio https://ieeexplore.ieee.org/document/7056445 https://ieeexplore.ieee.org/document/7056445
glenneroo 2y ago
This is probably a very odd and roundabout way of doing something akin to what you are asking for, but my recently released VR game DodgeALL[0] has a form of modding support for (wav/mono) audio files which can be uploaded (via SideQuest) to the headset. These are then assigned to "obstacles" which are launched at the player i.e. you could move your head around and hear spatialized audio files as they fly around the arena. You don't even need special headphones, the ones on the Quest do a good enough job, and Meta's Spatial audio SDK does a wonderful job of spatializing audio. I was originally using SteamAudio[1] but switched mainly due to moving from PC-VR to Standalone. Sorry if this sounds like some self-promotion - I specifically built this functionality[2] for my previous audio engineer so that he could quickly design and test sounds without the hassle of having to build/install an APK with every new audio file. [0]: https://www.DodgeALL.com https://www.DodgeALL.com (only for Quest 2/3/3s/Pro headsets) [1]: https://valvesoftware.github.io/steam-audio/ https://valvesoftware.github.io/steam-audio/ [2]: https://docs.google.com/document/d/1IFQjkroomHh8FocjfA-a55JPnrgpmSGkf2bXzZHEkLI/edit?tab=t.0 https://docs.google.com/document/d/1IFQjkroomHh8FocjfA-a55JP...

by tines 2y ago

justinsaccount 2y ago

Something like https://en.wikipedia.org/wiki/Dolby_Atmos https://en.wikipedia.org/wiki/Dolby_Atmos and https://en.wikipedia.org/wiki/DTS,_Inc.#DTS:X https://en.wikipedia.org/wiki/DTS,_Inc.#DTS:X

hyperific 2y ago

Another way to do this would be to make an app that uses positional information from the headphones and feeds the sound of your choice into a head related transfer function (HRTF) to give the illusion of spatial audio.

bee_rider 2y ago

I sort of expected this sort of thing to be a nice use for VR, since it already needs to do the tracking. But VR generally didn’t seem to catch on, so…

PaulHoule 2y ago
I have a home theater which is 5.1 which is great for movies [1] and some music [2]. There are 'virtual surround' systems based on [3] of which Airpods are the most famous. Most of those don't work for me except in the sense that I learn 'sounds like I have a cold' means 'the sound is supposed to be above me'. The exception is my Hololens where if I use the web browser to play music and turn around it really sounds like the music is coming from behind me. I have yet to be impressed by spatial audio on my Meta Quest 3. [1] Oddly the surround sound soundtracks that impress me the most are old VHS tapes encoded with Dolby Pro Logic [2] let's see... Close to the Edge by Yes, Supernature by Goldfrapp, Maximum-Minimum by Kraftwerk [3] https://en.wikipedia.org/wiki/Head-related_transfer_function https://en.wikipedia.org/wiki/Head-related_transfer_function

_flux 2y ago

You could also just emulate what the music would sound like with a stereo system and support head tracking with that. dear reality does something like that with their dearVR digital audio workstation plugins. And then I recall there was this one old Kickstarter project that made headphones with that idea. It had an IR transmitter for head tracking. I think it might be quite difficult to track head movement with just earbuds and not have the tracking drift, though.

noahfk 2y ago

I'm working on a mobile app that lets you playback projects created in Audiocube and gives you full control over where your listening position. It's definitely possible to have head tracking, I think it will just depend on what headphones and hardware offer it. It would be cool to give you that kind of VR listening experience but just on headphones

colanderman 2y ago

Dolby Atmos is literally that, I think there are several physical formats that can support it. Ambisonics, especially higher-order ambisonics, also is capable of spatial panning, though it encodes the 3D soundfield rather than individual sound sources. Again, there are a few physical formats.

deleted 2y ago

[deleted]

mrandish 2y ago

In addition to Dolby Atmos there is MPEG-H 3D Audio. https://en.wikipedia.org/wiki/MPEG-H_3D_Audio https://en.wikipedia.org/wiki/MPEG-H_3D_Audio https://ieeexplore.ieee.org/document/7056445 https://ieeexplore.ieee.org/document/7056445

glenneroo 2y ago

This is probably a very odd and roundabout way of doing something akin to what you are asking for, but my recently released VR game DodgeALL[0] has a form of modding support for (wav/mono) audio files which can be uploaded (via SideQuest) to the headset. These are then assigned to "obstacles" which are launched at the player i.e. you could move your head around and hear spatialized audio files as they fly around the arena. You don't even need special headphones, the ones on the Quest do a good enough job, and Meta's Spatial audio SDK does a wonderful job of spatializing audio. I was originally using SteamAudio[1] but switched mainly due to moving from PC-VR to Standalone. Sorry if this sounds like some self-promotion - I specifically built this functionality[2] for my previous audio engineer so that he could quickly design and test sounds without the hassle of having to build/install an APK with every new audio file. [0]: https://www.DodgeALL.com https://www.DodgeALL.com (only for Quest 2/3/3s/Pro headsets) [1]: https://valvesoftware.github.io/steam-audio/ https://valvesoftware.github.io/steam-audio/ [2]: https://docs.google.com/document/d/1IFQjkroomHh8FocjfA-a55JPnrgpmSGkf2bXzZHEkLI/edit?tab=t.0 https://docs.google.com/document/d/1IFQjkroomHh8FocjfA-a55JP...