5 ms·

Spatial audio per se is not a scam, but its effective simulation is actually potentially rather complicated and I too worry about its successful implementation

Spatial audio per se is not a scam, but its effective simulation is actually potentially rather complicated and I too worry about its successful implementation by Apple… Funnily enough, I had plans for a system to implement spatial audio using smartphones/headphones originally back in 2006 and more recently wrote an app for iOS trying to implement it (…prototype/demo. that I made worked pretty well - used what I learnt working on games/interactive audio design…) I had toyed with starting a business myself over the years but life got in the way sort of… Before all this was announced I even (naively) wrote to Apple sort of roundabout-ly trying for a job in order to improve their spatial audio offering, believe it or not, but only got a polite sort of: ‘don’t call us we’ll call you’ kind of reply, which was understandable… (…don’t be thinking differently! :) Some students I taught/supervised/wrote a brief for a project of’s up at the university here subsequently started a company making a 3D binaural audio plugin thing - the business was bought by Facebook, whete they were afterwards employed implementing spatial audio into some of their offerings also… Anyhow, don’t want to bore anyone, but happy to answer any questions about it (do actually know what I’m talking about more or less in this area, as opposed to most stuff I read on HN.! :) P.S. Jens Blauert wrote (what used to be, at least) the standard reference on this: https://mitpress.mit.edu/books/spatial-hearing-revised-edition https://mitpress.mit.edu/books/spatial-hearing-revised-editi...

tinus_hn 5y ago
As far as I remember it’s a patent minefield
- Cybotron5000 5y ago
  If you can remember any further info. about this I’d be very interested.
  - tinus_hn 5y ago
    No details, I could look but I presume I wouldn’t be better than yourself
phiresky 5y ago
From what I understand, to really get spatial audio from stereo headphones, you need to use a HRTF (head-related transfer function) specific to the person. There's an open dataset of 50 different HRTFs [1] and a long video to compare them [2]. For me personally, only 2 of those 50 samples actually vaguely sound like audio is ever coming from in front of me. Left, right, and behind mostly works, but for most samples when it should be coming from in front of me it either comes from behind me or from above me. So it shouldn't be really possible to get good spatial audio without 3d-scanning a persons head or at least making them go through a calibration step where they rate "where is the sound coming from?" . And it definitely shouldn't be possible to get any good results by pre-mixing the audio down to stereo before a user-specific transform is applied. [1]: http://recherche.ircam.fr/equipes/salles/listen/index.html http://recherche.ircam.fr/equipes/salles/listen/index.html [2]: https://www.youtube.com/watch?v=VCXQp7swp5k https://www.youtube.com/watch?v=VCXQp7swp5k
- Cybotron5000 5y ago
  You’re absolutely right that the most effective simulation would use a custom HRTF for each person, as the shape of the head, shoulders, pinnae of the ear etc etc. all play some part... There are some headphones that attempt to go some way towards this by taking pictures of/otherwise ‘scanning’ the users ear (eg. https://electronics.sony.com/360-reality-audio https://electronics.sony.com/360-reality-audio https://mysofa.audio/ https://mysofa.audio/ …paper on the subject: https://www.aes.org/e-lib/browse.cfm?elib=19855 https://www.aes.org/e-lib/browse.cfm?elib=19855 …interestingly, for Apple’s future plans perhaps, ear canal used for biometric validation: https://dl.acm.org/doi/10.1145/3351239 https://dl.acm.org/doi/10.1145/3351239 ) The strongest ‘cue’ is probably the ‘interaural time delay’, which is easy enough to fake, but as you say, without fixed sources (imagine the calibration/setup of a VR headset) and subsequently taking into account that people actually turn/tilt their heads to better appreciate the direction a sound is coming from, as apparently Apple does relative to the device the tracks are playing from: https://appleinsider.com/articles/21/05/18/apples-spatial-audio-and-dolby-atmos-explained https://appleinsider.com/articles/21/05/18/apples-spatial-au... (in order to take best advantage of what you experienced where we can better locate left/right than forward/back...), the efficacy is reduced. Generally we are also much better at localizing sounds in the lateral plane than the vertical one (…evolutionarily perhaps this was more important in order to avoid being eaten by sabre-toothed tigers or something? :) There are other cues deriving from ‘head-shadow’ (somewhat like an eq.), reflection from shoulders and even reflections from the acoustic environment and so on that are subtler/more complicated… Traditionally dummy heads with anatomically ‘average’ characteristics and binaural microphones in the ‘ears’ have been used to make binaural recordings, but if you were to venture out with binaural microphones in your ears and then later play back the recordings the effect can be startlingly effective… [as an aside, when I posted the above comment in this thread a couple of days ago when it was first on HN, the title of the post was ‘Spatial Audio is a Scam’ :) ]

tinus_hn 5y ago

As far as I remember it’s a patent minefield

Cybotron5000 5y ago
If you can remember any further info. about this I’d be very interested.
- tinus_hn 5y ago
  No details, I could look but I presume I wouldn’t be better than yourself

phiresky 5y ago

From what I understand, to really get spatial audio from stereo headphones, you need to use a HRTF (head-related transfer function) specific to the person. There's an open dataset of 50 different HRTFs [1] and a long video to compare them [2]. For me personally, only 2 of those 50 samples actually vaguely sound like audio is ever coming from in front of me. Left, right, and behind mostly works, but for most samples when it should be coming from in front of me it either comes from behind me or from above me. So it shouldn't be really possible to get good spatial audio without 3d-scanning a persons head or at least making them go through a calibration step where they rate "where is the sound coming from?" . And it definitely shouldn't be possible to get any good results by pre-mixing the audio down to stereo before a user-specific transform is applied. [1]: http://recherche.ircam.fr/equipes/salles/listen/index.html http://recherche.ircam.fr/equipes/salles/listen/index.html [2]: https://www.youtube.com/watch?v=VCXQp7swp5k https://www.youtube.com/watch?v=VCXQp7swp5k

Cybotron5000 5y ago
You’re absolutely right that the most effective simulation would use a custom HRTF for each person, as the shape of the head, shoulders, pinnae of the ear etc etc. all play some part... There are some headphones that attempt to go some way towards this by taking pictures of/otherwise ‘scanning’ the users ear (eg. https://electronics.sony.com/360-reality-audio https://electronics.sony.com/360-reality-audio https://mysofa.audio/ https://mysofa.audio/ …paper on the subject: https://www.aes.org/e-lib/browse.cfm?elib=19855 https://www.aes.org/e-lib/browse.cfm?elib=19855 …interestingly, for Apple’s future plans perhaps, ear canal used for biometric validation: https://dl.acm.org/doi/10.1145/3351239 https://dl.acm.org/doi/10.1145/3351239 ) The strongest ‘cue’ is probably the ‘interaural time delay’, which is easy enough to fake, but as you say, without fixed sources (imagine the calibration/setup of a VR headset) and subsequently taking into account that people actually turn/tilt their heads to better appreciate the direction a sound is coming from, as apparently Apple does relative to the device the tracks are playing from: https://appleinsider.com/articles/21/05/18/apples-spatial-audio-and-dolby-atmos-explained https://appleinsider.com/articles/21/05/18/apples-spatial-au... (in order to take best advantage of what you experienced where we can better locate left/right than forward/back...), the efficacy is reduced. Generally we are also much better at localizing sounds in the lateral plane than the vertical one (…evolutionarily perhaps this was more important in order to avoid being eaten by sabre-toothed tigers or something? :) There are other cues deriving from ‘head-shadow’ (somewhat like an eq.), reflection from shoulders and even reflections from the acoustic environment and so on that are subtler/more complicated… Traditionally dummy heads with anatomically ‘average’ characteristics and binaural microphones in the ‘ears’ have been used to make binaural recordings, but if you were to venture out with binaural microphones in your ears and then later play back the recordings the effect can be startlingly effective… [as an aside, when I posted the above comment in this thread a couple of days ago when it was first on HN, the title of the post was ‘Spatial Audio is a Scam’ :) ]