12 ms·

The human ear detects half a millisecond delay in sound

242 points by pizza 5y ago

vbphprubyjsgo 5y ago
For some reason I thought this was about reaction time. Of course you can detect small variations in time. Even visually the eye will see differences between a light on for 1ms and one on for 10ms.
hoseja 5y ago
Yes, in fact, human ear can detect up to ~1/20000s delays in sound...
irjustin 5y ago
VLC and adding/removing the sound delay. Not half-millisecond timing, but man it got super annoying +/-5ms.
- gsich 5y ago
  Delay compared to what? Video? I doubt normal stereo audio files are out of sync.
  - jacquesm 5y ago
    The article is quite fluffy, but from what I gather it is about the difference in arrival time when a sound is perceived through multiple filters for different frequency bands, each of which has a programmable delay. One possible application for something like this would be to design a loudspeaker where the time-of-flight of the soundwave travelling from a tweeter to the listener would be the same as the same sound sent from a midrange speaker. Most designs do not adjust for such differences. The more interesting version - to me, at least - would be where you compare between the two arrivals at your left and right ear, which can do very small fractions of that even with relatively high pitched sounds allowing for sound source location based on phase.
    - tuatoru 5y ago
      > One possible application for something like this would be to design a loudspeaker where the time-of-flight of the soundwave travelling from a tweeter to the listener would be the same as the same sound sent from a midrange speaker. Most designs do not adjust for such differences. Most designs, true. Very high-end speakers do sometimes incorporate all-pass filters in parts of their crossovers to compensate for the set-back of the apparent sound source in woofers. But very few people tilt their speakers so that the distance from the midrange and woofer to their ears is exactly the same as that from the tweeter at the listening position, and getting complicated crossovers to be reliable at 100 watt power levels is very expensive. Time-of-flight compensation is trivial in the digital domain, so some active crossovers (which operate on line level signals, before the power amplifiers) also do this. The downside, of course, is needing one power amplifier per speaker cone, not just one per box, and the extra wires if using separate amplifier blocks rather than active speakers. The consensus seems to be that it's not worth the trouble in ordinary listening situations.
      - jacquesm 5y ago
        The one-amp-per-device is actually quite common in monitor speakers.
    - gsich 5y ago
      I think most speaker setups have a microphone that you place where the listener is to calibrate. Don't know if that involves delay measurements.
      - jacquesm 5y ago
        There was the Philips concept called MFB which integrated an accelerometer into the low frequency speakers in order to match the desired waveform with the one that actually went out. Quite a neat little concept, they are still somewhat popular in the HiFi scene. The microphone systems mostly calibrate for frequency loss across the spectrum, time-of-flight adjustments I haven't seen but it's quite possible they're out there somewhere, but they would require a pretty nifty delay mechanism or a digitization step and then turning the signal yet again back into analog which would probably cause more problems than it solved.
- globular-toast 5y ago
  I initially thought about synchronisation with video too (aka lip sync). However, I don't think they are talking about video but merely whether a difference is detectable (rather than acceptable). I suspect the threshold for lip sync acceptability is a lot higher than what they measured here. I would have thought the threshold was higher than 5ms, but I haven't done any rigorous testing.
  - drmpeg 5y ago
    See Figure 2 on page 4. https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.1359-1-199811-I!!PDF-E.pdf https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.1359-1-...
    - globular-toast 5y ago
      Thanks, that's really interesting, especially how video lag is more detectable than audio lag. I recently set up a display with a 130ms lag and this explains why it was so bad before I corrected the audio!
alex_smart 5y ago
This is news? :o
avnigo 5y ago
I guess that makes sense considering we use the delay of sounds reaching each ear separately to determine the direction sound comes from. Assuming the speed of sound is around 340 m/s, in half a millisecond it travels about 17 cm, which, I would guess, is larger than the average distance between our ears. So, using that rough estimation, I would cautiously extrapolate that we probably detect a delay under 0.5 ms, but I'd be interested to see what "detect" means exactly.
- smcl 5y ago
  That's a nice estimation, but I would have imagined direction is down to whether a sound is more muffled or quiet in one ear than the other.
  - Griffinsauce 5y ago
    That doesn't need to be the case when there are reflections.
    - smcl 5y ago
      I'm not sure I follow
      - jacquesm 5y ago
        A reflection can easily be louder than the original sound, but the longer time-of-flight allows the ear to distinguish between the two. That's how even in a hall with echoing walls you can still pinpoint a soundsource with relatively high accuracy.
        datameta 5y ago
        The first chapter of The Sound Book by Trevor Cox goes into great detail about this (but without getting deep into the math of it). Well worth the read for those interested in the acoustics of architecture and our perception of how it modifies the soundscape.
jacquesm 5y ago
It can do a whole lot better than that, it can detect phase changes at a very small fraction of a millisecond. It has to because that's how we determine direction of a sound source at low frequencies.
- quickthrower2 5y ago
  That it can do that with 2 ears and the latency between them signalling through the brain is amazing
  - da_chicken 5y ago
    Well, it's not just latency in many cases. If there's someone typing at the desk to your left, that noise has latency but it has different loudness in your left and right ears. The sound reaching your right ear has to make it's way around your head, so it sounds different not merely more delayed. Your brain has learned what all of that means.
    - phreeza 5y ago
      In addition to loudness and latency differences, there is even a frequency-specific effect due to the shape of the head and in particular your ear (your pinna to be precise). This is actually probably the reason why our ears have the weird shape they do. This is different for every person, which means that headphone-based simulated spatial audio can never be perfect.
  - tiagod 5y ago
    You can even hear the interference between two sine waves, without them actually "mixing" in the air, by feeding one to each ear. This doesn't work for every frequency, but when it does it sounds the same as it would if they were both played to both years, but it's happening in your brain! https://en.wikipedia.org/wiki/Beat_(acoustics)#Binaural_beats https://en.wikipedia.org/wiki/Beat_(acoustics)#Binaural_beat...
    - jacquesm 5y ago
BoxOfRain 5y ago
I spent lockdown coming up with a music podcast with a friend, it's amazing how much even a little delay can throw you off. I have a hybrid digital/analogue setup where an analogue mixer is fed either from a digital source by a DAC or an analogue source via my microphone or a turntable. The analogue audio then goes through some analogue processing equipment and is then fed into the ADC and recorded/streamed digitally. It's not the most efficient setup in the world, but I have a real thing for flashing lights and physical switches! If you talk into the mic with your headphones plugged into the analogue side then switch to monitoring the software instead, you can really notice the latency even though it's practically not that much. I read about a technology that's supposed to prevent angry customers from giving poor call centre employees a tirade of abuse by echoing their voices back at them on a very slight delay which is apparently quite intolerable, and I could believe it!
- Cyykratahk 5y ago
  This is known as Delayed Auditory Feedback https://en.wikipedia.org/wiki/Delayed_Auditory_Feedback https://en.wikipedia.org/wiki/Delayed_Auditory_Feedback
- toast0 5y ago
  > echoing their voices back at them on a very slight delay which is apparently quite intolerable, and I could believe it! Sometimes I get this effect on my normal calls, and it is pretty awful. Echo wasn't nearly so bad on circuit switched calls, but now that everything is digital and has sampling and codec delays, the echos come back so much slower.
erdewit 5y ago
> The sounds easiest to identify were a castanet, a percussion instrument, and short clicks. Not coincidentally, this is exactly the type of music where lossy encodings such as MP3, Ogg or AAC fail.
tuatoru 5y ago
> castanets and short clicks I was doing just this experiment three days ago. I have a new audio interface, and used audacity to try to establish its round-trip delay. I created a rhythm track using a click sound, connected one channel's playback to an input, and recorded it. The latency was about 35.1 ms. Audacity (the 2.x series) allows adjustment of latency compensation in whole milliseconds. Edit: I set the latency adjustment for -35 ms so there was only the 0.1-ish residual latency. Playing one channel of the original track and the recording together (after adjusting the channel balance for equal loudness) was quite disconcerting.
- FelipeCortez 5y ago
  Your buffer size is probably too high. Try 128 or 64
  - jacquesm 5y ago
    That is always going to be a compromise between the chance of a buffer underrun and annoying latency. Compared to analog digital audio solves some problems but introduces a whole raft of new ones.
nabla9 5y ago
>This work demonstrates how the group-delay response of headphones and loudspeakers can be perceptually tested, and leads to a better understanding of how audio systems should be equalized to avoid audible group-delay distortion.
- szszrk 5y ago
  I find all of this hilarious as what I've learned about audio production is that delays are used as equalizers, comb filtering effect used in practice. Not just on analogue devices, I was told once by (AFAIR?) Drumgizmo devs that digital as well! Btw. Drumgizmo is an amazing opensource drum plugin, that you mix like you would mix any real drum.
  - nabla9 5y ago
    You don't want your loudspeakers to have their own distortions. You want to transmit the distortions audio production made without add-ons.
junon 5y ago
Was just thinking about this the other day. In music production you can determine this; you have to be able to hear just a millisecond of difference sometimes to align things well, especially when you're working with instruments that have considerable delay (e.g. almost all Kontakt instruments).
- jacquesm 5y ago
  Essentially anything that is MIDI based or has an audio buffer somehwere.
  - junon 5y ago
    Yes but what I meant is that even though you place a midi note directly on a beat, many instruments still have a pretty considerably delay, in the order of 10-100ms.
    - jacquesm 5y ago
      Not only that, those delays themselves tend to change depending on how 'busy' the instrument is, for instance, whether you are using layered patches, multiple notes at once, different instruments at once and so on. It can change from one note to the next, sometimes without any clear hint as to why that is the case.
  - beardyw 5y ago
    And also with samples. If you take, say, a tuba the player must start blowing well before the beat so there is a natural breath you want to preserve. In midi there is no real way to preempt that except by playing the sample early as best as possible. I trained tuba player will just know instinctively.
NDizzle 5y ago
I remember way back when the original Xbox was out, and modded. Whatever video player it had let you adjust the sound delay for the (new at the time) mpeg4 rips of whatever movie or show you downloaded. It was like a light switch went off when you got the audio delay synced up perfectly. I guess I recall this vividly because 1) it was a huge problem and 2) the adjustment to audio delay was on the shoulder buttons, while fast forward and rewind were on the triggers. Such a large problem that the adjustment wasn’t buried in menus, but right there on the controller. Prime real estate.
- Guillaume86 5y ago
  It might have been XBMC (xbox media center). Great software and it still exists today on several platforms (PC, Android TV, etc) under the name of Kodi.
- Aerroon 5y ago
  Humans can ignore the desync between audio and video to about -125 ms to +45 ms. Ideally you don't want to be off by more than a frame or two though. If the desync goes out of these bounds then watching someone speak becomes uncomfortable.
  - nitrogen 5y ago
    Using a rough rule of thumb that one millisecond equals one foot, it kind of makes sense that we would be able to follow speech coming from someone up to 45 feet away. But the tolerance on the negative side is surprising.
cesaref 5y ago
It's not clear from the article, but I think what they are doing is delaying part of the frequency range of a signal. All this talk of negative delays is really confusing, it's just that they are delaying frequencies outside the range they are considering by a larger amount than within their range of interest. The overall effect is the same, there is going to have to be latency through the system. So, I presume, their listeners are hearing a timbral shift caused by the harmonics being advanced/retarded. This is not the same effect as feeding a delayed signal into one ear and a non-delayed into the other (the relative phase being used for locating a sound). They are I guess exploring how accurately they need to reproduce an impulse response to make an accurate transducer, and from the article, I believe they have concluded that they need to be more accurate than they previously thought. Given Genelec make a range of speakers with DSP for this sort of thing, it's I guess partly a marketing campaign to convince people that there is some benefit from their DSP corrected monitors. Of course, for a producer to have an accurate monitor is useful, but if the listening public have non-aligned drivers, the sonic benefits from worrying about this stuff are of somewhat limited value.
- Ballas 5y ago
  Yes, the way they describe it with "time traveling" is especially confusing. I still don't understand exactly what their claim is or what they intend on selling.
- xen2xen1 5y ago
  Takes me a minute to remember monitors and speakers can be the same exact thing, and not a monitor with speakers built in (like a TV).
- dr_dshiv 5y ago
  Right, it seems people can detect the .5ms phase difference in the timbre. It is a noticeable difference, but they aren't detecting the delay, per se.
- TrackerFF 5y ago
  I believe that the BBE Sonic Maximizer (http://www.bbesound.com/products/sonic-maximizers/default.aspx http://www.bbesound.com/products/sonic-maximizers/default.as...) products fix that. They (time) align the components in the frequency spectrum, for lack of a better description. edit: Or rather, it works as a dynamic EQ to dampen/amplify certain frequencies. Most guitarists that have a rack-setup, probably have or have tried these - in practice, the effect is just a more smoothed sound.
zwieback 5y ago
So what does that mean for a large symphony where musicians are spread out much farther than sound travels in a millisecond? Let's say it's 10m, that would be about 29ms. I've heard really good musicians and conductors compensate for that but I'm doubtful. So we must be talking about more subtle effects, like phase shifts, and not about how the waveform envelopes line up.
- herendin2 5y ago
  It's one reason that a conductor sends visual cues to the whole orchestra at the speed of light. That's the base clock source
  - oscardssmith 5y ago
    one interesting thing is that you will fairly frequently see professional orchestras be out of phase with the conductor, but everything still works out.
    - zwieback 5y ago
      Does a really good conductor train each section to come in on different parts of the visual cue then, e.g. double bass would play before the violins in the front? I guess you can't precisely solve this problem for more than a single listener.
      - InitialLastName 5y ago
        If the listener is relatively further away from the orchestra than the orchestra members are to each other, it mostly all balances out, especially when you add the room and bandshell reverberation (which tends to wash the note onsets to some degree). This is also helped by the instruments in the back of the orchestra favoring the low frequencies, where the listener has less access to precise timing information.
      - toast0 5y ago
        As a (former) bass player, you need to start a bit early, yeah. But that's not so much for distance but because it takes longer for the sound to start. Pipe organ is of course much worse. I would do one concert a year sitting next to an organist... The keyboard noises and motion from him playing way before me were pretty distracting. And when we stopped during rehersal, he'd stop on the keys right away, but the organ would keep going for a bit.
deleted 5y ago
[deleted]
tintt 5y ago
Given the speed of sound, 0.5ms delay occurs when you talk to anybody more than half-a-feet away
archibaldJ 5y ago
I was playing with my synthesiser the other day and realized what I was really doing is using my eardrum to explore the search space of oscillation patterns, and their combinatorial sets. Music in this sense is the class of all time series of combinations of such patterns that we find resonating as we are able to encode and decode emotions and feelings from these sounds (as well as appreaciation for them). I wonder what other cool things we can do with this singal processing ability of ours as we enter the age of brain computer interfaces and psychedelics.
LatteLazy 5y ago
Isn't this how we hear directionally? 300m/s means 150mm distance. Ears are only about 250mm apart, so if you couldn't hear a 0.5ms diff you'd have no idea where sound was coming from right?
- _Microft 5y ago
  A timing difference is not the only way to tell from which side a sound came. Since the ears are at opposite sides of the head and the external ears have particular (but unchanging [0]) shapes, a sound will be changed in a different but predictable way depending on the direction it comes from. This effect also helps with detecting whether a sound comes from the front/back or top/bottom of the listener. https://en.wikipedia.org/wiki/Head-related_transfer_function https://en.wikipedia.org/wiki/Head-related_transfer_function [0] Change the environment around the ear a bit, e.g. by putting a hand relatively close to the external ear and listen how doing that changes sounds.
  - LatteLazy 5y ago
    Thanks! That was a good read!.
AYBABTME 5y ago
Something I've been wondering is, assuming that the brain works at a couple dozen hertz frequency, how can we perform any sub-10ms task?
- chousuke 5y ago
  Wouldn't the brain send the signals pre-emptively such that they trigger the action at the correct time, accounting for delays? I don't think it's possible to perform any conscious action that quickly, but if you've trained to perform some pattern, your brain would learn to compensate for any processing delays.
bob1029 5y ago
Its amazing how much difference .5ms can make in stereo imaging. Any decent home audio receiver has the ability to set the distance of each speaker individually so that you can compensate for physical placement constraints. If you get a stereo pair perfectly locked in on delay to eardrums, you can produce an extremely compelling listening experience for those in a very specific region of the room. Finding out about all this can be revolutionary for some music enthusiasts. Once you get a good listening setup (or headphones), you start going through old things to see how the "stage presence" sounds, or if you are now able to physically place each instrument in the virtual space.
- JKCalhoun 5y ago
  I built a pair of full-range speakers and was therefore introduced to that effect — an effect I had only experienced before with headphones. No special modern receiver in my case, just simple speakers. It seems the 2-way, 3-way speakers I grew up with kill stereo imaging. (Never mind the crossovers eat power and diminish the efficiency of the speaker — requiring a higher current amp, etc... Lovely what a small ½ Watt tube amp and a pair of full-range drivers can sound like ... and throw in a sub.)
  - bob1029 5y ago
    > it seems the 2-way, 3-way speakers I grew up with kill stereo imaging. Yeah there are ways to build crossover networks that can minimize these issues (phase shift) across the frequency range. The most ideal crossover would dissipate 100% of the undesired acoustic power as heat rather than storing it as reactive energy in inductors, but the frequency domain be a tricky beast to dance with. The best overall approach is probably the 4th order Linkwitz-Riley filter: https://en.wikipedia.org/wiki/Linkwitz%E2%80%93Riley_filter#Fourth-order_Linkwitz%E2%80%93Riley_crossover_(LR4,_LR-4) https://en.wikipedia.org/wiki/Linkwitz%E2%80%93Riley_filter#...
- function_seven 5y ago
  My dad bought an Isuzu Impulse in 1985. It had a fancy OEM stereo system with Technics branding, and a button for the driver to toggle the imaging. The button was on the 7-band graphic EQ that game with the car. (I don't think I've ever seen another factory stereo with physical EQ sliders like that.) It was magic, if you were sitting in the driver's seat. Toggling that button made the sound switch from "okay" to "magically spatial." I'm pretty sure that lead to a conversation about how we locate sounds. Also wonder how it was implemented in 1985. I doubt there were DSP chips in there. What's the "simple" way of adding delay to some speakers?
1-6 5y ago
There goes my ambition in syncing musicians remotely. I guess introducing a little bit of delay and using GNSS clocks would sort of help.
JoeAltmaier 5y ago
Oh I knew this. Doing the Sococo media mixer, we understood that audio sensitivity is orders of magnitude more critical that video. Heck, video could stall and delay and be fuzzy and folks didn't say much. But miss a single sample of audio at 44KHz and folks would speak up.
vernie 5y ago
There's a saying in real-time audio processing that "the ears don't blink"
mauvehaus 5y ago
Highly recommended: Seeing The Visitors at ICA Boston. It's a dozen-ish musicians playing in separate rooms of a big house where they can't see each other and can only hear each other over headphones. It's presented as nine channels of video, each with their own audio. The effect is pretty wild and magical. The music, I suspect, is intentionally written to not require precise timing, and part of the charm of the piece is the musicians feeling each other out as they play. It definitely plays with your expectations of what constitutes music and how you hear timing in music. The ICA owns the piece (a copy of the piece?), but it isn't currently on display :-( https://www.icaboston.org/exhibitions/ragnar-kjartansson-visitors-1 https://www.icaboston.org/exhibitions/ragnar-kjartansson-vis...
- bazeblackwood 5y ago
  I saw it at the Broad in LA, and it was mind-blowing.
matchagaucho 5y ago
Hearing predators approaching from behind is the evolutioning justification for this trait. Oculus, and other 3D platforms, utilize HRTF (head related transfer functions) with subtle millisecond phase shifts to localize objects.
phreeza 5y ago
Humans can do way better than that, somewhere in the 20 microsecond range. See for example Mills (1958). https://scholar.google.com/scholar?cluster=1140169613625988666&hl=de&as_sdt=0,5#d=gs_qabs&u=%23p%3DOhbfAzmy0g8J https://scholar.google.com/scholar?cluster=11401696136259886... Edit: in fact, humans are among the best animals at all at this task, I think it is not really clear why. They are on par with highly specialist auditory hunters like barn owls. May be that humans can use their higher cognitive faculties to somehow get better at the task than the average animal but it's not clear to me how that would happen.
- andi999 5y ago
  Since humans rely on groups and communication it is probably to hear/locate who in a group is talking.
- chiefalchemist 5y ago
  I would imagine this might not be an ear mechanics issue, but a brain processing issue. Some animals have dedicated "processor" and other do not. While in the larger more agile human brain it's simply one of many features. Maybe?
webdevtway87 5y ago
Meanwhile, Android Chrome deliberately introduces a ~300 millisecond audio delay to make web games impossible and force game devs to use the Play Store so that Google gets its pound of flesh. The superpowered[1] database has many measurements showing how bad Android is. Those of you with Android devices can try this ancient web audio demo to see that the issue remains unfixed all these years later, except on certain select premium devices (certain Samsungs, etc.). https://webaudiodemos.appspot.com/TouchPad/index.html https://webaudiodemos.appspot.com/TouchPad/index.html You should hear sound the instant a rectangle is touched/clicked and you will with a desktop browser but not with most Androids. Apple also employs dark patterns on iOS Safari to make web games impossible: 1. No full screen. ("But the user wouldn't know how to exit full-screen - unless they buy from the App Store! We also plan to make full screen PWAs impossible because of, like, Reasons and totally not because of our 30% App Store cut!") 2. Slow WebGL and no WebGL 2.0. ("But we need to run extra shader validation security checks every single frame even though we technically only need to do it once up front! But this extra Privacy and Security[TM] is totally unnecessary if you buy from the App Store!") 3. No device motion tilt control. ("Allowing the user to consent to tilt control would violate the user's Privacy and Security[TM] - unless they buy from the App Store!") 4. No JS debug console unless you register an Apple developer account, download 10+ gig Xcode on a Mac-only computer, and ask Apple for authorization to debug javascript on your own freakin iPhone you paid for with your own damn money. I was forced to give up using "open" web technologies for games and instead go with native code game engines (Godot, Unreal, Unity, ...). [1] https://superpowered.com/latency https://superpowered.com/latency
abeppu 5y ago
I'm confused about why the methodology that they talk about is innovative. They talk about shifting when a sound event of a given frequency arrives, relative to another sound. But with digital audio, esp that you can prepare in advance, isn't this pretty simple? Like, given signals A and B, apply a band-pass filter to A (for the desired frequency window), truncate off the first k samples based on the time shift of interest, and add to B. Even if you do need it in real time, why isn't this equivalent to delaying B by k samples? I must be missing something, because the researcher makes this sound like a cool new technique.
AceyMan 5y ago
Most any audio engineer or audiophile will confess the vast majority of a music's spaciality is dictated by the room. For the 3-nines of us who cannot dedicate a room for ideal echo, decay,absorption making all the temporal DSP tech overtake the influence of the room is a sysyphean task. So when my son was born profoundly deaf in one ear, as an audiophile, I soon realized "time to prove the merits of single channel." So far that just means my everyday workstation has a single Tivoli Model One radio as the output transducer and it sounds good enough for watching films and playing Spotify on the desktop. My intended future hifi is a single Lowther speaker cabinet driven by a modestly powered single ended tube triode amplifier. Long live monaural! /Acey
LargoLasskhyfv 5y ago
When walking along the fence on the right side of this street [1] https://www.google.de/maps/@53.5546251,9.8651002,3a,75y,192.33h,90t/data=!3m6!1e1!3m4!1s-pNeXAXSpeJcy9-YkD8tdA!2e0!7i13312!8i6656 https://www.google.de/maps/@53.5546251,9.8651002,3a,75y,192.... it feels like I could count every single echo of my steps from every single element of the fence if I could count fast enough. Similar when riding my bicycle slowly down there (rarely, because downwards, so I'm doing 50 kph there usually), ring my bell, or click my tongue. edit: I mean, I'm hearing the difference in the delay depending on the distance.
France_is_bacon 5y ago
I don't understand this. Delayed how? Does this mean if you are watching a video, and the person is talking, that a delay of milliseconds between the person's lips moving and when you hear it? But what does that have to do with castinets? I wouldn't be able to see them they probably move so fast. But they don't mention video matched against audo delay, that I read. If you start a song and they delay it by 10 seconds, it just starts 10 seconds later, how would you even know there's a delay? Or if there's a delay in the middle of a song, how would you know any difference, as maybe that's how the music piece is supposed to sound. What am I not getting?

242 points by pizza 5y ago

For some reason I thought this was about reaction time. Of course you can detect small variations in time. Even visually the eye will see differences between a light on for 1ms and one on for 10ms.

hoseja 5y ago

Yes, in fact, human ear can detect up to ~1/20000s delays in sound...

irjustin 5y ago

VLC and adding/removing the sound delay. Not half-millisecond timing, but man it got super annoying +/-5ms.

gsich 5y ago
Delay compared to what? Video? I doubt normal stereo audio files are out of sync.
- jacquesm 5y ago
  The article is quite fluffy, but from what I gather it is about the difference in arrival time when a sound is perceived through multiple filters for different frequency bands, each of which has a programmable delay. One possible application for something like this would be to design a loudspeaker where the time-of-flight of the soundwave travelling from a tweeter to the listener would be the same as the same sound sent from a midrange speaker. Most designs do not adjust for such differences. The more interesting version - to me, at least - would be where you compare between the two arrivals at your left and right ear, which can do very small fractions of that even with relatively high pitched sounds allowing for sound source location based on phase.
  - tuatoru 5y ago
    > One possible application for something like this would be to design a loudspeaker where the time-of-flight of the soundwave travelling from a tweeter to the listener would be the same as the same sound sent from a midrange speaker. Most designs do not adjust for such differences. Most designs, true. Very high-end speakers do sometimes incorporate all-pass filters in parts of their crossovers to compensate for the set-back of the apparent sound source in woofers. But very few people tilt their speakers so that the distance from the midrange and woofer to their ears is exactly the same as that from the tweeter at the listening position, and getting complicated crossovers to be reliable at 100 watt power levels is very expensive. Time-of-flight compensation is trivial in the digital domain, so some active crossovers (which operate on line level signals, before the power amplifiers) also do this. The downside, of course, is needing one power amplifier per speaker cone, not just one per box, and the extra wires if using separate amplifier blocks rather than active speakers. The consensus seems to be that it's not worth the trouble in ordinary listening situations.
    - jacquesm 5y ago
      The one-amp-per-device is actually quite common in monitor speakers.
  - gsich 5y ago
    I think most speaker setups have a microphone that you place where the listener is to calibrate. Don't know if that involves delay measurements.
    - jacquesm 5y ago
      There was the Philips concept called MFB which integrated an accelerometer into the low frequency speakers in order to match the desired waveform with the one that actually went out. Quite a neat little concept, they are still somewhat popular in the HiFi scene. The microphone systems mostly calibrate for frequency loss across the spectrum, time-of-flight adjustments I haven't seen but it's quite possible they're out there somewhere, but they would require a pretty nifty delay mechanism or a digitization step and then turning the signal yet again back into analog which would probably cause more problems than it solved.
globular-toast 5y ago
I initially thought about synchronisation with video too (aka lip sync). However, I don't think they are talking about video but merely whether a difference is detectable (rather than acceptable). I suspect the threshold for lip sync acceptability is a lot higher than what they measured here. I would have thought the threshold was higher than 5ms, but I haven't done any rigorous testing.
- drmpeg 5y ago
  See Figure 2 on page 4. https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.1359-1-199811-I!!PDF-E.pdf https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.1359-1-...
  - globular-toast 5y ago
    Thanks, that's really interesting, especially how video lag is more detectable than audio lag. I recently set up a display with a 130ms lag and this explains why it was so bad before I corrected the audio!

alex_smart 5y ago

This is news? :o

avnigo 5y ago

I guess that makes sense considering we use the delay of sounds reaching each ear separately to determine the direction sound comes from. Assuming the speed of sound is around 340 m/s, in half a millisecond it travels about 17 cm, which, I would guess, is larger than the average distance between our ears. So, using that rough estimation, I would cautiously extrapolate that we probably detect a delay under 0.5 ms, but I'd be interested to see what "detect" means exactly.

smcl 5y ago
That's a nice estimation, but I would have imagined direction is down to whether a sound is more muffled or quiet in one ear than the other.
- Griffinsauce 5y ago
  That doesn't need to be the case when there are reflections.
  - smcl 5y ago
    I'm not sure I follow
    - jacquesm 5y ago
      A reflection can easily be louder than the original sound, but the longer time-of-flight allows the ear to distinguish between the two. That's how even in a hall with echoing walls you can still pinpoint a soundsource with relatively high accuracy.
      - datameta 5y ago
        The first chapter of The Sound Book by Trevor Cox goes into great detail about this (but without getting deep into the math of it). Well worth the read for those interested in the acoustics of architecture and our perception of how it modifies the soundscape.

jacquesm 5y ago

It can do a whole lot better than that, it can detect phase changes at a very small fraction of a millisecond. It has to because that's how we determine direction of a sound source at low frequencies.

quickthrower2 5y ago
That it can do that with 2 ears and the latency between them signalling through the brain is amazing
- da_chicken 5y ago
  Well, it's not just latency in many cases. If there's someone typing at the desk to your left, that noise has latency but it has different loudness in your left and right ears. The sound reaching your right ear has to make it's way around your head, so it sounds different not merely more delayed. Your brain has learned what all of that means.
  - phreeza 5y ago
    In addition to loudness and latency differences, there is even a frequency-specific effect due to the shape of the head and in particular your ear (your pinna to be precise). This is actually probably the reason why our ears have the weird shape they do. This is different for every person, which means that headphone-based simulated spatial audio can never be perfect.
- tiagod 5y ago
  You can even hear the interference between two sine waves, without them actually "mixing" in the air, by feeding one to each ear. This doesn't work for every frequency, but when it does it sounds the same as it would if they were both played to both years, but it's happening in your brain! https://en.wikipedia.org/wiki/Beat_(acoustics)#Binaural_beats https://en.wikipedia.org/wiki/Beat_(acoustics)#Binaural_beat...
  - jacquesm 5y ago

BoxOfRain 5y ago

I spent lockdown coming up with a music podcast with a friend, it's amazing how much even a little delay can throw you off. I have a hybrid digital/analogue setup where an analogue mixer is fed either from a digital source by a DAC or an analogue source via my microphone or a turntable. The analogue audio then goes through some analogue processing equipment and is then fed into the ADC and recorded/streamed digitally. It's not the most efficient setup in the world, but I have a real thing for flashing lights and physical switches! If you talk into the mic with your headphones plugged into the analogue side then switch to monitoring the software instead, you can really notice the latency even though it's practically not that much. I read about a technology that's supposed to prevent angry customers from giving poor call centre employees a tirade of abuse by echoing their voices back at them on a very slight delay which is apparently quite intolerable, and I could believe it!

Cyykratahk 5y ago
This is known as Delayed Auditory Feedback https://en.wikipedia.org/wiki/Delayed_Auditory_Feedback https://en.wikipedia.org/wiki/Delayed_Auditory_Feedback
toast0 5y ago
> echoing their voices back at them on a very slight delay which is apparently quite intolerable, and I could believe it! Sometimes I get this effect on my normal calls, and it is pretty awful. Echo wasn't nearly so bad on circuit switched calls, but now that everything is digital and has sampling and codec delays, the echos come back so much slower.

erdewit 5y ago

> The sounds easiest to identify were a castanet, a percussion instrument, and short clicks. Not coincidentally, this is exactly the type of music where lossy encodings such as MP3, Ogg or AAC fail.

tuatoru 5y ago

> castanets and short clicks I was doing just this experiment three days ago. I have a new audio interface, and used audacity to try to establish its round-trip delay. I created a rhythm track using a click sound, connected one channel's playback to an input, and recorded it. The latency was about 35.1 ms. Audacity (the 2.x series) allows adjustment of latency compensation in whole milliseconds. Edit: I set the latency adjustment for -35 ms so there was only the 0.1-ish residual latency. Playing one channel of the original track and the recording together (after adjusting the channel balance for equal loudness) was quite disconcerting.

FelipeCortez 5y ago
Your buffer size is probably too high. Try 128 or 64
- jacquesm 5y ago
  That is always going to be a compromise between the chance of a buffer underrun and annoying latency. Compared to analog digital audio solves some problems but introduces a whole raft of new ones.

nabla9 5y ago

>This work demonstrates how the group-delay response of headphones and loudspeakers can be perceptually tested, and leads to a better understanding of how audio systems should be equalized to avoid audible group-delay distortion.

szszrk 5y ago
I find all of this hilarious as what I've learned about audio production is that delays are used as equalizers, comb filtering effect used in practice. Not just on analogue devices, I was told once by (AFAIR?) Drumgizmo devs that digital as well! Btw. Drumgizmo is an amazing opensource drum plugin, that you mix like you would mix any real drum.
- nabla9 5y ago
  You don't want your loudspeakers to have their own distortions. You want to transmit the distortions audio production made without add-ons.

junon 5y ago

Was just thinking about this the other day. In music production you can determine this; you have to be able to hear just a millisecond of difference sometimes to align things well, especially when you're working with instruments that have considerable delay (e.g. almost all Kontakt instruments).

jacquesm 5y ago
Essentially anything that is MIDI based or has an audio buffer somehwere.
- junon 5y ago
  Yes but what I meant is that even though you place a midi note directly on a beat, many instruments still have a pretty considerably delay, in the order of 10-100ms.
  - jacquesm 5y ago
    Not only that, those delays themselves tend to change depending on how 'busy' the instrument is, for instance, whether you are using layered patches, multiple notes at once, different instruments at once and so on. It can change from one note to the next, sometimes without any clear hint as to why that is the case.
- beardyw 5y ago
  And also with samples. If you take, say, a tuba the player must start blowing well before the beat so there is a natural breath you want to preserve. In midi there is no real way to preempt that except by playing the sample early as best as possible. I trained tuba player will just know instinctively.

NDizzle 5y ago

I remember way back when the original Xbox was out, and modded. Whatever video player it had let you adjust the sound delay for the (new at the time) mpeg4 rips of whatever movie or show you downloaded. It was like a light switch went off when you got the audio delay synced up perfectly. I guess I recall this vividly because 1) it was a huge problem and 2) the adjustment to audio delay was on the shoulder buttons, while fast forward and rewind were on the triggers. Such a large problem that the adjustment wasn’t buried in menus, but right there on the controller. Prime real estate.

Guillaume86 5y ago
It might have been XBMC (xbox media center). Great software and it still exists today on several platforms (PC, Android TV, etc) under the name of Kodi.
Aerroon 5y ago
Humans can ignore the desync between audio and video to about -125 ms to +45 ms. Ideally you don't want to be off by more than a frame or two though. If the desync goes out of these bounds then watching someone speak becomes uncomfortable.
- nitrogen 5y ago
  Using a rough rule of thumb that one millisecond equals one foot, it kind of makes sense that we would be able to follow speech coming from someone up to 45 feet away. But the tolerance on the negative side is surprising.

cesaref 5y ago

It's not clear from the article, but I think what they are doing is delaying part of the frequency range of a signal. All this talk of negative delays is really confusing, it's just that they are delaying frequencies outside the range they are considering by a larger amount than within their range of interest. The overall effect is the same, there is going to have to be latency through the system. So, I presume, their listeners are hearing a timbral shift caused by the harmonics being advanced/retarded. This is not the same effect as feeding a delayed signal into one ear and a non-delayed into the other (the relative phase being used for locating a sound). They are I guess exploring how accurately they need to reproduce an impulse response to make an accurate transducer, and from the article, I believe they have concluded that they need to be more accurate than they previously thought. Given Genelec make a range of speakers with DSP for this sort of thing, it's I guess partly a marketing campaign to convince people that there is some benefit from their DSP corrected monitors. Of course, for a producer to have an accurate monitor is useful, but if the listening public have non-aligned drivers, the sonic benefits from worrying about this stuff are of somewhat limited value.

Ballas 5y ago
Yes, the way they describe it with "time traveling" is especially confusing. I still don't understand exactly what their claim is or what they intend on selling.
xen2xen1 5y ago
Takes me a minute to remember monitors and speakers can be the same exact thing, and not a monitor with speakers built in (like a TV).
dr_dshiv 5y ago
Right, it seems people can detect the .5ms phase difference in the timbre. It is a noticeable difference, but they aren't detecting the delay, per se.
TrackerFF 5y ago
I believe that the BBE Sonic Maximizer (http://www.bbesound.com/products/sonic-maximizers/default.aspx http://www.bbesound.com/products/sonic-maximizers/default.as...) products fix that. They (time) align the components in the frequency spectrum, for lack of a better description. edit: Or rather, it works as a dynamic EQ to dampen/amplify certain frequencies. Most guitarists that have a rack-setup, probably have or have tried these - in practice, the effect is just a more smoothed sound.

zwieback 5y ago

So what does that mean for a large symphony where musicians are spread out much farther than sound travels in a millisecond? Let's say it's 10m, that would be about 29ms. I've heard really good musicians and conductors compensate for that but I'm doubtful. So we must be talking about more subtle effects, like phase shifts, and not about how the waveform envelopes line up.

herendin2 5y ago
It's one reason that a conductor sends visual cues to the whole orchestra at the speed of light. That's the base clock source
- oscardssmith 5y ago
  one interesting thing is that you will fairly frequently see professional orchestras be out of phase with the conductor, but everything still works out.
  - zwieback 5y ago
    Does a really good conductor train each section to come in on different parts of the visual cue then, e.g. double bass would play before the violins in the front? I guess you can't precisely solve this problem for more than a single listener.
    - InitialLastName 5y ago
      If the listener is relatively further away from the orchestra than the orchestra members are to each other, it mostly all balances out, especially when you add the room and bandshell reverberation (which tends to wash the note onsets to some degree). This is also helped by the instruments in the back of the orchestra favoring the low frequencies, where the listener has less access to precise timing information.
    - toast0 5y ago
      As a (former) bass player, you need to start a bit early, yeah. But that's not so much for distance but because it takes longer for the sound to start. Pipe organ is of course much worse. I would do one concert a year sitting next to an organist... The keyboard noises and motion from him playing way before me were pretty distracting. And when we stopped during rehersal, he'd stop on the keys right away, but the organ would keep going for a bit.

deleted 5y ago

[deleted]

tintt 5y ago

Given the speed of sound, 0.5ms delay occurs when you talk to anybody more than half-a-feet away

archibaldJ 5y ago

I was playing with my synthesiser the other day and realized what I was really doing is using my eardrum to explore the search space of oscillation patterns, and their combinatorial sets. Music in this sense is the class of all time series of combinations of such patterns that we find resonating as we are able to encode and decode emotions and feelings from these sounds (as well as appreaciation for them). I wonder what other cool things we can do with this singal processing ability of ours as we enter the age of brain computer interfaces and psychedelics.

LatteLazy 5y ago

Isn't this how we hear directionally? 300m/s means 150mm distance. Ears are only about 250mm apart, so if you couldn't hear a 0.5ms diff you'd have no idea where sound was coming from right?

_Microft 5y ago
A timing difference is not the only way to tell from which side a sound came. Since the ears are at opposite sides of the head and the external ears have particular (but unchanging [0]) shapes, a sound will be changed in a different but predictable way depending on the direction it comes from. This effect also helps with detecting whether a sound comes from the front/back or top/bottom of the listener. https://en.wikipedia.org/wiki/Head-related_transfer_function https://en.wikipedia.org/wiki/Head-related_transfer_function [0] Change the environment around the ear a bit, e.g. by putting a hand relatively close to the external ear and listen how doing that changes sounds.
- LatteLazy 5y ago
  Thanks! That was a good read!.

AYBABTME 5y ago

Something I've been wondering is, assuming that the brain works at a couple dozen hertz frequency, how can we perform any sub-10ms task?

chousuke 5y ago
Wouldn't the brain send the signals pre-emptively such that they trigger the action at the correct time, accounting for delays? I don't think it's possible to perform any conscious action that quickly, but if you've trained to perform some pattern, your brain would learn to compensate for any processing delays.

bob1029 5y ago

Its amazing how much difference .5ms can make in stereo imaging. Any decent home audio receiver has the ability to set the distance of each speaker individually so that you can compensate for physical placement constraints. If you get a stereo pair perfectly locked in on delay to eardrums, you can produce an extremely compelling listening experience for those in a very specific region of the room. Finding out about all this can be revolutionary for some music enthusiasts. Once you get a good listening setup (or headphones), you start going through old things to see how the "stage presence" sounds, or if you are now able to physically place each instrument in the virtual space.

JKCalhoun 5y ago
I built a pair of full-range speakers and was therefore introduced to that effect — an effect I had only experienced before with headphones. No special modern receiver in my case, just simple speakers. It seems the 2-way, 3-way speakers I grew up with kill stereo imaging. (Never mind the crossovers eat power and diminish the efficiency of the speaker — requiring a higher current amp, etc... Lovely what a small ½ Watt tube amp and a pair of full-range drivers can sound like ... and throw in a sub.)
- bob1029 5y ago
  > it seems the 2-way, 3-way speakers I grew up with kill stereo imaging. Yeah there are ways to build crossover networks that can minimize these issues (phase shift) across the frequency range. The most ideal crossover would dissipate 100% of the undesired acoustic power as heat rather than storing it as reactive energy in inductors, but the frequency domain be a tricky beast to dance with. The best overall approach is probably the 4th order Linkwitz-Riley filter: https://en.wikipedia.org/wiki/Linkwitz%E2%80%93Riley_filter#Fourth-order_Linkwitz%E2%80%93Riley_crossover_(LR4,_LR-4) https://en.wikipedia.org/wiki/Linkwitz%E2%80%93Riley_filter#...
function_seven 5y ago
My dad bought an Isuzu Impulse in 1985. It had a fancy OEM stereo system with Technics branding, and a button for the driver to toggle the imaging. The button was on the 7-band graphic EQ that game with the car. (I don't think I've ever seen another factory stereo with physical EQ sliders like that.) It was magic, if you were sitting in the driver's seat. Toggling that button made the sound switch from "okay" to "magically spatial." I'm pretty sure that lead to a conversation about how we locate sounds. Also wonder how it was implemented in 1985. I doubt there were DSP chips in there. What's the "simple" way of adding delay to some speakers?

1-6 5y ago

There goes my ambition in syncing musicians remotely. I guess introducing a little bit of delay and using GNSS clocks would sort of help.

JoeAltmaier 5y ago

Oh I knew this. Doing the Sococo media mixer, we understood that audio sensitivity is orders of magnitude more critical that video. Heck, video could stall and delay and be fuzzy and folks didn't say much. But miss a single sample of audio at 44KHz and folks would speak up.

vernie 5y ago

There's a saying in real-time audio processing that "the ears don't blink"

mauvehaus 5y ago

Highly recommended: Seeing The Visitors at ICA Boston. It's a dozen-ish musicians playing in separate rooms of a big house where they can't see each other and can only hear each other over headphones. It's presented as nine channels of video, each with their own audio. The effect is pretty wild and magical. The music, I suspect, is intentionally written to not require precise timing, and part of the charm of the piece is the musicians feeling each other out as they play. It definitely plays with your expectations of what constitutes music and how you hear timing in music. The ICA owns the piece (a copy of the piece?), but it isn't currently on display :-( https://www.icaboston.org/exhibitions/ragnar-kjartansson-visitors-1 https://www.icaboston.org/exhibitions/ragnar-kjartansson-vis...

bazeblackwood 5y ago
I saw it at the Broad in LA, and it was mind-blowing.

matchagaucho 5y ago

Hearing predators approaching from behind is the evolutioning justification for this trait. Oculus, and other 3D platforms, utilize HRTF (head related transfer functions) with subtle millisecond phase shifts to localize objects.

phreeza 5y ago

Humans can do way better than that, somewhere in the 20 microsecond range. See for example Mills (1958). https://scholar.google.com/scholar?cluster=1140169613625988666&hl=de&as_sdt=0,5#d=gs_qabs&u=%23p%3DOhbfAzmy0g8J https://scholar.google.com/scholar?cluster=11401696136259886... Edit: in fact, humans are among the best animals at all at this task, I think it is not really clear why. They are on par with highly specialist auditory hunters like barn owls. May be that humans can use their higher cognitive faculties to somehow get better at the task than the average animal but it's not clear to me how that would happen.

andi999 5y ago
Since humans rely on groups and communication it is probably to hear/locate who in a group is talking.
chiefalchemist 5y ago
I would imagine this might not be an ear mechanics issue, but a brain processing issue. Some animals have dedicated "processor" and other do not. While in the larger more agile human brain it's simply one of many features. Maybe?

webdevtway87 5y ago

Meanwhile, Android Chrome deliberately introduces a ~300 millisecond audio delay to make web games impossible and force game devs to use the Play Store so that Google gets its pound of flesh. The superpowered[1] database has many measurements showing how bad Android is. Those of you with Android devices can try this ancient web audio demo to see that the issue remains unfixed all these years later, except on certain select premium devices (certain Samsungs, etc.). https://webaudiodemos.appspot.com/TouchPad/index.html https://webaudiodemos.appspot.com/TouchPad/index.html You should hear sound the instant a rectangle is touched/clicked and you will with a desktop browser but not with most Androids. Apple also employs dark patterns on iOS Safari to make web games impossible: 1. No full screen. ("But the user wouldn't know how to exit full-screen - unless they buy from the App Store! We also plan to make full screen PWAs impossible because of, like, Reasons and totally not because of our 30% App Store cut!") 2. Slow WebGL and no WebGL 2.0. ("But we need to run extra shader validation security checks every single frame even though we technically only need to do it once up front! But this extra Privacy and Security[TM] is totally unnecessary if you buy from the App Store!") 3. No device motion tilt control. ("Allowing the user to consent to tilt control would violate the user's Privacy and Security[TM] - unless they buy from the App Store!") 4. No JS debug console unless you register an Apple developer account, download 10+ gig Xcode on a Mac-only computer, and ask Apple for authorization to debug javascript on your own freakin iPhone you paid for with your own damn money. I was forced to give up using "open" web technologies for games and instead go with native code game engines (Godot, Unreal, Unity, ...). [1] https://superpowered.com/latency https://superpowered.com/latency

abeppu 5y ago

I'm confused about why the methodology that they talk about is innovative. They talk about shifting when a sound event of a given frequency arrives, relative to another sound. But with digital audio, esp that you can prepare in advance, isn't this pretty simple? Like, given signals A and B, apply a band-pass filter to A (for the desired frequency window), truncate off the first k samples based on the time shift of interest, and add to B. Even if you do need it in real time, why isn't this equivalent to delaying B by k samples? I must be missing something, because the researcher makes this sound like a cool new technique.

AceyMan 5y ago

Most any audio engineer or audiophile will confess the vast majority of a music's spaciality is dictated by the room. For the 3-nines of us who cannot dedicate a room for ideal echo, decay,absorption making all the temporal DSP tech overtake the influence of the room is a sysyphean task. So when my son was born profoundly deaf in one ear, as an audiophile, I soon realized "time to prove the merits of single channel." So far that just means my everyday workstation has a single Tivoli Model One radio as the output transducer and it sounds good enough for watching films and playing Spotify on the desktop. My intended future hifi is a single Lowther speaker cabinet driven by a modestly powered single ended tube triode amplifier. Long live monaural! /Acey

LargoLasskhyfv 5y ago

When walking along the fence on the right side of this street [1] https://www.google.de/maps/@53.5546251,9.8651002,3a,75y,192.33h,90t/data=!3m6!1e1!3m4!1s-pNeXAXSpeJcy9-YkD8tdA!2e0!7i13312!8i6656 https://www.google.de/maps/@53.5546251,9.8651002,3a,75y,192.... it feels like I could count every single echo of my steps from every single element of the fence if I could count fast enough. Similar when riding my bicycle slowly down there (rarely, because downwards, so I'm doing 50 kph there usually), ring my bell, or click my tongue. edit: I mean, I'm hearing the difference in the delay depending on the distance.

France_is_bacon 5y ago

I don't understand this. Delayed how? Does this mean if you are watching a video, and the person is talking, that a delay of milliseconds between the person's lips moving and when you hear it? But what does that have to do with castinets? I wouldn't be able to see them they probably move so fast. But they don't mention video matched against audo delay, that I read. If you start a song and they delay it by 10 seconds, it just starts 10 seconds later, how would you even know there's a delay? Or if there's a delay in the middle of a song, how would you know any difference, as maybe that's how the music piece is supposed to sound. What am I not getting?