6 ms·

As a keyboard player, I can tell you that anything over 10ms latency noticeably degrades the experience. 100ms of keypress-to-sound latency makes the instrument

As a keyboard player, I can tell you that anything over 10ms latency noticeably degrades the experience. 100ms of keypress-to-sound latency makes the instrument unplayable. Your ears will detect very small latencies; offset something just by 1-2ms, and you get an echo feel; 50ms will be off-beat. (Organ players have to deal with that kind of latency on their instrument[1], but it's a hard skill to learn, and limits the music that can be performed on the instrument. Say, following a drummer is out of the question.) Why people tolerate huge amounts of unnecessary latency is beyond me. And everything is laggy these days. Except MIDI/audio setups, that is. And even in that scene, Android got a low-latency API only last year. I say, the decade of 2020 will be not about increasing performance, but decreasing the goddamn latency down to unnoticeable levels. Or I really hope so. [1]https://music.stackexchange.com/questions/30454/how-does-a-pipe-organist-deal-with-latency-or-delay https://music.stackexchange.com/questions/30454/how-does-a-p...

musicale 8y ago
Life can be rough for bass players, given the wavelength and speed of sound - you have to play ahead of the beat just for a note to show up on time! ;-) 10ms of added latency is very irritating - even adding 4ms feels clunky on guitar/headphones and that's typical for playing through a DAW and digital effects. As much as I like the iPad ... drawing on it is still like drawing underwater, and touch-to-sound latency is still too high in many cases. On the other hand... I can almost tolerate a bit of latency playing a weighted keyboard because I imagine that a hammer is hitting a string so I adjust my playing accordingly. The keyboard also seems to trigger before the note bottoms out, which helps a bit (except when you're playing very quietly.) There was a terrible idea in the CRT era that when a user clicked on a menu you could do all sorts of computation to update menu contents and still have time to draw the menu before the next ~80Hz refresh. Even if you missed a frame "humans couldn't notice it." Unfortunately this failed horribly in practice because it frequently pushed things over the edge, resulting in two frames of latency (or many more if latency wasn't managed well) which made things feel sluggish and unresponsive. Even in 2019, though we have 120+ Hz user interfaces, asynchronous refresh, multi-gigahertz processors, and insane GPUs, menus are still clunky. Every time I click on a menu in macOS (Safari, for example, which I am using as I write this) I can almost hear it go "kachunk." Not even 10ms. ;-( Maybe the delay is an intentional fade-in or ... something, but I still hate it. Moral of the story: 10ms is terrible, and every bit of delay you add to a UI can push it (and your users) closer to the edge.
- microcolonel 8y ago
  > Every time I click on a menu in macOS (Safari, for example, which I am using as I write this) I can almost hear it go "kachunk." Not even 10ms. ;-( Maybe the delay is an intentional fade-in or ... something, but I still hate it. Not to be too down on one player, but Apple is especially bad for this. GTK+ is considered slow by many, but compared to Cocoa, it's an SR-71. Cocoa rarely ever completes a window resize fast enough to hit a 60Hz VSync, GTK+ at least usually keeps up with 144Hz (for me) if you have a good GPU, and 60Hz if you have anything faster than a mid-grade Sandybridge IGP. Depending on the application, Qt is also sometimes dramatically faster than GTK+ (though on Linux the controls are somewhat clunky, since they seem to be emulating some Windows control behaviours which did not stand the test of time).
  - johnzim 8y ago
    Cooperative Multitasking may have been a massive hack but boy was it ever responsive when you selected a menu back in System 7
    - microcolonel 8y ago
      I think some of the benefit of this could be replicated in a preemptive system if your process could yield to a specific other process, and grant your remaining scheduler time (and maybe some of the target's) to the target of your IPC, and preempt the preemption.
stcredzero 8y ago
As a keyboard player, I can tell you that anything over 10ms latency noticeably degrades the experience. 100ms of keypress-to-sound latency makes the instrument unplayable. When I've used Audacity to record multi-track, here's my experience: Over time very good players will notice 5ms or above of de-sync.
dearrifling 8y ago
100ms of keypress-to-sound latency makes the instrument unplayable. What about organ players?
- romwell 8y ago
  Read on :)
johnchristopher 8y ago
> (Organ players have to deal with that kind of latency on their instrument[1], but it's a hard skill to learn, and limits the music that can be performed on the instrument. Say, following a drummer is out of the question.) I suppose following a metronome instead of the drum isn't practical/possible ?
- bluGill 8y ago
  No, they have to lead. In some of the worst cases there is more than a full second between pressing a key and hearing a sound. For fast music that can be several beats. Learning an organ means you need to learn to trust the instrument (not easy when it is a mechanical beast with many parts that can fail) and feel what the music will be not react to what it is.
squarefoot 8y ago
"As a keyboard player, I can tell you that anything over 10ms latency noticeably degrades the experience." Which explains why unless I missed some new development, there's no way professional music can be played live on any Android based product. By music I mean music, not triggering prerecorded loops, a thing which could be done on a 50+ years old Mellotron as well. I'm 100% sure Java is to blame: low latency and virtual machines don't mix well.
- nneonneo 8y ago
  You can develop apps in C/C++ with the NDK, cutting out Java almost entirely and avoiding its associated latencies. This is what all serious games do. However, the lack of a widely-supported low-latency audio stack is still a killer.
  - romwell 8y ago
    Android Oboe[1] might fix that (an API that goes through (OpenSL ES or AAudio depending on what is available). Then, in future far, far away we might get a touch input method that isn't too laggy for interaction with audio. Last year, the best we could get was around 30ms[2] on iSomething; and I hope that 120hz refresh rate becomes the standard (but even that is huuuuge compared to a MIDI controller). [1]https://github.com/google/oboe https://github.com/google/oboe [2]https://www.reddit.com/r/Android/comments/7teiwe/lets_discuss_touch_latencyaccuracy_on_flagships/ https://www.reddit.com/r/Android/comments/7teiwe/lets_discus...
- romwell 8y ago
  >There's no way professional music can be played live on any Android based product. There are some very recent developments on that frontier[1]. Let's hope that the new Oboe API will result in usable music software on Android. I have a DX7 emulator on Android that is pretty good latency-wise when used with a MIDI controller. Sadly, touchscreen latency is still way too big for anything musical (and the new Oboe API doesn't address that). [1]https://android-developers.googleblog.com/2018/10/introducing-oboe-c-library-for-low.html https://android-developers.googleblog.com/2018/10/introducin...
- 8y ago
yellowapple 8y ago
> Organ players have to deal with that kind of latency on their instrument[1], but it's a hard skill to learn, and limits the music that can be performed on the instrument. Say, following a drummer is out of the question. Field musicians (marching band, drum & bugle corps, etc.) have a similar issue because of the sheer size of the "stage"; the distances between performers is large enough that the speed of sound actually causes notes to be off-beat to the audience if they're "on-beat" to the performer. Instead, each performer has to basically ignore one's own ears (at least for rhythm; one should by all means use one's ears for matching pitch, though in theory a sufficiently-fast-moving performer might need to pitch up/down relative to what one hears to account for Doppler shifts) and watch someone up front (e.g. a drum major). Even in non-marching settings (e.g. your typical orchestra or symphonic band or even rock/jazz/etc. band), the percussion section is typically positioned rear-most on the stage for this exact reason (and in this case, it is okay to listen for the beat, since the sound of said beat would be coming from behind you). So for an organ player (organist?), your best bet would be to place the drummer behind the organ, then give the organist a mirror to watch the drummer's sticks and keep time with the visual beat. That way, by the time the organ's sound actually comes out, the drummer's sound will have arrived and they'll be in sync (assuming you have the distance right between the drums and organ; this is something you'll probably have to tune for every stage, given air-pressure-related differences in the speed of sound).
- bluGill 8y ago
  Organs are even harder because it isn't just speed of sound. There is often mechanical delay before the pipe even gets the air after the key is pressed. In some organs the pipes are "slow" to speak meaning that rank of pipes won't make a sound for some time after the air arrives. Skilled pipe manufactures can minimize this (called voicing, though voicing covers other areas of pipe fine tuning), but in some cases that involves other compromises that aren't desired. Organs have gotten better and worse in this regard several times over the hundreds of years since they have been known.
  - yellowapple 8y ago
    My point is that if you put the drummer far enough behind the organist and have the organist track the drummer by eye instead of by ear, the speed-of-sound-induced delay will match the organ's mechanically-induced delay and everything will sound right. Another option would be to place the organist next to the drummer, but keep the organ itself far enough in front of the drummer such that by the time the sound comes out it's in time with the drummer. I don't know if there are any organs that work this way, though (I'd imagine it'd have to be either mechanically complex or electronically controlled; the latter, though, gives an option for a synthesized "monitor" that the organist can hear, which should hopefully make the delayed sound of the actual organ less of a mindscrew). This all assumes you have a large enough stage to be able to pull this off, and (in the case of fixed organs) have actually built the organ such that the sound does emerge from in front of the drums.
andy786 8y ago
For reference, sound travels 0.343 meters per millisecond, so 10ms latency is the equivalent of sitting 3.4 meters from a speaker and playing your keyboard.
- OJFord 8y ago
  Sure, but not constantly, or it wouldn't be an issue (which I guess is what you're implying). It's like jumping instantaneously 3.4m away or toward, each note the timing is off by 10ms.
  - temptemptemp111 8y ago
    Good catch; important distinction.
  - andy786 8y ago
    Not sure what you are saying, the time it takes after you hit a key for the sound to reach your ears is 10ms when sitting at that distance. It's the same time it takes if you use headphones and use a plugin with a 10ms latency. If anything the speaker situation could be worse since you may add room reverbation to the sound as well. If this is an issue or not depends on the player and how sensitive they are to this.
SilasX 8y ago
As a Dance Dance Revolution player, I can tell you that it affects rhythm games too. I don't have the exact values, but DDR is unplayable with a PS2 on most hi-def flatscreens because of the scale-up lag, even when you choose the "gaming mode" that optimizes for speed. It's related to most software engineering not being engineering: http://tonsky.me/blog/disenchantment/ http://tonsky.me/blog/disenchantment/
jancsika 8y ago
That's orthogonal to the article. There aren't that many situations in programming where one urgently needs to adjust the speed (or velocity!) of what one is typing based on the feedback of what they see on the screen. You, however, must constantly make such adjustments or the entire musical endeavor may fall apart. :)
- nitrogen 8y ago
  You do need rapid feedback to know if you've made a mistake typing at 110wpm. Latency also makes a terminal feel like molasses; just sticky and unresponsive. The feel of a musical instrument and the feel of a technical instrument are similarly affected by latency.
  - romwell 8y ago
    I agree with both you and the parent comment :) Yes, I don't even always need to look at the screen when I am typing fast. However, when I do, molasses UIs make me sad. As an interesting aside, I notice that my brain tends to make interesting typos when I am typing without looking at the words that I type and get into a flow state when I am not thinking about the spelling of words, especially at high WPM. By "interesting" I mean "I would have never consciously chose to spell that word like that" interesting. It shows that typing is happening on a different level than spelling; it's a more direct pipeline than writing with a pen where one has to think about shapes of letters. It sort of happens as an independent process, where <send a word to keyboard> happens without active thinking, and can have interesting "optimizations" along the way. I am looking at the monitor to make sure that these "optimizations" do not, in fact, occur. To give an example of this, here's a snippet of what I type with my eyes closed: "I'm a rocket ship on my way to mars, on a collision course. I'm a sattelite, I'm out of contrrol. " So, I need to occasionally look at the screen to catch "contrrol" errors. And it's annoying to do so when the letters lag behind key presses :)
MagicPropmaker 8y ago
This isn't just a high-tech problem. I'm a pianist who sometimes plays Pipe Organ, and you always have to "play ahead." It can take 250ms or more for some 16' or 32' pipes to start sounding.
- caf 8y ago
  I think Carillonists have the same problem.
swiley 8y ago
I hooked a ukulele up to my friend’s Tesla cool the other day using my laptop as a buffer by just piping arecord into aplay (hooking it up directly didn’t work) and the half second latency made it totally impossible to focus even though I could still hear the notes immediately after playing them.
z3t4 8y ago
Music is extremely sensitive for timings, but when it comes to visuals you seem to adapt quickly and tolerate lag up to 1/10s, how is that !? And what about "hidden" messages in videos (running as low as 30 FPS!) ? That only the subconscious can process. It seems that our human image processing only runs at around 30-60 FPS when idle, but can increase in stressful situations. I recall a study where people bungy jumping was able to read something off a clock during a jump, but not when safely on the ground.
- romwell 8y ago
  >Music is extremely sensitive for timings, but when it comes to visuals you seem to adapt quickly and tolerate lag up to 1/10s, how is that? Our visual systems are not optimized to give accurate, realtime representation of geometry around us. They evolved to give us a somewhat abstract representation that is most useful. There is a lot of post-processing involved between photons hitting your eye and your brain "seeing" things. In that post-processing, both geometry and time information are "corrected". The most striking example of this is saccadic masking[1]. Basically, your eyeballs move all the time, but you will never see that when you are looking at yourself in the mirror. However, look at your eyeballs on your selfie camera, and ta-dah! They move! That's because your eyes essentially block your visual processing pipeline while the eye is moving, and then backfill the missing frames. That's why if you move your eyes around rapidly while looking at an analog clock, you will perceive some seconds to last longer than others. The eyes are lying about how long things take. >Our human image processing only runs at around 30-60 FPS when idle, but can increase in stressful situations In general, our vision doesn't really have a "frame rate", and depending on how you define that concept, our "frame rate" goes well into the hundreds of FPS[0]. But something like what you said is happening. Visual perception of things gets on-the-fly adjustments as necessary. For example, baseball players see balls as being larger than you would see the same ball while being stationary at the same distance[2]. That's why, yes, we can get away with a lot of visual trickery, taking advantage of the visual post-processing pipeline (monitors giving the illusion of smooth motion is one example). However, this all doesn't mean we won't feel latency when it's there. As a good example, compare drawing a line with a pencil vs. drawing a line on a touchscreen device with a stylus. You will notice even the smallest latency. In fact, there is no device on the market where something simple as drawing a line following a stylus would happen fast enough for you to be fooled. As long as there is any visible distance between the tip of the stylus and the end of the line appearing on the screen, you will see it (or feel it), even if takes a very small time to fill that gap. Same with scroll latency: it's something you will not be able to see on a video recording of someone scrolling on a touchscreen, but you will absolutely feel the difference when you do it. And all that said, our aural perception is more sensitive to timing, and we also do Fourier transforms on the fly with our ears (that's how we can tell chords apart!), amonf other things. Our perception is an interesting thing. [0]https://www.pcgamer.com/how-many-frames-per-second-can-the-human-eye-really-see/ https://www.pcgamer.com/how-many-frames-per-second-can-the-h... [1]https://en.wikipedia.org/wiki/Saccadic_masking https://en.wikipedia.org/wiki/Saccadic_masking [2]https://www.thecut.com/2016/10/the-science-of-how-baseball-players-hit-fastballs.html https://www.thecut.com/2016/10/the-science-of-how-baseball-p...

musicale 8y ago

Life can be rough for bass players, given the wavelength and speed of sound - you have to play ahead of the beat just for a note to show up on time! ;-) 10ms of added latency is very irritating - even adding 4ms feels clunky on guitar/headphones and that's typical for playing through a DAW and digital effects. As much as I like the iPad ... drawing on it is still like drawing underwater, and touch-to-sound latency is still too high in many cases. On the other hand... I can almost tolerate a bit of latency playing a weighted keyboard because I imagine that a hammer is hitting a string so I adjust my playing accordingly. The keyboard also seems to trigger before the note bottoms out, which helps a bit (except when you're playing very quietly.) There was a terrible idea in the CRT era that when a user clicked on a menu you could do all sorts of computation to update menu contents and still have time to draw the menu before the next ~80Hz refresh. Even if you missed a frame "humans couldn't notice it." Unfortunately this failed horribly in practice because it frequently pushed things over the edge, resulting in two frames of latency (or many more if latency wasn't managed well) which made things feel sluggish and unresponsive. Even in 2019, though we have 120+ Hz user interfaces, asynchronous refresh, multi-gigahertz processors, and insane GPUs, menus are still clunky. Every time I click on a menu in macOS (Safari, for example, which I am using as I write this) I can almost hear it go "kachunk." Not even 10ms. ;-( Maybe the delay is an intentional fade-in or ... something, but I still hate it. Moral of the story: 10ms is terrible, and every bit of delay you add to a UI can push it (and your users) closer to the edge.

microcolonel 8y ago
> Every time I click on a menu in macOS (Safari, for example, which I am using as I write this) I can almost hear it go "kachunk." Not even 10ms. ;-( Maybe the delay is an intentional fade-in or ... something, but I still hate it. Not to be too down on one player, but Apple is especially bad for this. GTK+ is considered slow by many, but compared to Cocoa, it's an SR-71. Cocoa rarely ever completes a window resize fast enough to hit a 60Hz VSync, GTK+ at least usually keeps up with 144Hz (for me) if you have a good GPU, and 60Hz if you have anything faster than a mid-grade Sandybridge IGP. Depending on the application, Qt is also sometimes dramatically faster than GTK+ (though on Linux the controls are somewhat clunky, since they seem to be emulating some Windows control behaviours which did not stand the test of time).
- johnzim 8y ago
  Cooperative Multitasking may have been a massive hack but boy was it ever responsive when you selected a menu back in System 7
  - microcolonel 8y ago
    I think some of the benefit of this could be replicated in a preemptive system if your process could yield to a specific other process, and grant your remaining scheduler time (and maybe some of the target's) to the target of your IPC, and preempt the preemption.

stcredzero 8y ago

As a keyboard player, I can tell you that anything over 10ms latency noticeably degrades the experience. 100ms of keypress-to-sound latency makes the instrument unplayable. When I've used Audacity to record multi-track, here's my experience: Over time very good players will notice 5ms or above of de-sync.

dearrifling 8y ago

100ms of keypress-to-sound latency makes the instrument unplayable. What about organ players?

romwell 8y ago
Read on :)

johnchristopher 8y ago

> (Organ players have to deal with that kind of latency on their instrument[1], but it's a hard skill to learn, and limits the music that can be performed on the instrument. Say, following a drummer is out of the question.) I suppose following a metronome instead of the drum isn't practical/possible ?

bluGill 8y ago
No, they have to lead. In some of the worst cases there is more than a full second between pressing a key and hearing a sound. For fast music that can be several beats. Learning an organ means you need to learn to trust the instrument (not easy when it is a mechanical beast with many parts that can fail) and feel what the music will be not react to what it is.

squarefoot 8y ago

"As a keyboard player, I can tell you that anything over 10ms latency noticeably degrades the experience." Which explains why unless I missed some new development, there's no way professional music can be played live on any Android based product. By music I mean music, not triggering prerecorded loops, a thing which could be done on a 50+ years old Mellotron as well. I'm 100% sure Java is to blame: low latency and virtual machines don't mix well.

nneonneo 8y ago
You can develop apps in C/C++ with the NDK, cutting out Java almost entirely and avoiding its associated latencies. This is what all serious games do. However, the lack of a widely-supported low-latency audio stack is still a killer.
- romwell 8y ago
  Android Oboe[1] might fix that (an API that goes through (OpenSL ES or AAudio depending on what is available). Then, in future far, far away we might get a touch input method that isn't too laggy for interaction with audio. Last year, the best we could get was around 30ms[2] on iSomething; and I hope that 120hz refresh rate becomes the standard (but even that is huuuuge compared to a MIDI controller). [1]https://github.com/google/oboe https://github.com/google/oboe [2]https://www.reddit.com/r/Android/comments/7teiwe/lets_discuss_touch_latencyaccuracy_on_flagships/ https://www.reddit.com/r/Android/comments/7teiwe/lets_discus...
romwell 8y ago
>There's no way professional music can be played live on any Android based product. There are some very recent developments on that frontier[1]. Let's hope that the new Oboe API will result in usable music software on Android. I have a DX7 emulator on Android that is pretty good latency-wise when used with a MIDI controller. Sadly, touchscreen latency is still way too big for anything musical (and the new Oboe API doesn't address that). [1]https://android-developers.googleblog.com/2018/10/introducing-oboe-c-library-for-low.html https://android-developers.googleblog.com/2018/10/introducin...
8y ago

yellowapple 8y ago

> Organ players have to deal with that kind of latency on their instrument[1], but it's a hard skill to learn, and limits the music that can be performed on the instrument. Say, following a drummer is out of the question. Field musicians (marching band, drum & bugle corps, etc.) have a similar issue because of the sheer size of the "stage"; the distances between performers is large enough that the speed of sound actually causes notes to be off-beat to the audience if they're "on-beat" to the performer. Instead, each performer has to basically ignore one's own ears (at least for rhythm; one should by all means use one's ears for matching pitch, though in theory a sufficiently-fast-moving performer might need to pitch up/down relative to what one hears to account for Doppler shifts) and watch someone up front (e.g. a drum major). Even in non-marching settings (e.g. your typical orchestra or symphonic band or even rock/jazz/etc. band), the percussion section is typically positioned rear-most on the stage for this exact reason (and in this case, it is okay to listen for the beat, since the sound of said beat would be coming from behind you). So for an organ player (organist?), your best bet would be to place the drummer behind the organ, then give the organist a mirror to watch the drummer's sticks and keep time with the visual beat. That way, by the time the organ's sound actually comes out, the drummer's sound will have arrived and they'll be in sync (assuming you have the distance right between the drums and organ; this is something you'll probably have to tune for every stage, given air-pressure-related differences in the speed of sound).

bluGill 8y ago
Organs are even harder because it isn't just speed of sound. There is often mechanical delay before the pipe even gets the air after the key is pressed. In some organs the pipes are "slow" to speak meaning that rank of pipes won't make a sound for some time after the air arrives. Skilled pipe manufactures can minimize this (called voicing, though voicing covers other areas of pipe fine tuning), but in some cases that involves other compromises that aren't desired. Organs have gotten better and worse in this regard several times over the hundreds of years since they have been known.
- yellowapple 8y ago
  My point is that if you put the drummer far enough behind the organist and have the organist track the drummer by eye instead of by ear, the speed-of-sound-induced delay will match the organ's mechanically-induced delay and everything will sound right. Another option would be to place the organist next to the drummer, but keep the organ itself far enough in front of the drummer such that by the time the sound comes out it's in time with the drummer. I don't know if there are any organs that work this way, though (I'd imagine it'd have to be either mechanically complex or electronically controlled; the latter, though, gives an option for a synthesized "monitor" that the organist can hear, which should hopefully make the delayed sound of the actual organ less of a mindscrew). This all assumes you have a large enough stage to be able to pull this off, and (in the case of fixed organs) have actually built the organ such that the sound does emerge from in front of the drums.

andy786 8y ago

For reference, sound travels 0.343 meters per millisecond, so 10ms latency is the equivalent of sitting 3.4 meters from a speaker and playing your keyboard.

OJFord 8y ago
Sure, but not constantly, or it wouldn't be an issue (which I guess is what you're implying). It's like jumping instantaneously 3.4m away or toward, each note the timing is off by 10ms.
- temptemptemp111 8y ago
  Good catch; important distinction.
- andy786 8y ago
  Not sure what you are saying, the time it takes after you hit a key for the sound to reach your ears is 10ms when sitting at that distance. It's the same time it takes if you use headphones and use a plugin with a 10ms latency. If anything the speaker situation could be worse since you may add room reverbation to the sound as well. If this is an issue or not depends on the player and how sensitive they are to this.

SilasX 8y ago

As a Dance Dance Revolution player, I can tell you that it affects rhythm games too. I don't have the exact values, but DDR is unplayable with a PS2 on most hi-def flatscreens because of the scale-up lag, even when you choose the "gaming mode" that optimizes for speed. It's related to most software engineering not being engineering: http://tonsky.me/blog/disenchantment/ http://tonsky.me/blog/disenchantment/

jancsika 8y ago

That's orthogonal to the article. There aren't that many situations in programming where one urgently needs to adjust the speed (or velocity!) of what one is typing based on the feedback of what they see on the screen. You, however, must constantly make such adjustments or the entire musical endeavor may fall apart. :)

nitrogen 8y ago
You do need rapid feedback to know if you've made a mistake typing at 110wpm. Latency also makes a terminal feel like molasses; just sticky and unresponsive. The feel of a musical instrument and the feel of a technical instrument are similarly affected by latency.
- romwell 8y ago
  I agree with both you and the parent comment :) Yes, I don't even always need to look at the screen when I am typing fast. However, when I do, molasses UIs make me sad. As an interesting aside, I notice that my brain tends to make interesting typos when I am typing without looking at the words that I type and get into a flow state when I am not thinking about the spelling of words, especially at high WPM. By "interesting" I mean "I would have never consciously chose to spell that word like that" interesting. It shows that typing is happening on a different level than spelling; it's a more direct pipeline than writing with a pen where one has to think about shapes of letters. It sort of happens as an independent process, where <send a word to keyboard> happens without active thinking, and can have interesting "optimizations" along the way. I am looking at the monitor to make sure that these "optimizations" do not, in fact, occur. To give an example of this, here's a snippet of what I type with my eyes closed: "I'm a rocket ship on my way to mars, on a collision course. I'm a sattelite, I'm out of contrrol. " So, I need to occasionally look at the screen to catch "contrrol" errors. And it's annoying to do so when the letters lag behind key presses :)

MagicPropmaker 8y ago

This isn't just a high-tech problem. I'm a pianist who sometimes plays Pipe Organ, and you always have to "play ahead." It can take 250ms or more for some 16' or 32' pipes to start sounding.

caf 8y ago
I think Carillonists have the same problem.

swiley 8y ago

I hooked a ukulele up to my friend’s Tesla cool the other day using my laptop as a buffer by just piping arecord into aplay (hooking it up directly didn’t work) and the half second latency made it totally impossible to focus even though I could still hear the notes immediately after playing them.

z3t4 8y ago

Music is extremely sensitive for timings, but when it comes to visuals you seem to adapt quickly and tolerate lag up to 1/10s, how is that !? And what about "hidden" messages in videos (running as low as 30 FPS!) ? That only the subconscious can process. It seems that our human image processing only runs at around 30-60 FPS when idle, but can increase in stressful situations. I recall a study where people bungy jumping was able to read something off a clock during a jump, but not when safely on the ground.

romwell 8y ago
>Music is extremely sensitive for timings, but when it comes to visuals you seem to adapt quickly and tolerate lag up to 1/10s, how is that? Our visual systems are not optimized to give accurate, realtime representation of geometry around us. They evolved to give us a somewhat abstract representation that is most useful. There is a lot of post-processing involved between photons hitting your eye and your brain "seeing" things. In that post-processing, both geometry and time information are "corrected". The most striking example of this is saccadic masking[1]. Basically, your eyeballs move all the time, but you will never see that when you are looking at yourself in the mirror. However, look at your eyeballs on your selfie camera, and ta-dah! They move! That's because your eyes essentially block your visual processing pipeline while the eye is moving, and then backfill the missing frames. That's why if you move your eyes around rapidly while looking at an analog clock, you will perceive some seconds to last longer than others. The eyes are lying about how long things take. >Our human image processing only runs at around 30-60 FPS when idle, but can increase in stressful situations In general, our vision doesn't really have a "frame rate", and depending on how you define that concept, our "frame rate" goes well into the hundreds of FPS[0]. But something like what you said is happening. Visual perception of things gets on-the-fly adjustments as necessary. For example, baseball players see balls as being larger than you would see the same ball while being stationary at the same distance[2]. That's why, yes, we can get away with a lot of visual trickery, taking advantage of the visual post-processing pipeline (monitors giving the illusion of smooth motion is one example). However, this all doesn't mean we won't feel latency when it's there. As a good example, compare drawing a line with a pencil vs. drawing a line on a touchscreen device with a stylus. You will notice even the smallest latency. In fact, there is no device on the market where something simple as drawing a line following a stylus would happen fast enough for you to be fooled. As long as there is any visible distance between the tip of the stylus and the end of the line appearing on the screen, you will see it (or feel it), even if takes a very small time to fill that gap. Same with scroll latency: it's something you will not be able to see on a video recording of someone scrolling on a touchscreen, but you will absolutely feel the difference when you do it. And all that said, our aural perception is more sensitive to timing, and we also do Fourier transforms on the fly with our ears (that's how we can tell chords apart!), amonf other things. Our perception is an interesting thing. [0]https://www.pcgamer.com/how-many-frames-per-second-can-the-human-eye-really-see/ https://www.pcgamer.com/how-many-frames-per-second-can-the-h... [1]https://en.wikipedia.org/wiki/Saccadic_masking https://en.wikipedia.org/wiki/Saccadic_masking [2]https://www.thecut.com/2016/10/the-science-of-how-baseball-players-hit-fastballs.html https://www.thecut.com/2016/10/the-science-of-how-baseball-p...