4 ms·

Yeah, I think the numbers in the article should be good enough for a bunch of music use cases, but remember it used to be like half a second which is obviously

by kall 6y ago

Yeah, I think the numbers in the article should be good enough for a bunch of music use cases, but remember it used to be like half a second which is obviously unworkable. Well financed developers like ROLI already have instruments on android. I think the incredible instruments available on iOS are also there because the App Store promise actually kind of works for them. They write a great application, upload it to server and people buy it for more than 2.99$, upfront. And unlike VSTs, no one even pirates it. The general idea/stereotype is that no one wants that on android. An open source port of VCV Rack would be feasable and wonderful though. That's the only free instrument I have installed on my music iPad. Edit: nevermind, that was not free either.

de6u99er 6y ago
Totally agree. We should not forget that our brain is great at adjusting to latency. In the old days, when I was excessively playing Quake 3 Team Arena it took me just a couple of minutes to adjust to network latency depending on the server I was connected to. Professional musicians do this constantly and are much better at that.
- lloeki 6y ago
  What. Our brain is absolutely terrible at adjusting for latency. It can somewhat compensate, but only to an extent, and at severe cost. There’s this video around that puts people in a VR helmet playing A/V with a delay, even the lowest setting completely destroys any form of efficiency. Anything below 60Hz in VR has a significantly increased chance of giving motion sickness due to latency. Playing Super Mario Bros in an emulator is strangely subtly harder than on the real NES hardware, just because of a slight lag added (can’t recal on NES but on SNES buttons literally bit-flip the RAM, hard to beat that). You can train all you want on an emulator, when you get back to the real thing it’s obvious it was just so crippling. Try Guitar Hero / Rock Band with various audio / video latency settings. 10ms is obviously impactful, anything higher borderlines on unplayable, and even 5ms has a subtle impact increasing missing notes. As for pro musicians, they absolutely loathe latency. This is why bassists (tempo) / drummers (rhythm) / conductors (tempo) exist: they serve as a single source of truth to sync on. In fact, we’re so sensitive to latency that the slightest deviation is unbearable, hence why following them put you back in sync. Same for effect pedals, amps, feedbacks... any latency is destroying. Try playing on Zoom, it’s absolutely impossible (unless you assume latency is fixed, which it is not, and play in a tempo matching latency exactly, and follow the lead but time-warp your play by one beat, which I imagine is possible as a sort of canon but absolutely horrendous) unless you completely ignore the other part and find a common time source to sync upon. Now you mention Quake 3, and it might just be you were severely misled by several critical innovations that game introduced, like network prediction and frame interpolation, negating most of the practical effects of latency. https://www.jfedor.org/quake3/ https://www.jfedor.org/quake3/
  - indymike 6y ago
    Can confirm that many musicians have to play with latency. Pipe organs, heavily processed guitars (delay is an effect that is very commonly used) all are laggy and you have to play ahead of your accompaniment.
    - lloeki 6y ago
      True for organists (for which I don’t know much about, but read on, as I believe the following applies to them as well) The fact that the effects are laggy doesn’t mean you need to sync on them, you’re mostly listening whether the sound envelope is correct (e.g adjust for overshoot/undershoot a bend). Checking the output of effects only matters at timescales of whole beats/measures in terms of how you have to react, definitely not like keeping up with tempo, which has to be precise across the band. You don’t get delay on drums, and exceedingly rarely so on bass, which is what you have to react agains efficiently, plus the instrument you’re playing provides its own subtle and not-so-subtle haptic feedback. On a guitar, this means the moment you pluck the chord is based on your internal clock synced by listening to the tempo giver (usually bass). Source: amateur musician myself, wife is pro.
  - bumbada 6y ago
    Input in NES is not lag's source. Input is only read once per frame. Today it is very simple and cheap to improve over old controllers. Lag comes from video. If you do not use FPGA emulators like Mister the video delay is enormous. I don't understand what you are calling 10ms lag. Have you measured the delay between your computer and your screen? It is way bigger than that, in any computer or NES console(console-TV). Are you maybe referring to 10ms NETWORK packet delay? All TVs or projectors use buffers, and it takes time for information to travel around serial cables. Network delay is inconsistent, but electronic delay is not.
    - lloeki 6y ago
      > Input in NES is not lag's source. Input is only read once per frame. That’s exactly what I said ;) By virtue of being a different platform, emulation introduces a lag that is just not present on the real hardware (and can’t be lower by virtue of literally being a wire to memory). USB alone often introduces an astoundingly high minimal floor on latency. > I don't understand what you are calling 10ms lag. Have you measured the delay between your computer and your screen? It is way bigger than that, in any computer or NES console(console-TV). Yes I have measured it (on all my devices, accurately so) in order to minimise it as much as possible (I can work with but can’t stand latency, even in a terminal). Rock Band, which I used to be an avid player of, allows you to adjust a relative delay (audio and video separately), timewarping A/V so that all three are perfectly in sync relative to real-world input. > Network delay is inconsistent, but electronic delay is not. I agree (and agree to buffers as well), but by and large any modern device is far removed from being just electronic, from USB (USB 2.0 has a 125us polling loop for multiplexing, USB 3 is point to point so can fare much better at 30us, but even then there’s kernel context switch to read and process that data) to HDMI (packet based with loads of multiplexing), many things end up being surprisingly subject to soft-real-time firmware/software before it hits the display itself. Hard real time stuff probably exists because the specs now allow for it but good luck finding that in consumer space where it’s much easier (read: cheaper) to push a best effort implementation. But my point isn’t about the minutiae of lag source, it’s that the human brain is severely impacted by lag, can adjust to some extent but the cost is paid heavily at multiple levels and with various efficacy crippling consequences; and that Q3 and subsequent games have increasing numbers of facilities to create the illusion that everything happens in sync so that we’re not so impacted by it.
tekstar 6y ago
I released free AUv3 ports of mutable instruments modules, turned into instruments. Look up Spectrum Synthesizer Bundle on App Store.
- kall 6y ago
  These are excellent! And you also made synthmodes which I use all the time. Thank you!
  - tekstar 6y ago
    Awesome! You are welcome!

by kall 6y ago

de6u99er 6y ago

Totally agree. We should not forget that our brain is great at adjusting to latency. In the old days, when I was excessively playing Quake 3 Team Arena it took me just a couple of minutes to adjust to network latency depending on the server I was connected to. Professional musicians do this constantly and are much better at that.

lloeki 6y ago
What. Our brain is absolutely terrible at adjusting for latency. It can somewhat compensate, but only to an extent, and at severe cost. There’s this video around that puts people in a VR helmet playing A/V with a delay, even the lowest setting completely destroys any form of efficiency. Anything below 60Hz in VR has a significantly increased chance of giving motion sickness due to latency. Playing Super Mario Bros in an emulator is strangely subtly harder than on the real NES hardware, just because of a slight lag added (can’t recal on NES but on SNES buttons literally bit-flip the RAM, hard to beat that). You can train all you want on an emulator, when you get back to the real thing it’s obvious it was just so crippling. Try Guitar Hero / Rock Band with various audio / video latency settings. 10ms is obviously impactful, anything higher borderlines on unplayable, and even 5ms has a subtle impact increasing missing notes. As for pro musicians, they absolutely loathe latency. This is why bassists (tempo) / drummers (rhythm) / conductors (tempo) exist: they serve as a single source of truth to sync on. In fact, we’re so sensitive to latency that the slightest deviation is unbearable, hence why following them put you back in sync. Same for effect pedals, amps, feedbacks... any latency is destroying. Try playing on Zoom, it’s absolutely impossible (unless you assume latency is fixed, which it is not, and play in a tempo matching latency exactly, and follow the lead but time-warp your play by one beat, which I imagine is possible as a sort of canon but absolutely horrendous) unless you completely ignore the other part and find a common time source to sync upon. Now you mention Quake 3, and it might just be you were severely misled by several critical innovations that game introduced, like network prediction and frame interpolation, negating most of the practical effects of latency. https://www.jfedor.org/quake3/ https://www.jfedor.org/quake3/
- indymike 6y ago
  Can confirm that many musicians have to play with latency. Pipe organs, heavily processed guitars (delay is an effect that is very commonly used) all are laggy and you have to play ahead of your accompaniment.
  - lloeki 6y ago
    True for organists (for which I don’t know much about, but read on, as I believe the following applies to them as well) The fact that the effects are laggy doesn’t mean you need to sync on them, you’re mostly listening whether the sound envelope is correct (e.g adjust for overshoot/undershoot a bend). Checking the output of effects only matters at timescales of whole beats/measures in terms of how you have to react, definitely not like keeping up with tempo, which has to be precise across the band. You don’t get delay on drums, and exceedingly rarely so on bass, which is what you have to react agains efficiently, plus the instrument you’re playing provides its own subtle and not-so-subtle haptic feedback. On a guitar, this means the moment you pluck the chord is based on your internal clock synced by listening to the tempo giver (usually bass). Source: amateur musician myself, wife is pro.
- bumbada 6y ago
  Input in NES is not lag's source. Input is only read once per frame. Today it is very simple and cheap to improve over old controllers. Lag comes from video. If you do not use FPGA emulators like Mister the video delay is enormous. I don't understand what you are calling 10ms lag. Have you measured the delay between your computer and your screen? It is way bigger than that, in any computer or NES console(console-TV). Are you maybe referring to 10ms NETWORK packet delay? All TVs or projectors use buffers, and it takes time for information to travel around serial cables. Network delay is inconsistent, but electronic delay is not.
  - lloeki 6y ago
    > Input in NES is not lag's source. Input is only read once per frame. That’s exactly what I said ;) By virtue of being a different platform, emulation introduces a lag that is just not present on the real hardware (and can’t be lower by virtue of literally being a wire to memory). USB alone often introduces an astoundingly high minimal floor on latency. > I don't understand what you are calling 10ms lag. Have you measured the delay between your computer and your screen? It is way bigger than that, in any computer or NES console(console-TV). Yes I have measured it (on all my devices, accurately so) in order to minimise it as much as possible (I can work with but can’t stand latency, even in a terminal). Rock Band, which I used to be an avid player of, allows you to adjust a relative delay (audio and video separately), timewarping A/V so that all three are perfectly in sync relative to real-world input. > Network delay is inconsistent, but electronic delay is not. I agree (and agree to buffers as well), but by and large any modern device is far removed from being just electronic, from USB (USB 2.0 has a 125us polling loop for multiplexing, USB 3 is point to point so can fare much better at 30us, but even then there’s kernel context switch to read and process that data) to HDMI (packet based with loads of multiplexing), many things end up being surprisingly subject to soft-real-time firmware/software before it hits the display itself. Hard real time stuff probably exists because the specs now allow for it but good luck finding that in consumer space where it’s much easier (read: cheaper) to push a best effort implementation. But my point isn’t about the minutiae of lag source, it’s that the human brain is severely impacted by lag, can adjust to some extent but the cost is paid heavily at multiple levels and with various efficacy crippling consequences; and that Q3 and subsequent games have increasing numbers of facilities to create the illusion that everything happens in sync so that we’re not so impacted by it.

tekstar 6y ago

I released free AUv3 ports of mutable instruments modules, turned into instruments. Look up Spectrum Synthesizer Bundle on App Store.

kall 6y ago
These are excellent! And you also made synthmodes which I use all the time. Thank you!
- tekstar 6y ago
  Awesome! You are welcome!