4 ms·

I’ve always found that latency number to be suspect. In some contexts, it’s imperceivable. In others it is. For example, dragging an item around a screen with

I’ve always found that latency number to be suspect. In some contexts, it’s imperceivable. In others it is. For example, dragging an item around a screen with your finger, 100ms is definitely noticeable. Typing seems to be less although I had a coworker claim he could. So it’s hard to say if we don’t notice it vs we’ve just become accustomed to interacting with text with 100ms of latency (or something about the task of typing is more latency insensitive). I would be interested in seeing the the same data driven analysis for human factors as HCI is even less intuitive than computer algorithm performance. Searching is not a particularly latency sensitive task as your next match is probably significantly within 1 gib where you’re paying 250ms at worst. But yeah, if regex searching is the task to optimize around, ropes in Rust won’t work well due to the lack of incremental search at this time.

kaba0 3y ago
Our brains are very good at adjusting to latency, but only if it’s a constant.
burntsushi 3y ago
What regex engines do people use for this task generally? Most regex engines I'm aware of don't support stream/incremental searching.
- zogrodea 3y ago
  This comment by someone at GitHub mentioned that Atom used PCRE’s partial match functionality, but I have no idea about what is used most commonly. https://news.ycombinator.com/item?id=15386155 https://news.ycombinator.com/item?id=15386155
- vlovich123 3y ago
  I was just going from the linked issue where I thought I read that Go’s engine supports incremental search. Maybe I misread? While I have you, how close is the Rust incremental search support? It sounded like regex-automata might make it possible but hard to easily get a sense of high level progress from a GitHub issue.
  - burntsushi 3y ago
    Go has this: https://pkg.go.dev/regexp#Regexp.MatchReader https://pkg.go.dev/regexp#Regexp.MatchReader --- That just tells you whether an io.Reader contains a match anywhere or not. I don't think it tells you anything else, like the position of the match. > While I have you, how close is the Rust incremental search support? It sounded like regex-automata might make it possible but hard to easily get a sense of high level progress from a GitHub issue. I'm not working on it. The current status is that other people are working on trying to write it themselves on top of regex-automata in a way that works for their specific use case. That was my high level strategy: release a separately versioned library with the regex internals[1] exposed so that others can experiment and build on top of it. The regex-automata crate exposes the low level DFA (and lazy DFA) transition function. So you can pretty much do some kind of stream searching out of the box today (certainly at least what Go provides): https://docs.rs/regex-automata/latest/regex_automata/#build-a-full-dfa-and-walk-it-manually https://docs.rs/regex-automata/latest/regex_automata/#build-... Bottom line here is that if you're looking to implement stream searching in Rust, then you should be able to go out and build something to do it today with regex-automata. But you aren't going to get a streamlined experience. (And whether you ever will or not remains unclear to me.) [1]: https://blog.burntsushi.net/regex-internals/ https://blog.burntsushi.net/regex-internals/
    - LukeShu 3y ago
      > Go ... I don't think it tells you anything else, like the position of the match. The regexp.Regexp.FindReaderIndex and .FindReaderSubmatchIndex methods tell you the position of the match.
      - burntsushi 3y ago
        Despite having studied Go's regex engine in depth, I have somehow missed those methods. To the point that when you responded, I assumed they were added recently. I checked to be sure, and indeed not. They've been there since Go 1.0. Thanks for the correction!
alpaca128 3y ago
> ropes in Rust won’t work well due to the lack of incremental search at this time Unless you don't mind implementing a couple dozen lines yourself. The regex_automata crate allows you to directly access the DFA, so with iteration through the rope and a bit of state tracking you can already do this, just not as convenient (yet) as a single method call.
- celeritascelery 3y ago
  It’s not quite that simple, but folks are working on it. https://github.com/rust-lang/regex/issues/425#issuecomment-1730365783 https://github.com/rust-lang/regex/issues/425#issuecomment-1... https://github.com/helix-editor/helix/pull/211#issuecomment-859583586 https://github.com/helix-editor/helix/pull/211#issuecomment-...
  - alpaca128 3y ago
    Maybe not in the general case for arbitrary abstracted datastructures, but I have done it in a basic test to see if it's viable for my use-case, which is why I made that claim at all.
  - burntsushi 3y ago
    Note that the lazy DFA and its lower level APIs exists now and threads this needle.
userbinator 3y ago
I would definitely notice 100ms of latency when typing, considering that at my usual speed, the time between characters is around 70ms.
- vlovich123 3y ago
  The problem is that your thought is at the level of sentences with your muscle memory being automatic and your visual system mainly scanning for errors. Your “what character do I type next” at speed is less likely to be driven via visual feedback and more by direct proprieception / tactile feedback. Also, it’s not like you have a fixed 100ms delay between each character -the display system is asynchronous and will effectively batch everything up. So that 100ms delay means 100ms after your most recently typed character, not an additive 100ms delay. Again, I’m not saying you won’t notice it, but I know I’m a very fast typer and when I was working on Oculus AirLink we had a prototype virtual desktop thing and I know I personally didn’t really notice any lag despite our internal metrics saying ~100ms of latency (we’d artificially inject extra latency for studies). It may be people differences, but keep in mind that how you measure is also important and almost no one measures true key stroke to display latency nor do they run human factors studies to accurately characterize if there really is anyone who does notice it (even at Oculus - most studies are very poor quality often with limited sample size, skewed population, and very poor reproduction/analysis because the time allotted for these projects is just enough to try to get guidance on next steps)
- XorNot 3y ago
  I notice 100ms latency when flipping a mechanical light switch (found out while dialling in debounce on ESPHome).

kaba0 3y ago

Our brains are very good at adjusting to latency, but only if it’s a constant.

burntsushi 3y ago

What regex engines do people use for this task generally? Most regex engines I'm aware of don't support stream/incremental searching.

zogrodea 3y ago
This comment by someone at GitHub mentioned that Atom used PCRE’s partial match functionality, but I have no idea about what is used most commonly. https://news.ycombinator.com/item?id=15386155 https://news.ycombinator.com/item?id=15386155
vlovich123 3y ago
I was just going from the linked issue where I thought I read that Go’s engine supports incremental search. Maybe I misread? While I have you, how close is the Rust incremental search support? It sounded like regex-automata might make it possible but hard to easily get a sense of high level progress from a GitHub issue.
- burntsushi 3y ago
  Go has this: https://pkg.go.dev/regexp#Regexp.MatchReader https://pkg.go.dev/regexp#Regexp.MatchReader --- That just tells you whether an io.Reader contains a match anywhere or not. I don't think it tells you anything else, like the position of the match. > While I have you, how close is the Rust incremental search support? It sounded like regex-automata might make it possible but hard to easily get a sense of high level progress from a GitHub issue. I'm not working on it. The current status is that other people are working on trying to write it themselves on top of regex-automata in a way that works for their specific use case. That was my high level strategy: release a separately versioned library with the regex internals[1] exposed so that others can experiment and build on top of it. The regex-automata crate exposes the low level DFA (and lazy DFA) transition function. So you can pretty much do some kind of stream searching out of the box today (certainly at least what Go provides): https://docs.rs/regex-automata/latest/regex_automata/#build-a-full-dfa-and-walk-it-manually https://docs.rs/regex-automata/latest/regex_automata/#build-... Bottom line here is that if you're looking to implement stream searching in Rust, then you should be able to go out and build something to do it today with regex-automata. But you aren't going to get a streamlined experience. (And whether you ever will or not remains unclear to me.) [1]: https://blog.burntsushi.net/regex-internals/ https://blog.burntsushi.net/regex-internals/
  - LukeShu 3y ago
    > Go ... I don't think it tells you anything else, like the position of the match. The regexp.Regexp.FindReaderIndex and .FindReaderSubmatchIndex methods tell you the position of the match.
    - burntsushi 3y ago
      Despite having studied Go's regex engine in depth, I have somehow missed those methods. To the point that when you responded, I assumed they were added recently. I checked to be sure, and indeed not. They've been there since Go 1.0. Thanks for the correction!

alpaca128 3y ago

> ropes in Rust won’t work well due to the lack of incremental search at this time Unless you don't mind implementing a couple dozen lines yourself. The regex_automata crate allows you to directly access the DFA, so with iteration through the rope and a bit of state tracking you can already do this, just not as convenient (yet) as a single method call.

celeritascelery 3y ago
It’s not quite that simple, but folks are working on it. https://github.com/rust-lang/regex/issues/425#issuecomment-1730365783 https://github.com/rust-lang/regex/issues/425#issuecomment-1... https://github.com/helix-editor/helix/pull/211#issuecomment-859583586 https://github.com/helix-editor/helix/pull/211#issuecomment-...
- alpaca128 3y ago
  Maybe not in the general case for arbitrary abstracted datastructures, but I have done it in a basic test to see if it's viable for my use-case, which is why I made that claim at all.
- burntsushi 3y ago
  Note that the lazy DFA and its lower level APIs exists now and threads this needle.

userbinator 3y ago

I would definitely notice 100ms of latency when typing, considering that at my usual speed, the time between characters is around 70ms.

vlovich123 3y ago
The problem is that your thought is at the level of sentences with your muscle memory being automatic and your visual system mainly scanning for errors. Your “what character do I type next” at speed is less likely to be driven via visual feedback and more by direct proprieception / tactile feedback. Also, it’s not like you have a fixed 100ms delay between each character -the display system is asynchronous and will effectively batch everything up. So that 100ms delay means 100ms after your most recently typed character, not an additive 100ms delay. Again, I’m not saying you won’t notice it, but I know I’m a very fast typer and when I was working on Oculus AirLink we had a prototype virtual desktop thing and I know I personally didn’t really notice any lag despite our internal metrics saying ~100ms of latency (we’d artificially inject extra latency for studies). It may be people differences, but keep in mind that how you measure is also important and almost no one measures true key stroke to display latency nor do they run human factors studies to accurately characterize if there really is anyone who does notice it (even at Oculus - most studies are very poor quality often with limited sample size, skewed population, and very poor reproduction/analysis because the time allotted for these projects is just enough to try to get guidance on next steps)
XorNot 3y ago
I notice 100ms latency when flipping a mechanical light switch (found out while dialling in debounce on ESPHome).