8 ms·

depends on the context. The rule of thumb I have seen is that anything under 100ms is perceived as instantaneous by a user. So for interactive editing those lat

depends on the context. The rule of thumb I have seen is that anything under 100ms is perceived as instantaneous by a user. So for interactive editing those latencies are acceptable. Though is should be noted those latencies are only for 1 GB file, so they will get worse as the edited file gets larger. But if you are building something like a CRDT where you can have edits coming in from many sources then those will start to compound, and will destroy your responsiveness. Also if you are performing edits while doing something like scrolling you will start to drop frames. Jumprope and Xi-rope were specifically designed for the CRDT use case. As Raph points out, the latencies are what will really bite you. And latency is actually why I picked a gap buffer; the ropes have too high of latency with regex searches. If that was ever fixed I would be tempted to switch to ropes.

raphlinus 3y ago
Search is 100% a valid justification to use contiguous storage. It wasn't high in my list of considerations when I was starting out.
gary_0 3y ago
An occasional 100ms pause while I'm editing a huge 1GB file wouldn't be a deal-breaker for me, as long as regular-sized files have <10ms latency. It's been a long time since I opened a text file that huge, and I think the editor I was using at the time chugged something awful. VSCode is slightly laggy for me just editing a 10k line text file, so my standards are sadly not that high, even though I find typing latency really annoying.
vlovich123 3y ago
I’ve always found that latency number to be suspect. In some contexts, it’s imperceivable. In others it is. For example, dragging an item around a screen with your finger, 100ms is definitely noticeable. Typing seems to be less although I had a coworker claim he could. So it’s hard to say if we don’t notice it vs we’ve just become accustomed to interacting with text with 100ms of latency (or something about the task of typing is more latency insensitive). I would be interested in seeing the the same data driven analysis for human factors as HCI is even less intuitive than computer algorithm performance. Searching is not a particularly latency sensitive task as your next match is probably significantly within 1 gib where you’re paying 250ms at worst. But yeah, if regex searching is the task to optimize around, ropes in Rust won’t work well due to the lack of incremental search at this time.
- kaba0 3y ago
  Our brains are very good at adjusting to latency, but only if it’s a constant.
- burntsushi 3y ago
  What regex engines do people use for this task generally? Most regex engines I'm aware of don't support stream/incremental searching.
  - zogrodea 3y ago
    This comment by someone at GitHub mentioned that Atom used PCRE’s partial match functionality, but I have no idea about what is used most commonly. https://news.ycombinator.com/item?id=15386155 https://news.ycombinator.com/item?id=15386155
  - vlovich123 3y ago
    I was just going from the linked issue where I thought I read that Go’s engine supports incremental search. Maybe I misread? While I have you, how close is the Rust incremental search support? It sounded like regex-automata might make it possible but hard to easily get a sense of high level progress from a GitHub issue.
    - burntsushi 3y ago
      Go has this: https://pkg.go.dev/regexp#Regexp.MatchReader https://pkg.go.dev/regexp#Regexp.MatchReader --- That just tells you whether an io.Reader contains a match anywhere or not. I don't think it tells you anything else, like the position of the match. > While I have you, how close is the Rust incremental search support? It sounded like regex-automata might make it possible but hard to easily get a sense of high level progress from a GitHub issue. I'm not working on it. The current status is that other people are working on trying to write it themselves on top of regex-automata in a way that works for their specific use case. That was my high level strategy: release a separately versioned library with the regex internals[1] exposed so that others can experiment and build on top of it. The regex-automata crate exposes the low level DFA (and lazy DFA) transition function. So you can pretty much do some kind of stream searching out of the box today (certainly at least what Go provides): https://docs.rs/regex-automata/latest/regex_automata/#build-a-full-dfa-and-walk-it-manually https://docs.rs/regex-automata/latest/regex_automata/#build-... Bottom line here is that if you're looking to implement stream searching in Rust, then you should be able to go out and build something to do it today with regex-automata. But you aren't going to get a streamlined experience. (And whether you ever will or not remains unclear to me.) [1]: https://blog.burntsushi.net/regex-internals/ https://blog.burntsushi.net/regex-internals/
_dain_ 3y ago
>The rule of thumb I have seen is that anything under 100ms is perceived as instantaneous by a user. for typing characters into a text box this is not imperceptible. it should happen in a single frame.
- deagle50 3y ago
  And it gets more perceptible when moving the caret (more pixels to track and you're actively tracking).
deagle50 3y ago
Users can easily feel one frame of latency at 60hz given enough visual feedback (pixels in this case). It gets harder at 120hz and above. I have a toy editor with Ropey and WGPU, just added a 100ms wait on char insert and it's rubber banding when typing fast.
4death4 3y ago
Instead of the rule of thumb you’ve seen how about the rule of thumb you’ve tried? Make two simple HTML forms: one with an input that instantaneously accepts input and another with 100 ms delay. You can definitely tell the difference.
- kaba0 3y ago
  You are not measuring what you want. You measure 0+~20ms vs 100+~20ms.
  - 4death4 3y ago
    Ok, change the experiment to add 80ms artificial delay instead…
alpaca128 3y ago
I'd say 100ms is perceptible in almost any context, at least if it's on top of the already existing latencies of the hardware & OS (e.g. some laptops already add 150ms latency to every keypress [0], so good luck) [0] https://danluu.com/input-lag/ https://danluu.com/input-lag/
Hammershaft 3y ago
100ms is 6 frames of input latency at 60hz! Absolutely noticable.

raphlinus 3y ago

Search is 100% a valid justification to use contiguous storage. It wasn't high in my list of considerations when I was starting out.

gary_0 3y ago

An occasional 100ms pause while I'm editing a huge 1GB file wouldn't be a deal-breaker for me, as long as regular-sized files have <10ms latency. It's been a long time since I opened a text file that huge, and I think the editor I was using at the time chugged something awful. VSCode is slightly laggy for me just editing a 10k line text file, so my standards are sadly not that high, even though I find typing latency really annoying.

vlovich123 3y ago

I’ve always found that latency number to be suspect. In some contexts, it’s imperceivable. In others it is. For example, dragging an item around a screen with your finger, 100ms is definitely noticeable. Typing seems to be less although I had a coworker claim he could. So it’s hard to say if we don’t notice it vs we’ve just become accustomed to interacting with text with 100ms of latency (or something about the task of typing is more latency insensitive). I would be interested in seeing the the same data driven analysis for human factors as HCI is even less intuitive than computer algorithm performance. Searching is not a particularly latency sensitive task as your next match is probably significantly within 1 gib where you’re paying 250ms at worst. But yeah, if regex searching is the task to optimize around, ropes in Rust won’t work well due to the lack of incremental search at this time.

kaba0 3y ago
Our brains are very good at adjusting to latency, but only if it’s a constant.
burntsushi 3y ago
What regex engines do people use for this task generally? Most regex engines I'm aware of don't support stream/incremental searching.
- zogrodea 3y ago
  This comment by someone at GitHub mentioned that Atom used PCRE’s partial match functionality, but I have no idea about what is used most commonly. https://news.ycombinator.com/item?id=15386155 https://news.ycombinator.com/item?id=15386155
- vlovich123 3y ago
  I was just going from the linked issue where I thought I read that Go’s engine supports incremental search. Maybe I misread? While I have you, how close is the Rust incremental search support? It sounded like regex-automata might make it possible but hard to easily get a sense of high level progress from a GitHub issue.
  - burntsushi 3y ago
    Go has this: https://pkg.go.dev/regexp#Regexp.MatchReader https://pkg.go.dev/regexp#Regexp.MatchReader --- That just tells you whether an io.Reader contains a match anywhere or not. I don't think it tells you anything else, like the position of the match. > While I have you, how close is the Rust incremental search support? It sounded like regex-automata might make it possible but hard to easily get a sense of high level progress from a GitHub issue. I'm not working on it. The current status is that other people are working on trying to write it themselves on top of regex-automata in a way that works for their specific use case. That was my high level strategy: release a separately versioned library with the regex internals[1] exposed so that others can experiment and build on top of it. The regex-automata crate exposes the low level DFA (and lazy DFA) transition function. So you can pretty much do some kind of stream searching out of the box today (certainly at least what Go provides): https://docs.rs/regex-automata/latest/regex_automata/#build-a-full-dfa-and-walk-it-manually https://docs.rs/regex-automata/latest/regex_automata/#build-... Bottom line here is that if you're looking to implement stream searching in Rust, then you should be able to go out and build something to do it today with regex-automata. But you aren't going to get a streamlined experience. (And whether you ever will or not remains unclear to me.) [1]: https://blog.burntsushi.net/regex-internals/ https://blog.burntsushi.net/regex-internals/

_dain_ 3y ago

>The rule of thumb I have seen is that anything under 100ms is perceived as instantaneous by a user. for typing characters into a text box this is not imperceptible. it should happen in a single frame.

deagle50 3y ago
And it gets more perceptible when moving the caret (more pixels to track and you're actively tracking).

deagle50 3y ago

Users can easily feel one frame of latency at 60hz given enough visual feedback (pixels in this case). It gets harder at 120hz and above. I have a toy editor with Ropey and WGPU, just added a 100ms wait on char insert and it's rubber banding when typing fast.

4death4 3y ago

Instead of the rule of thumb you’ve seen how about the rule of thumb you’ve tried? Make two simple HTML forms: one with an input that instantaneously accepts input and another with 100 ms delay. You can definitely tell the difference.

kaba0 3y ago
You are not measuring what you want. You measure 0+~20ms vs 100+~20ms.
- 4death4 3y ago
  Ok, change the experiment to add 80ms artificial delay instead…

alpaca128 3y ago

I'd say 100ms is perceptible in almost any context, at least if it's on top of the already existing latencies of the hardware & OS (e.g. some laptops already add 150ms latency to every keypress [0], so good luck) [0] https://danluu.com/input-lag/ https://danluu.com/input-lag/

Hammershaft 3y ago

100ms is 6 frames of input latency at 60hz! Absolutely noticable.