5 ms·

The lesson I wish OS designers would learn from vintage computing is that when it comes to an interactive user interfaces, responsiveness matters far more than

The lesson I wish OS designers would learn from vintage computing is that when it comes to an interactive user interfaces, responsiveness matters far more than throughput, and so nothing required for interactivity should be swapped out, ever, under any circumstances.

Pr0ject217 4y ago
Do you have an example? (From a curious dev)
- robinsonb5 4y ago
  When I think of vintage computers I think of computers old enough that they have no concept of virtual memory. In particular, the Atari ST, the Amiga and the Acorn Archimedes - on all of those platforms the user interface wasn't merely another task running among dozens, it was the most important thing the computer was doing - and while it's hard to describe, the feeling of immediacy is noticeable (even if the task your mouse click set in motion takes minutes or hours to complete - the widget will give you instant visual feedback when you click it) and its absence on more modern machines is also noticeable. (If a modern machine's thrashing the hard disk - especially on spinning rust - it can take seconds for the same feedback to happen, even if the machine itself is thousands of times faster.) Yesterday's topic about the memory footprint of desktop environments touched on this - the fact that if a memory-hungry process triggers a swapstorm it can take minutes to regain control of a Linux desktop. I call that sense of immediacy the feeling of "having the computer's full attention" - and ensuring that nothing required for interactivity gets paged out is the only robust way I can see of creating that feeling - but instead we use the somewhat crude, brute-force approach of throwing more RAM at the computer until the swap-outs stop happening!
  - Pr0ject217 4y ago
    Thank you for the detailed explanation! It sounds like programming computers back then would have been a joy. Casey Muratori's talks[0] on performance and optimization sparked an interest in me to start putting thought into what the CPU actually does (which wasn't taught when I learned web development). Since discovering his talks, I've been on the hunt for more learning materials. [0] https://www.youtube.com/watch?v=Ge3aKEmZcqY&list=PLEMXAbCVnmY4JbNByvpgEzWsLRKVaF_pk https://www.youtube.com/watch?v=Ge3aKEmZcqY&list=PLEMXAbCVnm...
- jacquesm 4y ago
  QnX. http://toastytech.com/guis/qnxdemo.html http://toastytech.com/guis/qnxdemo.html
  - Pr0ject217 4y ago
    Thanks!
Karrot_Kream 4y ago
I'm not sure if I'd weigh latency as "far more" important than throughput (instead a more intelligent tradeoff), but I do think an exploration of low latency systems for day-to-day use would be very interesting. I've thought about trying this myself, starting simple with something like RT Linux and try to present a low latency UX for viewing books, media, and certain webpages. Almost every step of the computation pipeline has optimized throughput over latency so it would take some "radical" rework, but I think it would be a very fun and informative experiment. I've spent a lot of time archiving and scraping my own content and optimizing my home network (and ingress points, routing, etc.) to access all of these things with minimal (< 3 ms) retrieval latency and the effects are joyful. The UX is bad as it's something I've written for low latency use and my UX skills aren't very good, but it makes me feel like I have "instant" access to content, which feels like a superpower. I'd love to explore topics like this for a broader audience for more than just nerdy media I consume.
- robinsonb5 4y ago
  There are certainly times when a heavy weighting in favour of latency makes sense and times when it doesn't - and interactivity is the defining factor for me. If I'm synthesizing an FPGA core or rendering a video I don't want latency optimisation to slow it down if I've wandered away from the computer, and left it to do its thing. If, on the other hand, I want to get on with something else while the long-winded task is progressing, I want what I'm doing now to be have priority, to be regarded as the "foreground" task, and I don't mind if the long-winded process takes longer as a result (within reason!) And, crucially, if that long-winded task eats all the RAM and I decide to cancel it, I want the user-interface to remain responsive so I can do that. I'm glad you've recognised the delights of a low-latency interface. The ideal interface is invisible, unnoticed by the user - my own thoughts are that an unpredictable response time is jarring, causing the interface to be noticed in a way that it shouldn't be - I've heard it called "jank" in the context of cellphone apps. So I believe people do recognise it, but probably underestimate how much it detracts from a pleasant user experience.
  - interroboink 4y ago
    I remember a cute trick Windows did (at least in the ~XP days) where if a window had focus, it got more CPU cycles. You could, for instance, create two identical console windows running a program that just does math in a loop, and if you give focus to one of them and let them churn, it will start to out-strip the other. Of course, the underlying problem is that it's not always easy to know where the user's attention is (or if they have walked away, as you mention), but it was a simple "somewhat right" trick.
taneq 4y ago
This is why you should develop user interfaces on last-gen (or gen-before-that) hardware. Developers optimize things that look slow to them so if you're on the latest firebreathing top end hardware, nothing seems slow. If you're on a 5-year-old i5 then you become very aware of which code is OK and which code is unacceptably slow. That said, VR is the unsung hero of finally improving the awful jitter that most gaming hardware and software was developing 10 years ago. I don't care if your new graphics card can maintain 200fps on average if it has 100ms latency spikes every second. Smooth is important.
- robinsonb5 4y ago
  Not just last-gen hardware, but also with a network that deliberately drops maybe 0.5% of packets, and delays some at random! I hadn't considered the VR aspect, but you're absolutely right - the jarring nature of UI "jank" would be literally nauseating in VR.
  - rbanffy 4y ago
    > with a network that deliberately drops maybe 0.5% of packets, and delays some at random! I've seen Sun's Crossbow stuff in use once, but it made quite an impression - it allowed testing many failure scenarios with relatively little effort.
  - taneq 4y ago
    Yeah, there's a hard latency threshold beyond which most users very quickly get VR sickness. Abrash has some fascinating writeups on the research he was involved in at Oculus back in the early days. Like you say, modern devs really need to remember to test against a high latency unreliable network. Or even just against a lack of network at all... it constantly astounds me that companies like Microsoft can let critical, central components like the start menu cease functioning without an internet connection. I recently had the start menu refuse to find any programs on my computer because it was trying to get to Bing and I was on a remote site with only local Ethernet plugged in.
neilv 4y ago
I made (what today would seem) insanely fast UIs... under MS-DOS on 8088. We've spent the last decade-plus focused mostly on slick brochures you can poke at with your finger. Where the point is not to get stuff done, but to "engage" and manipulate the user. But not all is lost. There are some high-performance niches in tools for professionals, and in some facets of video games.
deterministic 4y ago
Agree. Games developers know this and put a lot of effort into reducing the key-press-to-display-update loop.
jacquesm 4y ago
User interaction is a hard real time problem, unfortunately no single OS vendor seems to want to acknowledge this fact.
- PaulDavisThe1st 4y ago
  It absolutely is not. At best, maybe a soft real time problem. Hard real time problems are ones where people die, stuff is destroyed etc. when deadlines are missed. User interaction is never (or almost never) like this.
  - jacquesm 4y ago
    Hard real time simply puts hard upper limits on constraints, and if those aren't met then the program is considered to have failed. It does not imply that people die or that stuff is destroyed. I've used hard real time, soft real time and regular OSs for GUI based interaction and the only ones that really feel like they are satisfactory are the hard real time ones. The soft real time ones are obviously better than the general purpose ones but they still tend to give regular moments of unresponsiveness.
DoneWithAllThat 4y ago
For the first time recently I started going to a McDonald’s that uses the kiosks. The response lag to touch inputs is surprisingly bad. It’s quite shocking actually how awkward it feels to tap something and then wait awhile for it to react at all. The urge to tap again is overwhelming for even a few hundred milliseconds lag. Another example: using the payment thing at a 7-11. You have to say if you want cashback or not by tapping on the screen (can’t use pin pad). Tapping a selection results in a minuscule change in contrast on the button that’s almost impossible to see in part because the buttons are smaller than a finger. And then it still doesn’t move on to the next screen for a few seconds, leaving you to wonder if you actually made a selection of if it’s still waiting because it didn’t register your tap.
- robinsonb5 4y ago
  The worst example of UI lag I've ever seen is on the shiny new treadmills they installed at my local gym a few years ago. While there are a few physical controls, most of them are now touchscreen-based. The speed-up and speed-down controls adjust the target speed, but the only visual feedback shows how fast the treatmill's currently running, and there's no indication of whether it's reached the target speed, which can take several seconds... So a new user hops on the treadmill, leans on the speed-up button until it feels about right, lets go then hammers the speed down button in a panic as it keeps getting faster! Fun to watch, but a poster-child for how not to do UX!
  - criddell 4y ago
    The worst example for me is the volume knob on the radio in my car. When I start it in the morning I often find out that I was listening to something too loud last time I drove and I fruitlessly spin the knob for the ten seconds it takes for the radio to finish booting. I think the digital version of the volume knob is a serious downgrade from the old analog version. It's probably cheaper though. The manufacturer saves $10 and I spend the next 5 years swearing at my car right after starting it.
BizarroLand 4y ago
Honestly, we should never wait a human-countable amount of time between a click and a response. Throw up a splash screen. Make the button appear to depress on click, show me a spinny wheel to indicate that some process is chugging away to bring me my latest dopamine hit. And on top of that, either completely eliminate anything that bogs down the interactivity or at least give me a range of options between Win 98 and Modern lush.

Pr0ject217 4y ago

Do you have an example? (From a curious dev)

robinsonb5 4y ago
When I think of vintage computers I think of computers old enough that they have no concept of virtual memory. In particular, the Atari ST, the Amiga and the Acorn Archimedes - on all of those platforms the user interface wasn't merely another task running among dozens, it was the most important thing the computer was doing - and while it's hard to describe, the feeling of immediacy is noticeable (even if the task your mouse click set in motion takes minutes or hours to complete - the widget will give you instant visual feedback when you click it) and its absence on more modern machines is also noticeable. (If a modern machine's thrashing the hard disk - especially on spinning rust - it can take seconds for the same feedback to happen, even if the machine itself is thousands of times faster.) Yesterday's topic about the memory footprint of desktop environments touched on this - the fact that if a memory-hungry process triggers a swapstorm it can take minutes to regain control of a Linux desktop. I call that sense of immediacy the feeling of "having the computer's full attention" - and ensuring that nothing required for interactivity gets paged out is the only robust way I can see of creating that feeling - but instead we use the somewhat crude, brute-force approach of throwing more RAM at the computer until the swap-outs stop happening!
- Pr0ject217 4y ago
  Thank you for the detailed explanation! It sounds like programming computers back then would have been a joy. Casey Muratori's talks[0] on performance and optimization sparked an interest in me to start putting thought into what the CPU actually does (which wasn't taught when I learned web development). Since discovering his talks, I've been on the hunt for more learning materials. [0] https://www.youtube.com/watch?v=Ge3aKEmZcqY&list=PLEMXAbCVnmY4JbNByvpgEzWsLRKVaF_pk https://www.youtube.com/watch?v=Ge3aKEmZcqY&list=PLEMXAbCVnm...
jacquesm 4y ago
QnX. http://toastytech.com/guis/qnxdemo.html http://toastytech.com/guis/qnxdemo.html
- Pr0ject217 4y ago
  Thanks!

Karrot_Kream 4y ago

I'm not sure if I'd weigh latency as "far more" important than throughput (instead a more intelligent tradeoff), but I do think an exploration of low latency systems for day-to-day use would be very interesting. I've thought about trying this myself, starting simple with something like RT Linux and try to present a low latency UX for viewing books, media, and certain webpages. Almost every step of the computation pipeline has optimized throughput over latency so it would take some "radical" rework, but I think it would be a very fun and informative experiment. I've spent a lot of time archiving and scraping my own content and optimizing my home network (and ingress points, routing, etc.) to access all of these things with minimal (< 3 ms) retrieval latency and the effects are joyful. The UX is bad as it's something I've written for low latency use and my UX skills aren't very good, but it makes me feel like I have "instant" access to content, which feels like a superpower. I'd love to explore topics like this for a broader audience for more than just nerdy media I consume.

robinsonb5 4y ago
There are certainly times when a heavy weighting in favour of latency makes sense and times when it doesn't - and interactivity is the defining factor for me. If I'm synthesizing an FPGA core or rendering a video I don't want latency optimisation to slow it down if I've wandered away from the computer, and left it to do its thing. If, on the other hand, I want to get on with something else while the long-winded task is progressing, I want what I'm doing now to be have priority, to be regarded as the "foreground" task, and I don't mind if the long-winded process takes longer as a result (within reason!) And, crucially, if that long-winded task eats all the RAM and I decide to cancel it, I want the user-interface to remain responsive so I can do that. I'm glad you've recognised the delights of a low-latency interface. The ideal interface is invisible, unnoticed by the user - my own thoughts are that an unpredictable response time is jarring, causing the interface to be noticed in a way that it shouldn't be - I've heard it called "jank" in the context of cellphone apps. So I believe people do recognise it, but probably underestimate how much it detracts from a pleasant user experience.
- interroboink 4y ago
  I remember a cute trick Windows did (at least in the ~XP days) where if a window had focus, it got more CPU cycles. You could, for instance, create two identical console windows running a program that just does math in a loop, and if you give focus to one of them and let them churn, it will start to out-strip the other. Of course, the underlying problem is that it's not always easy to know where the user's attention is (or if they have walked away, as you mention), but it was a simple "somewhat right" trick.

taneq 4y ago

This is why you should develop user interfaces on last-gen (or gen-before-that) hardware. Developers optimize things that look slow to them so if you're on the latest firebreathing top end hardware, nothing seems slow. If you're on a 5-year-old i5 then you become very aware of which code is OK and which code is unacceptably slow. That said, VR is the unsung hero of finally improving the awful jitter that most gaming hardware and software was developing 10 years ago. I don't care if your new graphics card can maintain 200fps on average if it has 100ms latency spikes every second. Smooth is important.

robinsonb5 4y ago
Not just last-gen hardware, but also with a network that deliberately drops maybe 0.5% of packets, and delays some at random! I hadn't considered the VR aspect, but you're absolutely right - the jarring nature of UI "jank" would be literally nauseating in VR.
- rbanffy 4y ago
  > with a network that deliberately drops maybe 0.5% of packets, and delays some at random! I've seen Sun's Crossbow stuff in use once, but it made quite an impression - it allowed testing many failure scenarios with relatively little effort.
- taneq 4y ago
  Yeah, there's a hard latency threshold beyond which most users very quickly get VR sickness. Abrash has some fascinating writeups on the research he was involved in at Oculus back in the early days. Like you say, modern devs really need to remember to test against a high latency unreliable network. Or even just against a lack of network at all... it constantly astounds me that companies like Microsoft can let critical, central components like the start menu cease functioning without an internet connection. I recently had the start menu refuse to find any programs on my computer because it was trying to get to Bing and I was on a remote site with only local Ethernet plugged in.

neilv 4y ago

I made (what today would seem) insanely fast UIs... under MS-DOS on 8088. We've spent the last decade-plus focused mostly on slick brochures you can poke at with your finger. Where the point is not to get stuff done, but to "engage" and manipulate the user. But not all is lost. There are some high-performance niches in tools for professionals, and in some facets of video games.

deterministic 4y ago

Agree. Games developers know this and put a lot of effort into reducing the key-press-to-display-update loop.

jacquesm 4y ago

User interaction is a hard real time problem, unfortunately no single OS vendor seems to want to acknowledge this fact.

PaulDavisThe1st 4y ago
It absolutely is not. At best, maybe a soft real time problem. Hard real time problems are ones where people die, stuff is destroyed etc. when deadlines are missed. User interaction is never (or almost never) like this.
- jacquesm 4y ago
  Hard real time simply puts hard upper limits on constraints, and if those aren't met then the program is considered to have failed. It does not imply that people die or that stuff is destroyed. I've used hard real time, soft real time and regular OSs for GUI based interaction and the only ones that really feel like they are satisfactory are the hard real time ones. The soft real time ones are obviously better than the general purpose ones but they still tend to give regular moments of unresponsiveness.

DoneWithAllThat 4y ago

For the first time recently I started going to a McDonald’s that uses the kiosks. The response lag to touch inputs is surprisingly bad. It’s quite shocking actually how awkward it feels to tap something and then wait awhile for it to react at all. The urge to tap again is overwhelming for even a few hundred milliseconds lag. Another example: using the payment thing at a 7-11. You have to say if you want cashback or not by tapping on the screen (can’t use pin pad). Tapping a selection results in a minuscule change in contrast on the button that’s almost impossible to see in part because the buttons are smaller than a finger. And then it still doesn’t move on to the next screen for a few seconds, leaving you to wonder if you actually made a selection of if it’s still waiting because it didn’t register your tap.

robinsonb5 4y ago
The worst example of UI lag I've ever seen is on the shiny new treadmills they installed at my local gym a few years ago. While there are a few physical controls, most of them are now touchscreen-based. The speed-up and speed-down controls adjust the target speed, but the only visual feedback shows how fast the treatmill's currently running, and there's no indication of whether it's reached the target speed, which can take several seconds... So a new user hops on the treadmill, leans on the speed-up button until it feels about right, lets go then hammers the speed down button in a panic as it keeps getting faster! Fun to watch, but a poster-child for how not to do UX!
- criddell 4y ago
  The worst example for me is the volume knob on the radio in my car. When I start it in the morning I often find out that I was listening to something too loud last time I drove and I fruitlessly spin the knob for the ten seconds it takes for the radio to finish booting. I think the digital version of the volume knob is a serious downgrade from the old analog version. It's probably cheaper though. The manufacturer saves $10 and I spend the next 5 years swearing at my car right after starting it.

BizarroLand 4y ago

Honestly, we should never wait a human-countable amount of time between a click and a response. Throw up a splash screen. Make the button appear to depress on click, show me a spinny wheel to indicate that some process is chugging away to bring me my latest dopamine hit. And on top of that, either completely eliminate anything that bogs down the interactivity or at least give me a range of options between Win 98 and Modern lush.