3 ms·

For everyone else, your QNX comment from 2015: https://news.ycombinator.com/item?id=9872640 https://news.ycombinator.com/item?id=9872640

by Tomte 2y ago

stevesimmons 2y ago
As someone who knows only that QNX is a RTOS and that "real time" is good in some vague hand-wavy way, I found this 2015 comment a very succinct, concrete and illuminating description of one aspect of QNX's architecture that makes it properly real-time. So thank you very much for the link.
- scheme271 2y ago
  "Real time" is only good if you need guaranteed responsiveness within certain time limits. It can actually be suboptimal and slow you down if you don't need it and want to optimize other things like throughput.
  - AlotOfReading 2y ago
    Though in practice, the act of designing your code for realtime tends to produce better performance than the status quo of doing nothing. Most uses of this warning seem intended to promote the latter rather than to get people thinking about their actual requirements.
  - vbezhenar 2y ago
    Who needs throughput? Server software, but Linux is already dominant there and everyone's happy. For everything else, including desktop and mobile operating systems realtime sounds like a good idea.
    - mlyle 2y ago
      Absolutely deterministic latency isn't a requirement for the desktop like it is for industrial control. If 99% of the time performance is faster, and the remaining 1% of the time isn't that bad, most people will take the trade of higher average performance. If it's running the antilock braking in your car, that's pretty bad.
    - ChrisC-POV 2y ago
      That's waaaay too broad a grouping. You're thinking of PC's, basically, in one form or another (or their modern replacement, mobile devices). Industrial automation and hard real-time embedded devices like those that control surfaces in Fly-By-Wire systems are out there and with those it's not a matter of "throughput" but a matter of timeliness. I did a major project with QNX about a decade ago where the maximum allowed latency between any device on the (fiber-optic) network being asked a question by the master controller and the reply - then the processing of the reply - was less than 1 millisecond, full stop. Come in late and most of the time the sky doesn't fall in. But on the occasions that it does, you potentially have a 4 meter wide ribbon of red-hot steel flying out of the stands at 50mph. Or you blow one of the transistors that controls the 10-20 megawatt DC motors in each stand. (These "transistors" were boxes about 7' tall, filled with oil, and cost northwards of $200k). Sometimes it just doesn't pay to be late.
      - CoastalCoder 2y ago
        What a terrifying and cool project! Can you say what kind of shop / machine it was for?
        ChrisC-POV 2y ago
        It was a hot-strip mill; this video shows the general gist of it: <https://www.youtube.com/watch?v=13dUL9OQFvw&t=15s https://www.youtube.com/watch?v=13dUL9OQFvw&t=15s> (tho the ones I worked with were all made by GE).
    - caskstrength 2y ago
      > For everything else, including desktop and mobile operating systems realtime sounds like a good idea. Why on earth would you need real time on desktop? Every time the topic is discussed on internet bunch of confused people chime in with this sentiment that doesn't make much sense. "RTOS" is not some magic that somehow makes everything on your desktop fast. All it would do in reality is make everything slower for 99% of your interaction but guarantee that "slowness" is uniform and you don't have any weird latency spikes in other 1%. Note that for cases that are not "nuclear reactor control system that requires Very Certified OS with audited and provable reaction times" RTLinux is already available, but distros are not inclined to leverage it on desktop for reason described above.
      - vbezhenar 2y ago
        I've read many opinions of those who used QNX on desktop in the past, that it felt much more "snappy" compared to other systems. Desktop definitely has real time tasks. Any animation is real time task. When I'm listening to music, it's real time task. macOS sound interrupts for a short time, when something hardware-related happens, so it's not pleasant experience. You might argue that user can tolerate those interruptions, but for me those are just bugs.
        vacuity 2y ago
        Indeed. There are people who have a fair point that desktops for users (as opposed to servers) should have a focus on being responsive and useful. Like how writing on a piece of paper produces meaningful markings in real time, user interfaces should be able to perform tasks with analogous interactivity. See comments about BeOS elsewhere in this thread. I don't know about the extreme, but the overall sentiment is valuable and a far cry from the current state of the desktop experience.
      - AndriyKunitsyn 2y ago
        When recording live audio performances, some guarantees that the thread that records the signal will always get some attention for the scheduler, would be nice. Otherwise the recording program will miss on the signal, and the recording will have cracks. We sidestep this problem because our CPUs are insanely fast for this task, so as long as you don't game, compile and record audio at the same time, you should be fine. But we don't have any guarantees.
    - RandomThoughts3 2y ago
      > Who needs throughput? Server software, but Linux is already dominant there and everyone's happy. For everything else, including desktop and mobile operating systems realtime sounds like a good idea. No, not at all. The actual question is who needs realtime. Desktop is a typical place where you absolutely don't need realtime. You just want latency to be under the user perception threshold but that's easily achievable with a standard kernel. Plus the consequence of failing is basically nil. Why bother with the complexity introduced by a RT kernel? The latency in modern desktop is not a consequence of unpredictable scheduling. It's just poorly optimised applications. A RT kernel is not a magical solution to that.
      - cesarb 2y ago
        > Desktop is a typical place where you absolutely don't need realtime. On a desktop, you want at least soft real-time when playing any sort of multimedia (audio, video, or interactive applications like video games). Otherwise, you get things like glitched audio or dropped frames.
        RandomThoughts3 2y ago
        Yes, I agree especially for audio which is a typical exemple of a latency sensitive application (video games are generally purposefully design to be as latency insensitive as possible and will drop frames with no impact on actual gameplay). But a performent application and proper scheduler priorities will bring you there without a RT kernel.
    - chronogram 2y ago
      Desktop use at its lowest latency is typically 17ms and up to 33ms when using a 60Hz screen with one frame in the buffer and whether you have to wait for the one after that. But most operating systems and games default to more than one buffered frame. Still, even that lowest case scenario is a pretty big budget of time. The microsecond precision of a RTOS is not necessary then. If you want to increase responsiveness or reduce latency for interactive use of desktops or phones there are different avenues to approach to shave off milliseconds at a time.
mwcampbell 2y ago
> Somebody really should write a microkernel like this in Rust. I wonder how far Redox's microkernel design is from what that comment describes.
- Teever 2y ago
  A friend of mine is working on this OS: https://gitlab.com/uxrt https://gitlab.com/uxrt
  - tcbawo 2y ago
    I see this was submitted to HN a couple times over the last few years and has not received any comments. Can you describe the goals and status of this project? Does it fit into any larger ecosystem? Is it a research project or intended for eventual mainstream adoption?
    - Teever 2y ago
      I'll start by saying that micro-kernel OS development is a bit beyond my skill set but I can tell you that the goal is more or less to make an open source QNX with features from Plan 9 that is suitable for desktop use and higher level embedded platforms. I'm not sure what you mean by larger ecosystem but a compatibility layer with Linux binaries is on the feature list. You can read the extremely detailed architecture notes document here[0] It is absolutely intended for mainstream adoption and the project has recently received interest and funding from a very large OEM. I don't want to say who it is because I am not certain if they want it publicized yet. It's probably okay for me to say but I'm erring on the side of caution. If you or anyone you know is interested in contributing to the project let me know or reach out to Andrew Warkentin on gitlab.[1] [0] https://gitlab.com/uxrt/uxrt-toplevel/-/blob/master/architecture_notes?ref_type=heads https://gitlab.com/uxrt/uxrt-toplevel/-/blob/master/architec... [1] https://gitlab.com/andreww591 https://gitlab.com/andreww591
    - andreww591 2y ago
      Basically I'm trying to write a better Linux than Linux and a better QNX than QNX, at least for certain use cases that is. At least initially, my primary focus is on workstation/enthusiast desktops and higher-end embedded systems (some types of servers, especially ones where high security is important, are also a focus as well but maybe not quite as much for now). Pure consumer desktops aren't really much of a priority; the goal is to make something that's approachable but definitely not dumbed down. Almost every day I see people complaining about mainstream OSes and I think there's an opportunity for an OS that's cleaner and more integrated than Linux and has various advanced features not found in Linux, but still tries to maintain a high degree of compatibility with it. I'm not really trying to clone QNX or Linux 1:1, but I am trying to maintain similar enough semantics to allow easy implementation of compatibility layers for both systems. At the moment I have a working microkernel and an incomplete process server with a partially-implemented VFS but no support for running regular user processes. Currently I am working on some of the first special filesystems built into the process server (specifically an early log/console device at the moment). The kernel is a hard fork of seL4 (the biggest addition at the moment is a preliminary implementation of QNX-style arbitrary-length arbitrary-address message copying); I've forked it because seL4 is focused on lower-end static embedded systems, and full verification of just the kernel isn't really particularly useful for a dynamic OS when all of the servers are unverified. The process server is written in Rust, and it's likely most of the other low-level servers will be as well, although I'm not really writing a "Rust OS" as such. Many of the basic utilities will most likely be BSD-derived, and some of the higher-level desktop-related stuff may be written in various other languages (haven't completely decided there).
tcbawo 2y ago
Thank you for adding that link. What I find particularly remarkable is that the author of that comment is John Nagle himself (of Nagle’s Algorithm fame https://en.m.wikipedia.org/wiki/Nagle%27s_algorithm https://en.m.wikipedia.org/wiki/Nagle%27s_algorithm).

For everyone else, your QNX comment from 2015: https://news.ycombinator.com/item?id=9872640 https://news.ycombinator.com/item?id=9872640

by Tomte 2y ago

For everyone else, your QNX comment from 2015: https://news.ycombinator.com/item?id=9872640 https://news.ycombinator.com/item?id=9872640

stevesimmons 2y ago

As someone who knows only that QNX is a RTOS and that "real time" is good in some vague hand-wavy way, I found this 2015 comment a very succinct, concrete and illuminating description of one aspect of QNX's architecture that makes it properly real-time. So thank you very much for the link.

scheme271 2y ago
"Real time" is only good if you need guaranteed responsiveness within certain time limits. It can actually be suboptimal and slow you down if you don't need it and want to optimize other things like throughput.
- AlotOfReading 2y ago
  Though in practice, the act of designing your code for realtime tends to produce better performance than the status quo of doing nothing. Most uses of this warning seem intended to promote the latter rather than to get people thinking about their actual requirements.
- vbezhenar 2y ago
  Who needs throughput? Server software, but Linux is already dominant there and everyone's happy. For everything else, including desktop and mobile operating systems realtime sounds like a good idea.
  - mlyle 2y ago
    Absolutely deterministic latency isn't a requirement for the desktop like it is for industrial control. If 99% of the time performance is faster, and the remaining 1% of the time isn't that bad, most people will take the trade of higher average performance. If it's running the antilock braking in your car, that's pretty bad.
  - ChrisC-POV 2y ago
    That's waaaay too broad a grouping. You're thinking of PC's, basically, in one form or another (or their modern replacement, mobile devices). Industrial automation and hard real-time embedded devices like those that control surfaces in Fly-By-Wire systems are out there and with those it's not a matter of "throughput" but a matter of timeliness. I did a major project with QNX about a decade ago where the maximum allowed latency between any device on the (fiber-optic) network being asked a question by the master controller and the reply - then the processing of the reply - was less than 1 millisecond, full stop. Come in late and most of the time the sky doesn't fall in. But on the occasions that it does, you potentially have a 4 meter wide ribbon of red-hot steel flying out of the stands at 50mph. Or you blow one of the transistors that controls the 10-20 megawatt DC motors in each stand. (These "transistors" were boxes about 7' tall, filled with oil, and cost northwards of $200k). Sometimes it just doesn't pay to be late.
    - CoastalCoder 2y ago
      What a terrifying and cool project! Can you say what kind of shop / machine it was for?
      - ChrisC-POV 2y ago
        It was a hot-strip mill; this video shows the general gist of it: <https://www.youtube.com/watch?v=13dUL9OQFvw&t=15s https://www.youtube.com/watch?v=13dUL9OQFvw&t=15s> (tho the ones I worked with were all made by GE).
  - caskstrength 2y ago
    > For everything else, including desktop and mobile operating systems realtime sounds like a good idea. Why on earth would you need real time on desktop? Every time the topic is discussed on internet bunch of confused people chime in with this sentiment that doesn't make much sense. "RTOS" is not some magic that somehow makes everything on your desktop fast. All it would do in reality is make everything slower for 99% of your interaction but guarantee that "slowness" is uniform and you don't have any weird latency spikes in other 1%. Note that for cases that are not "nuclear reactor control system that requires Very Certified OS with audited and provable reaction times" RTLinux is already available, but distros are not inclined to leverage it on desktop for reason described above.
    - vbezhenar 2y ago
      I've read many opinions of those who used QNX on desktop in the past, that it felt much more "snappy" compared to other systems. Desktop definitely has real time tasks. Any animation is real time task. When I'm listening to music, it's real time task. macOS sound interrupts for a short time, when something hardware-related happens, so it's not pleasant experience. You might argue that user can tolerate those interruptions, but for me those are just bugs.
      - vacuity 2y ago
        Indeed. There are people who have a fair point that desktops for users (as opposed to servers) should have a focus on being responsive and useful. Like how writing on a piece of paper produces meaningful markings in real time, user interfaces should be able to perform tasks with analogous interactivity. See comments about BeOS elsewhere in this thread. I don't know about the extreme, but the overall sentiment is valuable and a far cry from the current state of the desktop experience.
    - AndriyKunitsyn 2y ago
      When recording live audio performances, some guarantees that the thread that records the signal will always get some attention for the scheduler, would be nice. Otherwise the recording program will miss on the signal, and the recording will have cracks. We sidestep this problem because our CPUs are insanely fast for this task, so as long as you don't game, compile and record audio at the same time, you should be fine. But we don't have any guarantees.
  - RandomThoughts3 2y ago
    > Who needs throughput? Server software, but Linux is already dominant there and everyone's happy. For everything else, including desktop and mobile operating systems realtime sounds like a good idea. No, not at all. The actual question is who needs realtime. Desktop is a typical place where you absolutely don't need realtime. You just want latency to be under the user perception threshold but that's easily achievable with a standard kernel. Plus the consequence of failing is basically nil. Why bother with the complexity introduced by a RT kernel? The latency in modern desktop is not a consequence of unpredictable scheduling. It's just poorly optimised applications. A RT kernel is not a magical solution to that.
    - cesarb 2y ago
      > Desktop is a typical place where you absolutely don't need realtime. On a desktop, you want at least soft real-time when playing any sort of multimedia (audio, video, or interactive applications like video games). Otherwise, you get things like glitched audio or dropped frames.
      - RandomThoughts3 2y ago
        Yes, I agree especially for audio which is a typical exemple of a latency sensitive application (video games are generally purposefully design to be as latency insensitive as possible and will drop frames with no impact on actual gameplay). But a performent application and proper scheduler priorities will bring you there without a RT kernel.
  - chronogram 2y ago
    Desktop use at its lowest latency is typically 17ms and up to 33ms when using a 60Hz screen with one frame in the buffer and whether you have to wait for the one after that. But most operating systems and games default to more than one buffered frame. Still, even that lowest case scenario is a pretty big budget of time. The microsecond precision of a RTOS is not necessary then. If you want to increase responsiveness or reduce latency for interactive use of desktops or phones there are different avenues to approach to shave off milliseconds at a time.

mwcampbell 2y ago

> Somebody really should write a microkernel like this in Rust. I wonder how far Redox's microkernel design is from what that comment describes.

Teever 2y ago
A friend of mine is working on this OS: https://gitlab.com/uxrt https://gitlab.com/uxrt
- tcbawo 2y ago
  I see this was submitted to HN a couple times over the last few years and has not received any comments. Can you describe the goals and status of this project? Does it fit into any larger ecosystem? Is it a research project or intended for eventual mainstream adoption?
  - Teever 2y ago
    I'll start by saying that micro-kernel OS development is a bit beyond my skill set but I can tell you that the goal is more or less to make an open source QNX with features from Plan 9 that is suitable for desktop use and higher level embedded platforms. I'm not sure what you mean by larger ecosystem but a compatibility layer with Linux binaries is on the feature list. You can read the extremely detailed architecture notes document here[0] It is absolutely intended for mainstream adoption and the project has recently received interest and funding from a very large OEM. I don't want to say who it is because I am not certain if they want it publicized yet. It's probably okay for me to say but I'm erring on the side of caution. If you or anyone you know is interested in contributing to the project let me know or reach out to Andrew Warkentin on gitlab.[1] [0] https://gitlab.com/uxrt/uxrt-toplevel/-/blob/master/architecture_notes?ref_type=heads https://gitlab.com/uxrt/uxrt-toplevel/-/blob/master/architec... [1] https://gitlab.com/andreww591 https://gitlab.com/andreww591
  - andreww591 2y ago
    Basically I'm trying to write a better Linux than Linux and a better QNX than QNX, at least for certain use cases that is. At least initially, my primary focus is on workstation/enthusiast desktops and higher-end embedded systems (some types of servers, especially ones where high security is important, are also a focus as well but maybe not quite as much for now). Pure consumer desktops aren't really much of a priority; the goal is to make something that's approachable but definitely not dumbed down. Almost every day I see people complaining about mainstream OSes and I think there's an opportunity for an OS that's cleaner and more integrated than Linux and has various advanced features not found in Linux, but still tries to maintain a high degree of compatibility with it. I'm not really trying to clone QNX or Linux 1:1, but I am trying to maintain similar enough semantics to allow easy implementation of compatibility layers for both systems. At the moment I have a working microkernel and an incomplete process server with a partially-implemented VFS but no support for running regular user processes. Currently I am working on some of the first special filesystems built into the process server (specifically an early log/console device at the moment). The kernel is a hard fork of seL4 (the biggest addition at the moment is a preliminary implementation of QNX-style arbitrary-length arbitrary-address message copying); I've forked it because seL4 is focused on lower-end static embedded systems, and full verification of just the kernel isn't really particularly useful for a dynamic OS when all of the servers are unverified. The process server is written in Rust, and it's likely most of the other low-level servers will be as well, although I'm not really writing a "Rust OS" as such. Many of the basic utilities will most likely be BSD-derived, and some of the higher-level desktop-related stuff may be written in various other languages (haven't completely decided there).

tcbawo 2y ago

Thank you for adding that link. What I find particularly remarkable is that the author of that comment is John Nagle himself (of Nagle’s Algorithm fame https://en.m.wikipedia.org/wiki/Nagle%27s_algorithm https://en.m.wikipedia.org/wiki/Nagle%27s_algorithm).