4 ms·

What current work (that might have a hope of becoming mainstream for some subset of devices) do you see that does this well?

by thom 8y ago

TheDong 8y ago
Binder on android shares some similarities. It's a low level RPC mechanism, it does reschedule to the new process immediately without a trip through the scheduler when it's used, it has similar blocking semantics, etc. It's also worth following the "bus1" work for linux which may end up being quite similar as well.
Retr0spectrum 8y ago
QNX itself had a hope of becoming mainstream as the core of Blackberry 10 OS. Unfortunately, Blackberry has moved over to Android these days.
- jschwartzi 8y ago
  QNX is mainstream as hell, just not anywhere you'd expect there to be an OS. It's an RTOS and gets used in automotive, medical, and network technologies where software latency needs to be tightly controlled. You wouldn't use it in the same place you'd use Linux anyway.
  - thom 8y ago
    This is true. I suppose I'm just curious about more general (server/desktop/mobile) use cases, because the IPC and scheduling issues the parent mentions must be extremely difficult to predict with a wide array of workloads.
    - lsaferite 8y ago
      I remember using the single disk QNX demo back in the day and being highly impressed with the functionality the crammed into that disk. Sad that it never went anywhere as a desktop/server OS.
      - Animats 8y ago
        I ran QNX as a desktop OS for three years when I was working on a DARPA Grand Challenge vehicle. The vehicle ran QNX, so the development systems did, too. It had an early version of Firefox, the Eclipse IDE, Thunderbird mail, and all the Gnu command line tools. Sadly, QNX under Blackberry no longer offers the desktop environment. And QNX's open source/closed source/open source/closed source transitions angered the development community so much that people stopped offering QNX versions of UNIX/Linux software. The most striking thing about running QNX on the desktop was the absolutely consistent response. It doesn't page, so everything is in memory. Going back to Linux felt so laggy. You pay about a 10%-20% overhead cost for all that interprocess communication. It's not a big deal on modern processors, because when you send a message, the receiving process is running on the same CPU and the data is usually in L1 cache since the sender just put it there. So copying is cheap. Incidentally, all the Boston Dynamics robots run QNX. They're coordinating all those limbs and valves from one CPU. It's not distributed; that would be too uncoordinated. Valve update is at 1KHz; balance update is at 100Hz.
        beagle3 8y ago
        > And QNX's open source/closed source/open source/closed source transitions angered the development community so much that people stopped offering QNX versions of UNIX/Linux software. Was it ever really open? I can't find copies of any version (regardless of how old)
        Animats 8y ago
        Yes.[1] Source made available in 2007. Source closed without warning after the Blackberry acquisition in 2010. QNX is doing well in automotive.[2] QNX is the part of Blackberry that's making money. [1] http://www.qnx.com/news/pr_2471_1.html http://www.qnx.com/news/pr_2471_1.html [2] http://canada.autonews.com/article/20180622/CANADA/180629921/blackberry-reports-q1-loss-but-qnx-auto-software-division-grows-31 http://canada.autonews.com/article/20180622/CANADA/180629921...
        beagle3 8y ago
    - jschwartzi 8y ago
      It's really not designed for those use cases though. We were using it for a GUI and or finding was that without constraining the resource usage of the GUI using Adaptive Partitioning user-driven workloads could stomp all over the lower-priority processes. This effectively meant that a poorly-configured system would eventually trip the watchdog due to user input. QNX does include features to control this but you have to know enough about it to realize you want them. And POSIX priorities are not enough to prevent this from happening because they are solving a different problem. You might find it useful in servers where controllable latency is more important than responsiveness, but I wouldn't use it in a user-driven workload like a desktop OS without very carefully configuring it. Out-of-the box it can be very unstable.
akavel 8y ago
Not sure what API approach they use, but in the area of microkernels: - GenodeOS is reportedly used in production by some commercial clients, though mostly undisclosed IIRC; it is a higher-level layer (~OS) compatible with numerous microkernels - Fuchsia OS - in dev; it's a recent development by Google; as much as Google is officially silent about it (I understand they're not sure how the experiment will work out for them), observers assume it's most probably hoped to be used as a successor to Android - Redox OS - in dev; no concrete news of mainstream usage plans I know of, but has some mindshare among developers - Minix 3 - used in production; infamously by Intel in their IME
- roblabla 8y ago
  The Nintendo Switch's Kernel, Horizon/NX , is an example of a microkernel, tailored for their specific use-case, and is in wide use[0]. It is, sadly, closed source, however it has been reverse engineered. Their IPC API is, I believe, pretty smart and elegant: - There is a per-thread IPC zone of 0x100 bytes. When doing IPC, the request is serialized and put into this. If bigger data than 0x100 bytes is necessary, pointers are passed around, and the Kernel maps it into the process servicing the call. - svcSendSyncRequest is used to call an IPC. It is a synchronous API that will block until the process servicing the call replies to it. - svcReplyAndReceive is used to receive an IPC request and reply to a request, and then wait until a new one is received. The syscalls are "merged" into a single one to avoid the syscall overhead: Almost all svcReply will be followed by an svcReceive, so merging them into a single call makes a lot of sense. You can find more information about the SVCs at [1] and the IPC layout at [2]. [0] Preempting the "I thought the switch used BSD": wikipedia was wrong, the OS is completely custom, and the kernel is tailor-made. [1] http://switchbrew.org/index.php?title=SVC http://switchbrew.org/index.php?title=SVC [2] http://switchbrew.org/index.php?title=IPC_Marshalling http://switchbrew.org/index.php?title=IPC_Marshalling
  - beefhash 8y ago
    > and the kernel is tailor-made A little note about this: Horizon/NX actually traces back to the Horizon OS on the Nintendo 3DS. The IPC marshalling was significantly more simple back then[1]. In all honesty, I'm not sure what made Nintendo thing the IPC marshalling on the NX was a good idea; to me it just looks like a hastily-designed mess (cf. "This one is packed even worse than A, they inserted the bit38-36 of the address on top of the counter field." on the [2] page linked by the parent comment). However, the NX incarnation was designed with "naturally" wrapping C++ methods in mind and it does a fairly decent job at that. [1] https://www.3dbrew.org/wiki/IPC#Message_Structure https://www.3dbrew.org/wiki/IPC#Message_Structure (Shoutouts to 3dbrew actually realizing that HTTPS is a thing that exists and maybe should be deployed.)

What current work (that might have a hope of becoming mainstream for some subset of devices) do you see that does this well?

by thom 8y ago

What current work (that might have a hope of becoming mainstream for some subset of devices) do you see that does this well?

TheDong 8y ago

Binder on android shares some similarities. It's a low level RPC mechanism, it does reschedule to the new process immediately without a trip through the scheduler when it's used, it has similar blocking semantics, etc. It's also worth following the "bus1" work for linux which may end up being quite similar as well.

Retr0spectrum 8y ago

QNX itself had a hope of becoming mainstream as the core of Blackberry 10 OS. Unfortunately, Blackberry has moved over to Android these days.

jschwartzi 8y ago
QNX is mainstream as hell, just not anywhere you'd expect there to be an OS. It's an RTOS and gets used in automotive, medical, and network technologies where software latency needs to be tightly controlled. You wouldn't use it in the same place you'd use Linux anyway.
- thom 8y ago
  This is true. I suppose I'm just curious about more general (server/desktop/mobile) use cases, because the IPC and scheduling issues the parent mentions must be extremely difficult to predict with a wide array of workloads.
  - lsaferite 8y ago
    I remember using the single disk QNX demo back in the day and being highly impressed with the functionality the crammed into that disk. Sad that it never went anywhere as a desktop/server OS.
    - Animats 8y ago
      I ran QNX as a desktop OS for three years when I was working on a DARPA Grand Challenge vehicle. The vehicle ran QNX, so the development systems did, too. It had an early version of Firefox, the Eclipse IDE, Thunderbird mail, and all the Gnu command line tools. Sadly, QNX under Blackberry no longer offers the desktop environment. And QNX's open source/closed source/open source/closed source transitions angered the development community so much that people stopped offering QNX versions of UNIX/Linux software. The most striking thing about running QNX on the desktop was the absolutely consistent response. It doesn't page, so everything is in memory. Going back to Linux felt so laggy. You pay about a 10%-20% overhead cost for all that interprocess communication. It's not a big deal on modern processors, because when you send a message, the receiving process is running on the same CPU and the data is usually in L1 cache since the sender just put it there. So copying is cheap. Incidentally, all the Boston Dynamics robots run QNX. They're coordinating all those limbs and valves from one CPU. It's not distributed; that would be too uncoordinated. Valve update is at 1KHz; balance update is at 100Hz.
      - beagle3 8y ago
        > And QNX's open source/closed source/open source/closed source transitions angered the development community so much that people stopped offering QNX versions of UNIX/Linux software. Was it ever really open? I can't find copies of any version (regardless of how old)
        Animats 8y ago
        Yes.[1] Source made available in 2007. Source closed without warning after the Blackberry acquisition in 2010. QNX is doing well in automotive.[2] QNX is the part of Blackberry that's making money. [1] http://www.qnx.com/news/pr_2471_1.html http://www.qnx.com/news/pr_2471_1.html [2] http://canada.autonews.com/article/20180622/CANADA/180629921/blackberry-reports-q1-loss-but-qnx-auto-software-division-grows-31 http://canada.autonews.com/article/20180622/CANADA/180629921...
        beagle3 8y ago
  - jschwartzi 8y ago
    It's really not designed for those use cases though. We were using it for a GUI and or finding was that without constraining the resource usage of the GUI using Adaptive Partitioning user-driven workloads could stomp all over the lower-priority processes. This effectively meant that a poorly-configured system would eventually trip the watchdog due to user input. QNX does include features to control this but you have to know enough about it to realize you want them. And POSIX priorities are not enough to prevent this from happening because they are solving a different problem. You might find it useful in servers where controllable latency is more important than responsiveness, but I wouldn't use it in a user-driven workload like a desktop OS without very carefully configuring it. Out-of-the box it can be very unstable.

akavel 8y ago

Not sure what API approach they use, but in the area of microkernels: - GenodeOS is reportedly used in production by some commercial clients, though mostly undisclosed IIRC; it is a higher-level layer (~OS) compatible with numerous microkernels - Fuchsia OS - in dev; it's a recent development by Google; as much as Google is officially silent about it (I understand they're not sure how the experiment will work out for them), observers assume it's most probably hoped to be used as a successor to Android - Redox OS - in dev; no concrete news of mainstream usage plans I know of, but has some mindshare among developers - Minix 3 - used in production; infamously by Intel in their IME

roblabla 8y ago
The Nintendo Switch's Kernel, Horizon/NX , is an example of a microkernel, tailored for their specific use-case, and is in wide use[0]. It is, sadly, closed source, however it has been reverse engineered. Their IPC API is, I believe, pretty smart and elegant: - There is a per-thread IPC zone of 0x100 bytes. When doing IPC, the request is serialized and put into this. If bigger data than 0x100 bytes is necessary, pointers are passed around, and the Kernel maps it into the process servicing the call. - svcSendSyncRequest is used to call an IPC. It is a synchronous API that will block until the process servicing the call replies to it. - svcReplyAndReceive is used to receive an IPC request and reply to a request, and then wait until a new one is received. The syscalls are "merged" into a single one to avoid the syscall overhead: Almost all svcReply will be followed by an svcReceive, so merging them into a single call makes a lot of sense. You can find more information about the SVCs at [1] and the IPC layout at [2]. [0] Preempting the "I thought the switch used BSD": wikipedia was wrong, the OS is completely custom, and the kernel is tailor-made. [1] http://switchbrew.org/index.php?title=SVC http://switchbrew.org/index.php?title=SVC [2] http://switchbrew.org/index.php?title=IPC_Marshalling http://switchbrew.org/index.php?title=IPC_Marshalling
- beefhash 8y ago
  > and the kernel is tailor-made A little note about this: Horizon/NX actually traces back to the Horizon OS on the Nintendo 3DS. The IPC marshalling was significantly more simple back then[1]. In all honesty, I'm not sure what made Nintendo thing the IPC marshalling on the NX was a good idea; to me it just looks like a hastily-designed mess (cf. "This one is packed even worse than A, they inserted the bit38-36 of the address on top of the counter field." on the [2] page linked by the parent comment). However, the NX incarnation was designed with "naturally" wrapping C++ methods in mind and it does a fairly decent job at that. [1] https://www.3dbrew.org/wiki/IPC#Message_Structure https://www.3dbrew.org/wiki/IPC#Message_Structure (Shoutouts to 3dbrew actually realizing that HTTPS is a thing that exists and maybe should be deployed.)