3 ms·

I have always heard that the windows kernel is actually pretty well designed. In particular the way it handles concurrency and asynchronous I/O. Unfortunately I

by srean 11y ago

I have always heard that the windows kernel is actually pretty well designed. In particular the way it handles concurrency and asynchronous I/O. Unfortunately I have not found (euphemism for: I have been lazy) an easily consumable description of its differences from the Unix, Linux APIs supporting concurrency and asynchrony features, in what ways are they better, with some code examples thrown in that illustrate the differences If anyone would be so kind to humor me with such a comparison or pointer...

Q6T46nT668w6i3m 11y ago
Windows Internals is the canonical resource. (I bet you would enjoy the concurrency and I/O sections from The Be Book and The Design and Implementation of the FreeBSD Operating System too.)
- twoodfin 11y ago
  Is The Be Book available in PDF or ebook format anywhere?
  - mattds 11y ago
    I believe you can find something about BeOS here: https://www.haiku-os.org/legacy-docs/bebook/index.html https://www.haiku-os.org/legacy-docs/bebook/index.html
- nickpsecurity 11y ago
  Specifically as described here: https://users.cs.jmu.edu/abzugcx/Public/Student-Produced-Term-Projects/Operating-Systems-2003-FALL/BeOS-by-Nicole-Chung-Brian-Hoffman-Robert-Robinson-AJ-Schuster-2003-Fall.doc https://users.cs.jmu.edu/abzugcx/Public/Student-Produced-Ter... The benaphores were a brilliant idea. Plus, the general threading approach that avoided issues with direct, pointer manipulation. Add microkernels, memory protection, and threading all the way down for quite a unique kernel with incredible performance on old hardware. I'd be curious what a vanilla port of BeOS kernel on modern, multicore hardware would do vs a typical OS in fair benchmarks. We'd also include a little note that only one does it relatively safely. ;)
ksk 11y ago
Well you have to be careful with what you're comparing. It would be rather unfair to compare the original UNIX design with a modern design like NT. UNIX did not have async I/O. It even blocked syscalls in its original design. I think Linux added async I/O quite recently (2.6?). I'm not really a kernel dev, but I have had to work on systems level stuff quite often on both OSs and from what I've seen NT has a much more richer I/O model. The I/O class/device/filter/driver model of NT makes writing driver code less of a chore IMHO. Also async I/O is supported on almost all I/O devices. I believe its only network and disk on Linux.
- nickpsecurity 11y ago
  UNIX intentionally deviated from monolithic systems that had many useful features because they couldn't run on their hardware. Then, they kept that even when hardware got better with some added back in band-aid on ways and some others cleanly. Windows NT's design came from experience and strategies of OpenVMS's designer. No wonder that a cleaner, monolithic design came out of it. Took them a while to get the reliability near their ancestor, though. Haha. Meanwhile, QNX and Minix 3 took a more UNIX philosophy approach than UNIX by decomposing the whole thing into communicating components. QNX did it fast and ultra-reliable where Minix 3 is at least reliable. Easier to update due to architecture, even live updates. They each did this with little manpower and resources compared to what went into UNIX's. My direct comparison for UNIX is with them. It shows UNIX architecture is fundamentally flawed in terms of flexibility, reliability, security, determinism, and ease of maintenance. I'd compare Windows kernel with other monoliths on servers and desktops. Usability and stability are better than most that are still around with other attributes hit and miss (or comparable). UNIX desktops have gotten much better than before but still too many WTF moments. And more dependency hell than I used to deal with on Windows. (sighs)
  - richardwhiuk 11y ago
    Dependency hell has very little to do with kernel design - it's entirely a user mode problem. Neither Linux or Windows NT in the modern iterations are true monolithic kernels - e.g. on Linux FUSE allows filesystems in user space, loadable and unloadable kernel modules, Windows has a User mode drivers framework which is increasingly used post Vista. Neither QNX or Minix are as feature full as Linux or Windows NT - almost all development on Linux and Windows is in driver development, not core kernel work (e.g. scheduler, IO system, memory manager). Windows NT remains a more modern kernel (in terms of internal consistent object model, basic kernel primitives) than either BSD or Linux, but whether it's absolutely better is a harder argument. In general the argument regarding micro/monolithic kernels is old hat in practice we can make extra-ordinarily reliable monolithic kernels, while we can also avoid a lot of the pain associated with microkernels - e.g. minimising context switch time. The reliability, security, flexibility argument is largely made redundant by man-power dropped into modern kernels - better use Linux because although it's theoretically inferior it has 1000s of days of development thrown at it. Kernel development also appears to be transitioning to using safety in compilers to improve reliability, but that's been a research topic for decades - the answer to 'can we build a sufficiently smart compiler' has been 'maybe' for a long time - though LLVM and Rust's compiler show a way forwards.

by srean 11y ago

Q6T46nT668w6i3m 11y ago

Windows Internals is the canonical resource. (I bet you would enjoy the concurrency and I/O sections from The Be Book and The Design and Implementation of the FreeBSD Operating System too.)

twoodfin 11y ago
Is The Be Book available in PDF or ebook format anywhere?
- mattds 11y ago
  I believe you can find something about BeOS here: https://www.haiku-os.org/legacy-docs/bebook/index.html https://www.haiku-os.org/legacy-docs/bebook/index.html
nickpsecurity 11y ago
Specifically as described here: https://users.cs.jmu.edu/abzugcx/Public/Student-Produced-Term-Projects/Operating-Systems-2003-FALL/BeOS-by-Nicole-Chung-Brian-Hoffman-Robert-Robinson-AJ-Schuster-2003-Fall.doc https://users.cs.jmu.edu/abzugcx/Public/Student-Produced-Ter... The benaphores were a brilliant idea. Plus, the general threading approach that avoided issues with direct, pointer manipulation. Add microkernels, memory protection, and threading all the way down for quite a unique kernel with incredible performance on old hardware. I'd be curious what a vanilla port of BeOS kernel on modern, multicore hardware would do vs a typical OS in fair benchmarks. We'd also include a little note that only one does it relatively safely. ;)

ksk 11y ago

Well you have to be careful with what you're comparing. It would be rather unfair to compare the original UNIX design with a modern design like NT. UNIX did not have async I/O. It even blocked syscalls in its original design. I think Linux added async I/O quite recently (2.6?). I'm not really a kernel dev, but I have had to work on systems level stuff quite often on both OSs and from what I've seen NT has a much more richer I/O model. The I/O class/device/filter/driver model of NT makes writing driver code less of a chore IMHO. Also async I/O is supported on almost all I/O devices. I believe its only network and disk on Linux.

nickpsecurity 11y ago
UNIX intentionally deviated from monolithic systems that had many useful features because they couldn't run on their hardware. Then, they kept that even when hardware got better with some added back in band-aid on ways and some others cleanly. Windows NT's design came from experience and strategies of OpenVMS's designer. No wonder that a cleaner, monolithic design came out of it. Took them a while to get the reliability near their ancestor, though. Haha. Meanwhile, QNX and Minix 3 took a more UNIX philosophy approach than UNIX by decomposing the whole thing into communicating components. QNX did it fast and ultra-reliable where Minix 3 is at least reliable. Easier to update due to architecture, even live updates. They each did this with little manpower and resources compared to what went into UNIX's. My direct comparison for UNIX is with them. It shows UNIX architecture is fundamentally flawed in terms of flexibility, reliability, security, determinism, and ease of maintenance. I'd compare Windows kernel with other monoliths on servers and desktops. Usability and stability are better than most that are still around with other attributes hit and miss (or comparable). UNIX desktops have gotten much better than before but still too many WTF moments. And more dependency hell than I used to deal with on Windows. (sighs)
- richardwhiuk 11y ago
  Dependency hell has very little to do with kernel design - it's entirely a user mode problem. Neither Linux or Windows NT in the modern iterations are true monolithic kernels - e.g. on Linux FUSE allows filesystems in user space, loadable and unloadable kernel modules, Windows has a User mode drivers framework which is increasingly used post Vista. Neither QNX or Minix are as feature full as Linux or Windows NT - almost all development on Linux and Windows is in driver development, not core kernel work (e.g. scheduler, IO system, memory manager). Windows NT remains a more modern kernel (in terms of internal consistent object model, basic kernel primitives) than either BSD or Linux, but whether it's absolutely better is a harder argument. In general the argument regarding micro/monolithic kernels is old hat in practice we can make extra-ordinarily reliable monolithic kernels, while we can also avoid a lot of the pain associated with microkernels - e.g. minimising context switch time. The reliability, security, flexibility argument is largely made redundant by man-power dropped into modern kernels - better use Linux because although it's theoretically inferior it has 1000s of days of development thrown at it. Kernel development also appears to be transitioning to using safety in compilers to improve reliability, but that's been a research topic for decades - the answer to 'can we build a sufficiently smart compiler' has been 'maybe' for a long time - though LLVM and Rust's compiler show a way forwards.