9 ms·
There's a large debate whether a 'hybrid' kernel is an actual thing, and/or whether NT is just a monolithic kernel. The processes section should be expanded up
by nullindividual 2y ago
There's a large debate whether a 'hybrid' kernel is an actual thing, and/or whether NT is just a monolithic kernel.
The processes section should be expanded upon. The NT kernel doesn't execute processes, it executes _threads_. Threads can be created in a few milliseconds where as noted, processes are heavy weight; essentially the opposite of Unicies. This is a big distinction.
io_uring would be the first true non-blocking async I/O implementation on Unicies.
It should also be noted that while NT as a product is much newer than Unicies, it's history is rooted in VMS fundamentals thanks to it's core team of ex-Digital devs lead by David Cutler. This pulls back that 'feature history' by a decade or more. Still not as old as UNIX, but "old enough", one could argue.
[0] https://stackoverflow.com/questions/8768083/difference-between-posix-aio-and-libaio-on-linux https://stackoverflow.com/questions/8768083/difference-betwe...
- deleted 2y ago[deleted]
- mananaysiempre 2y ago> io_uring would be the first true non-blocking async I/O implementation on Unices. I would agree with that statement in isolation, except by that standard (no need for a syscall per operation) the first “true” asynchronous I/O API on NT would be I/O ring (2021), a very close copy of io_uring. (Registered I/O, introduced in Windows 8, does not qualify because it only works on sockets.) The original NT API is absolutely in the same class as the FreeBSD and Solaris ones, it’s just that Linux didn’t have a satisfactory one for a long long time.
- nullindividual 2y agoPOSIX AIO is not non-blocking async I/O; it can block other threads requesting the resource. IOCP is a true non-blocking async I/O. IOCP also extends to all forms of I/O (file, TCP socket, network, mail slot, pipes, etc.) instead of a particular type. POSIX AIO has usability problems also outlined in the previously linked thread. Remember, all I/O in NT is async at the kernel level. It's not a "bolt-on". IoRing is limited to file reads, unlike io_uring.
- mananaysiempre 2y agoPOSIX AIO + FreeBSD kqueue or Solaris ports are functionally equivalent to IOCP as far as I can tell.
- nullindividual 2y agoThis describes kqueue: https://speakerdeck.com/trent/pyparallel-how-we-removed-the-gil-and-exploited-all-cores?slide=37 https://speakerdeck.com/trent/pyparallel-how-we-removed-the-...
- trentnelson 2y agoI should do an updated version of that deck with io_uring and sans the PyParallel element. I still think it’s a good resource for depicting the differences in I/O between NT & UNIX. And yeah, IOCP has implicit awareness of concurrency, and can schedule optimal threads to service a port automatically. There hasn’t been a way to do that on UNIX until io_uring.
- nullindividual 2y agoYes, please! And if you're interested, RegisteredIO and I assume you'd drop in IoRing. In a nicely wrapped PDF :-)
- trentnelson 2y agoYeah I’d definitely include RegisteredIO and IoRing. When I was interviewing at Microsoft a few years back, I was actually interviewed by the chap that wrote RegisteredIO! Thought that was neat.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- trentnelson 2y agoNone of the UNIXes have the notion of WriteFile with an OVERLAPPED structure, that’s the key to NT’s asynchronous I/O. Nor do they have anything like IOCP, where the kernel is aware of the number of threads servicing a completion port, and can make sure you only have as many threads running as there are underlying cores, avoiding context switches. If you write your programs to leverage these facilities (which are very unique to NT), you can max perform your hardware very nicely.
- a-dub 2y agonotably the NT equivalent of select(): WaitForSingleObject and WaitForMultipleObjects had a benefit that one select/wait type syscall could be tickled by any of a network, file or the NT equivalent of a pthread signal.
- formerly_proven 2y ago> There's a large debate whether a 'hybrid' kernel is an actual thing, and/or whether NT is just a monolithic kernel. I don't think it's a concept that meaningfully exists. Microkernels are primarily concerned with isolating non-executive functions (e.g. device drivers) for stability and/or security (POLA) reasons. NT achieves virtually none of that (see e.g. Crowdstrike). The fact that Windows ships a thin user-mode syscall shim which largely consists of thin-to-nonexistent wrappers of NtXXX functions is architecturally uninteresting at best. Arguably binfmt_misc would then also make Linux a hybrid kernel.
- hernandipietro 2y agoOriginally, Windows NT 3.x was more "microkernelithic" as graphics and printer drivers where isolated. NT 4 moved them to Kernel mode to speedup the system.
- delta_p_delta_x 2y ago> The NT kernel doesn't execute processes, it executes _threads_ This is amongst the most important and visible differences between NT and Unix-likes, really. The key idea is that processes manage threads. Pavel Yosifovich in Windows 10 System Internals Part I puts it succinctly: A process is a containment and management object that represents a running instance of a program. The term “process runs” which is used fairly often, is inaccurate. Processes don’t run - processes manage. Threads are the ones that execute code and technically run. NtCreateProcess is extremely expensive and its direct use strongly discouraged (but Cygwin and MSYS2, in their IMO misguided intention to force Unix paradigms onto Windows, wrote fork() anyway), but thread creation and management is extremely straightforward, and the Windows threading API is as a result much nicer than pthreads.
- PaulDavisThe1st 2y agoIt is hard to accept that this is written by someone with any idea about how Linux works (as a Unix). A process (really, a "task") is a containment and management object that represents a running instance of a program. A program ("task") does not run, its threads do. The significant difference between Windows-related OS kernels and Unix-y ones is that process creation is much more heavyweight on the former. Nevertheless, on both types of systems, it is threads that execute code and technically run.
- jjtheblunt 2y agoMach before Linux was using task and threads as you seem to attribute to Linux. A nice historical overview is… https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/KernelProgramming/Mach/Mach.html https://developer.apple.com/library/archive/documentation/Da...
- PaulDavisThe1st 2y agoI worked on Mach in the early 1990s :) For example: https://www.usenix.org/conference/usenix-mach-iii-symposium/adding-scheduler-activations-mach-30 https://www.usenix.org/conference/usenix-mach-iii-symposium/...
- PaulDavisThe1st 2y ago> The processes section should be expanded upon. The NT kernel doesn't execute processes, it executes _threads_. Threads can be created in a few milliseconds where as noted, processes are heavy weight; essentially the opposite of Unicies. This is a big distinction. I am not sure what point you are attempting to make here. As written, it is more or less completely wrong. NT and Unix kernels both execute threads. Threads can be created in a few microseconds. Processes are heavy weight on both NT and Unix kernels. The only thing I can think of is the long-standing idea that Unix tends to encourage creating new processes and Windows-related OS kernels tend to encourage creating new threads. This is not false - process creation on Windows-related OS kernels is an extremely heavyweight process, certainly comparing it with any Unix. But it doesn't make the quote from you above correct. On a separate note, the state of things at the point of creation of NT is really of very little interest other than than to computer historians. It has been more than 30 years, and every still-available Unix and presumably NT have continued to evolve since then. Linux has dozens to hundreds of design features in it that did not exist in any Unix when NT was released (and did not exist in NT either).
- epcoa 2y agoProcesses and threads on NT are distinct nominative types of objects (in a system where “object” has a much more precise meaning) and the GP is at least correct that the former are not schedulable entities. This distinction doesn’t really exist on Linux for instance where there are at one approximation on the user side only processes (at least to use the verbiage of the clone syscall - look elsewhere and they’re threads in part due to having to support pthreads), and the scheduler schedules “tasks” (task_struct) (whereas in NT the “thread” nomenclature carries throughout). FreeBSD may have separate thread and proc internally but this is more an implementation detail. I guess this all to say at the level lower than an API like pthreads, process/thread really isn’t easily comparable between NT and most Unixes. It’s not so much “heavyweight” vs “lightweight” but that NT has been by design more limited in how you can create new virtual memory spaces. For better or worse NT tied the creation of VM spaces to this relatively expensive object to create which has made emulating Unix like behavior historically a pain in the ass.
- 2y ago
- emily-c 2y agoBefore VMS there was the family of RSX-11 operating systems which also had ASTs (now called APCs in NT parlance), IRPs, etc. Dave Cutler led the RSX-11M variant which significantly influenced VMS. The various concepts and design styles of the DEC family of operating systems that culminated in NT goes back to the 1960s. It's sad that the article didn't mention VMS or MICA since NT didn't magically appear out of the void two years after Microsoft hired the NT team. MICA was being designed for years at DEC West as part of the PRISM project.
- rbanffy 2y agoIn many ways NT was a new, ground up implementation of “VMS NT”. It started elegant, but all the backwards compatibility, technical debt, bad ideas, and dozens of versions later, with an endless list of perpetual features driven by whoever had a bigger wand at Microsoft at the time of their inception, takes a toll. Windows now is much more complicated than it could be. It shocks me some apps get Windows NT4 style buttons even on Windows 11.
- emily-c 2y ago>In many ways NT was a new, ground up implementation of “VMS NT”. Most definitely. There was a lot of design cleanup from VMS (e.g. fork processes -> DPCs, removing global PTEs and balance slots, etc), optimizations (converging VMS's parallel array structure of the PFN database into one), and simplification (NT's Io subsystem with the "bring your own thread" model, removing P1 space, and much more). SMP was also designed into NT from the beginning. You can start seeing the start of these ideas in the MICA design documents but their implementation in C instead of Pillar (variant of Pascal designed for Mica) in NT was definitely the right thing at the time.
- markus_zhang 2y agoHow do you get Windows NT4 style buttons on 11? That's something I want to do with my application!
- dspillett 2y ago
- Dwedit 2y agoThreads aren't created in milliseconds, that would be really slow. It's more like microseconds.
- nullindividual 2y agoTypo, thanks for the correction. Too late to edit :-)
- qsdf38100 2y agoWNT is VMS+1 V->W N->M S->T
- queuebert 2y agoIn true Windows form, you have a memory access error.
- nullindividual 2y ago> https://www.youtube.com/watch?v=xi1Lq79mLeE&t=4314s https://www.youtube.com/watch?v=xi1Lq79mLeE&t=4314s "New Technology", but yes, it's funny
- steve1977 2y agoInitially actually (afaik) it stood for N10, for the Intel i860 CPU. I think “New Technology” came from marketing then.
- nullindividual 2y agoIf you watch the interview with David Cutler to the time code I linked to, he explains that NT stands for New Technology which marketing did not want.
- lproven 2y agoIn which he specifically says that there may have been some point, early on, where it stood for N-10. I just watched it. He does not say what you are maintaining he says.
- JeremyNT 2y agoOne of my very favorite facts about Windows 2000, as revealed in its boot screen, is that it's based on New Technology Technology. (I no longer work with Windows very much, but this little bit of trivia has stuck with me over the years)
- pmontra 2y ago
- lr1970 2y ago> It should also be noted that while NT as a product is much newer than Unicies, it's history is rooted in VMS fundamentals thanks to it's core team of ex-Digital devs lead by David Cutler. WNT = VMS + 1 (next letter in alphabet for all three)
- amatwl 2y agoFor the record, NT comes from the codename for the Intel i860 (N10) which was the original target platform for NT.
- nullindividual 2y agoThis is incorrect per David Cutler. https://www.youtube.com/watch?v=xi1Lq79mLeE&t=4314s https://www.youtube.com/watch?v=xi1Lq79mLeE&t=4314s
- lproven 2y agoHe doesn't actually disagree, you know.
- mannyv 2y agoNT used to mean "New Technology," if I remember correctly. Not sure if that was the internal codename or a marketing creation anymore.
- deleted 2y ago[deleted]
- slt2021 2y agoProcess is a way to segregate resources (memory, sockets, file descriptors, etc). You kill a proc - it will release all memory and file descriptors. Thread is a way to segregate computation. You spawn a thread and it will run some code scheduled by the OS. you kill/stop a thread and it will stop computation, but not the resources.
- TomMasz 2y agoComing from working with Vax/VMS to NT I was struck by the similarities between them. It was only years later I found out about NT's DEC connections.