5 ms·

You're sitting on the point of view of a software developer. Microsoft's point of view is that the underlying software doesn't matter. The user's software _has

You're sitting on the point of view of a software developer. Microsoft's point of view is that the underlying software doesn't matter. The user's software _has_ to run. The Application Compatibility Database (https://learn.microsoft.com/en-us/windows/win32/devnotes/application-compatibility-database https://learn.microsoft.com/en-us/windows/win32/devnotes/app...) is, overall, a relatively small component, and all it does is apply some shims if your executable is in that list. Performance issues in Windows do not stem from anywhere near the kernel. The kernel team is absolutely top tier. The kernel itself is of much higher quality than what you'd find on Linux, or MacOS. Now, the upper layers however...

rbanffy 3y ago
> The user's software _has_ to run. My concern is that a lot of security issues may have come from this. A clever attacker could grab recently freed memory from one of these programs and inject malicious code to enjoy whatever other weird privileges the original program has, because marketing said it can’t crash.
- ohgodplsno 3y ago
  The compatibility hacks are not "revert to old, terribly unsafe behavior". Rather, "use current, good behavior, lie to the application about what happened."
- saagarjha 3y ago
  This actually improves security by preventing UAF.
everial 3y ago
> The kernel team is absolutely top tier. The kernel itself is of much higher quality than what you'd find on Linux, or MacOS. Have recommended sources for this or learning more? My experience with Windows doesn't match this at all, though from my perspective it's hard to tell if it's kernel as opposed to any of the layers above it.
- PaulHoule 3y ago
  Notably Microsoft does not document an API to the kernel, the official userspace interface in Windows is the DLLs they provide. In that since, Wine, which provides DLLs that implement the same interface over the Linux kernel is conforming to the way windows works, and Cygwin, which provided a POSIX-like libc that ran on the Windows kernel is a Windows-centric way to implement POSIX. (That said, the path of WSL 1, which emulated the Linux syscall interface on Windows, takes advantage of the idea Windows NT had from the very beginning which was implementing "personalities" that could pretend to be other OS, such as the original plan for Windows NT to be OS/2 compatible.)
- ohgodplsno 3y ago
  There's an absurd number of components to the Windows Kernel, so here's a kind of disjointed list of various things, from different time frames. Windows Research Kernel - https://github.com/HighSchoolSoftwareClub/Windows-Research-Kernel-WRK-/tree/master https://github.com/HighSchoolSoftwareClub/Windows-Research-K... - More or less Windows XP I/O Completion ports - https://learn.microsoft.com/en-us/windows/win32/fileio/i-o-completion-ports https://learn.microsoft.com/en-us/windows/win32/fileio/i-o-c... - io_uring, but mostly better, since NT 3.5 General architecture info: https://en.wikipedia.org/wiki/Architecture_of_Windows_NT https://en.wikipedia.org/wiki/Architecture_of_Windows_NT A bunch of things you'll find in Windows Internals, which is pretty much the bible for Windows (https://empyreal96.github.io/nt-info-depot/Windows-Internals-PDFs/Windows%20System%20Internals%207e%20Part%201.pdf https://empyreal96.github.io/nt-info-depot/Windows-Internals..., or buy it online. Mark Russinovich is a treasure trove of Windows knowledge) The various Windows subsystems - Windows is built from the start to be able to impersonate other OSes. While the most obvious one is WSL (despite WSL2 being just a VM), there's an OS/2 Subsystem, a POSIX Subsystem, a Win32 subsystem... Very few things actually run in kernel mode. There exists a almost-kernel-but-not-quite mode called executive mode, which is a much better option than Linux's all or nothing user-or-kernel (and, as far as I know, Mach has the same problem) NT is a hybrid kernel: not quite monolithic, not quite micro. This allows Windows to do things like swapping your video drivers live as it's running, unlike Linux and Mach which would miserably crash. Hell, it can even recover from a video driver crash and restart it safely, and all you'll see is a few seconds of black screen. The breadth of devices it supports is absolutely breathtaking. (well, in part because they very much have a hand in forcing manufacturers to respect the standards that they write) All of Sysinternals (Mark Russinovich's work, again) is also an amazing resource: https://learn.microsoft.com/en-us/sysinternals/ https://learn.microsoft.com/en-us/sysinternals/ Now, mind you, this is purely about technical merits: the NT Kernel is a miracle of technology. The APIs it exposes, and that most Microsoft products expose are downright batshit insane sometimes. But that's also what happens when you support 35 years of software. Also, the HANDLE pattern that most Win32 API uses is the superior alternative to dumb pointers (https://floooh.github.io/2018/06/17/handles-vs-pointers.html https://floooh.github.io/2018/06/17/handles-vs-pointers.html) Oh and a bunch of The Old New Things articles, but I can't be arsed to look them up right now, sorry.
lamontcg 3y ago
> The kernel team is absolutely top tier. The kernel itself is of much higher quality than what you'd find on Linux, or MacOS. > > Now, the upper layers however... You're not selling me on the idea that the compatibility layer has no cost by pointing out that the upper layers that reside over it are a mess. That would actually be my argument.
- ohgodplsno 3y ago
  Would you consider LDPRELOAD a massive cost to Linux ? Because that's basically what it is. You could give me the best kernel in the world, if I end up reading a file whenever I push a pixel to the screen, my performance will be dogshit. Windows's performance problems are not due to the kernel (or rather, not due to problems/bugs: some performance issues are just a choice. See NTFS's dreadful performance with Git: NTFS simply wasn't thought out for having thousands of very small files all being touched at the same time.)
User23 3y ago
The NT kernel interfaces are lovely. I’m not at all a windows fanboy and haven’t done windows dev in well over a decade, but when I did the APIs were lightyears ahead of the competition. And Jeffrey Richter’s books were a marvelous resource on the documentation side.
renonce 3y ago
> Performance issues in Windows do not stem from anywhere near the kernel. I remember Windows uses a O(N^2) scheduler so the system slows down when it has a few thousand processes. Would that count as a performance issue in the kernel?
- ohgodplsno 3y ago
  As far as I know, Windows uses a multilevel feedback queue, so O(n²) would be surprising. The one issue that pops up with the Windows scheduler is when you have plenty of processes doing tiny little bits of IO instead of one huge slab of it. Could you count it as a performance issue in the kernel ? Maybe. But really, you're mostly hitting an issue in what it's built to do. Windows really likes two things: * don't spawn subprocesses * please do your I/O in one huge batch and thank you. The average Windows machine will barely have 100 processes running. I have 184 right now, and I run a lot of crap. This goes directly contrary to the philosophy of many UNIX programs: create a shitload of processes, fork everywhere, and write tiny files all the time. I wouldn't complain about a hammer not working well if I'm using the handle to nail things. Sure, it would be nice if it also worked there, but it's not exactly made for that either, and for good reason. POSIX fork() being such a can of worms is half the reason that WSL1 was abandoned. Windows does not have the internal machinery to do that, because it wasn't built for that.
xvilka 3y ago
Fork and IO performance are major problems when porting *nix software to Windows. It's visibly slower, one of the reasons WSL1 was way too slow to be used for many things while technically being a clever solution.
- CoolCold 3y ago
  to my understanding it's not really IO (reading/writing bytes), but overhead on OpenFile/CloseFile being under AV/Defender inspection. I'm taking that understanding basing on https://www.youtube.com/watch?v=qbKGw8MQ0i8 https://www.youtube.com/watch?v=qbKGw8MQ0i8 with " "NTFS really isn't that bad" - Robert Collins (LCA 2020) " > Why was rustup slow (3m30s to install (not including download time)) in early 2019 on Windows, and why isn't it slow (14s to install) now? >Early in 2019 I was developing some things in Rustlang on Windows, got frustrated at the performance of rustup (the installer for rust) and looked for bug. I found one which blamed a combination of NTFS, small files and anti-virus software for the problem. This wasn't an entirely satisfactory answer to my mind. Fast forward a bit, and with a combination of changes to rustup and to Windows itself and rustup is now pleasantly usable.... which also improved performance for rustup on NFS mounted home directories.

rbanffy 3y ago

> The user's software _has_ to run. My concern is that a lot of security issues may have come from this. A clever attacker could grab recently freed memory from one of these programs and inject malicious code to enjoy whatever other weird privileges the original program has, because marketing said it can’t crash.

ohgodplsno 3y ago
The compatibility hacks are not "revert to old, terribly unsafe behavior". Rather, "use current, good behavior, lie to the application about what happened."
saagarjha 3y ago
This actually improves security by preventing UAF.

everial 3y ago

> The kernel team is absolutely top tier. The kernel itself is of much higher quality than what you'd find on Linux, or MacOS. Have recommended sources for this or learning more? My experience with Windows doesn't match this at all, though from my perspective it's hard to tell if it's kernel as opposed to any of the layers above it.

PaulHoule 3y ago
Notably Microsoft does not document an API to the kernel, the official userspace interface in Windows is the DLLs they provide. In that since, Wine, which provides DLLs that implement the same interface over the Linux kernel is conforming to the way windows works, and Cygwin, which provided a POSIX-like libc that ran on the Windows kernel is a Windows-centric way to implement POSIX. (That said, the path of WSL 1, which emulated the Linux syscall interface on Windows, takes advantage of the idea Windows NT had from the very beginning which was implementing "personalities" that could pretend to be other OS, such as the original plan for Windows NT to be OS/2 compatible.)
ohgodplsno 3y ago
There's an absurd number of components to the Windows Kernel, so here's a kind of disjointed list of various things, from different time frames. Windows Research Kernel - https://github.com/HighSchoolSoftwareClub/Windows-Research-Kernel-WRK-/tree/master https://github.com/HighSchoolSoftwareClub/Windows-Research-K... - More or less Windows XP I/O Completion ports - https://learn.microsoft.com/en-us/windows/win32/fileio/i-o-completion-ports https://learn.microsoft.com/en-us/windows/win32/fileio/i-o-c... - io_uring, but mostly better, since NT 3.5 General architecture info: https://en.wikipedia.org/wiki/Architecture_of_Windows_NT https://en.wikipedia.org/wiki/Architecture_of_Windows_NT A bunch of things you'll find in Windows Internals, which is pretty much the bible for Windows (https://empyreal96.github.io/nt-info-depot/Windows-Internals-PDFs/Windows%20System%20Internals%207e%20Part%201.pdf https://empyreal96.github.io/nt-info-depot/Windows-Internals..., or buy it online. Mark Russinovich is a treasure trove of Windows knowledge) The various Windows subsystems - Windows is built from the start to be able to impersonate other OSes. While the most obvious one is WSL (despite WSL2 being just a VM), there's an OS/2 Subsystem, a POSIX Subsystem, a Win32 subsystem... Very few things actually run in kernel mode. There exists a almost-kernel-but-not-quite mode called executive mode, which is a much better option than Linux's all or nothing user-or-kernel (and, as far as I know, Mach has the same problem) NT is a hybrid kernel: not quite monolithic, not quite micro. This allows Windows to do things like swapping your video drivers live as it's running, unlike Linux and Mach which would miserably crash. Hell, it can even recover from a video driver crash and restart it safely, and all you'll see is a few seconds of black screen. The breadth of devices it supports is absolutely breathtaking. (well, in part because they very much have a hand in forcing manufacturers to respect the standards that they write) All of Sysinternals (Mark Russinovich's work, again) is also an amazing resource: https://learn.microsoft.com/en-us/sysinternals/ https://learn.microsoft.com/en-us/sysinternals/ Now, mind you, this is purely about technical merits: the NT Kernel is a miracle of technology. The APIs it exposes, and that most Microsoft products expose are downright batshit insane sometimes. But that's also what happens when you support 35 years of software. Also, the HANDLE pattern that most Win32 API uses is the superior alternative to dumb pointers (https://floooh.github.io/2018/06/17/handles-vs-pointers.html https://floooh.github.io/2018/06/17/handles-vs-pointers.html) Oh and a bunch of The Old New Things articles, but I can't be arsed to look them up right now, sorry.

lamontcg 3y ago

> The kernel team is absolutely top tier. The kernel itself is of much higher quality than what you'd find on Linux, or MacOS. > > Now, the upper layers however... You're not selling me on the idea that the compatibility layer has no cost by pointing out that the upper layers that reside over it are a mess. That would actually be my argument.

ohgodplsno 3y ago
Would you consider LDPRELOAD a massive cost to Linux ? Because that's basically what it is. You could give me the best kernel in the world, if I end up reading a file whenever I push a pixel to the screen, my performance will be dogshit. Windows's performance problems are not due to the kernel (or rather, not due to problems/bugs: some performance issues are just a choice. See NTFS's dreadful performance with Git: NTFS simply wasn't thought out for having thousands of very small files all being touched at the same time.)

User23 3y ago

The NT kernel interfaces are lovely. I’m not at all a windows fanboy and haven’t done windows dev in well over a decade, but when I did the APIs were lightyears ahead of the competition. And Jeffrey Richter’s books were a marvelous resource on the documentation side.

renonce 3y ago

> Performance issues in Windows do not stem from anywhere near the kernel. I remember Windows uses a O(N^2) scheduler so the system slows down when it has a few thousand processes. Would that count as a performance issue in the kernel?

ohgodplsno 3y ago
As far as I know, Windows uses a multilevel feedback queue, so O(n²) would be surprising. The one issue that pops up with the Windows scheduler is when you have plenty of processes doing tiny little bits of IO instead of one huge slab of it. Could you count it as a performance issue in the kernel ? Maybe. But really, you're mostly hitting an issue in what it's built to do. Windows really likes two things: * don't spawn subprocesses * please do your I/O in one huge batch and thank you. The average Windows machine will barely have 100 processes running. I have 184 right now, and I run a lot of crap. This goes directly contrary to the philosophy of many UNIX programs: create a shitload of processes, fork everywhere, and write tiny files all the time. I wouldn't complain about a hammer not working well if I'm using the handle to nail things. Sure, it would be nice if it also worked there, but it's not exactly made for that either, and for good reason. POSIX fork() being such a can of worms is half the reason that WSL1 was abandoned. Windows does not have the internal machinery to do that, because it wasn't built for that.

xvilka 3y ago

Fork and IO performance are major problems when porting *nix software to Windows. It's visibly slower, one of the reasons WSL1 was way too slow to be used for many things while technically being a clever solution.

CoolCold 3y ago
to my understanding it's not really IO (reading/writing bytes), but overhead on OpenFile/CloseFile being under AV/Defender inspection. I'm taking that understanding basing on https://www.youtube.com/watch?v=qbKGw8MQ0i8 https://www.youtube.com/watch?v=qbKGw8MQ0i8 with " "NTFS really isn't that bad" - Robert Collins (LCA 2020) " > Why was rustup slow (3m30s to install (not including download time)) in early 2019 on Windows, and why isn't it slow (14s to install) now? >Early in 2019 I was developing some things in Rustlang on Windows, got frustrated at the performance of rustup (the installer for rust) and looked for bug. I found one which blamed a combination of NTFS, small files and anti-virus software for the problem. This wasn't an entirely satisfactory answer to my mind. Fast forward a bit, and with a combination of changes to rustup and to Windows itself and rustup is now pleasantly usable.... which also improved performance for rustup on NFS mounted home directories.