13 ms·

> The real NtQueryDirectoryFile API takes 11 parameters Curiosity got the best of me here: I had to look this up in the docs to see how a linux syscall that ta

> The real NtQueryDirectoryFile API takes 11 parameters Curiosity got the best of me here: I had to look this up in the docs to see how a linux syscall that takes 3 parameters could possibly take 11 parameters. Spoiler alert: they are used for async callbacks, filtering by name, allowing only partial results, and the ability to progressively scan with repeated calls.

deprave 10y ago
A lot of Microsoft APIs and subsystems are similarly bloated. There are probably tons of factors at play, but I believe being closed-source and having to support many individual use cases is one fundamental reason. (See for example CreateProcess vs. fork...)
- bitwize 10y ago
  When it comes to system call interfaces, it's because Dave Cutler has forgotten more than many modern "kernel hackers" will ever know about how to design an OS.
  - deprave 10y ago
    I appreciate the name-dropping. Dave Cutler's skills aside, Unix predates Windows by decades, and to anyone remotely familiar with kernel development it is clear that the sheer quantity and complexity of subsystems stem from the fact that nobody but Microsoft can actually see, modify, and redistribute Windows' source code. Unless you can actually say "here's why Windows is qualitatively better" and point out specific tasks Windows does better, I'll just point you to the fact that the internet infrastructure and most of the servers on it, along with every Apple desktop and pretty much every mobile device, run Unix.
    - Locke1689 10y ago
      NT's approach to async IO is at least somewhat empirically better as it does not require an extra context switch between receiving a "ready" event and actually performing the IO operation.
      - deprave 10y ago
        You must mean "theoretically" because "empirically" implies that you're basing your statement on observations. Where are the numbers? :)
      - gpderetta 10y ago
        Completion notification requires committing a (potentially cache-hot) buffer for an operation that might not complete until some time in the future. With readiness notification you only need to have the buffer ready when you know you will use it. Also, on an high performance poll/epoll/kevent based system you only need to poll cold fds, while you can do speculative direct read/writes to hot fds, so no need for extra syscalls in the fast case. That doesn't mean that completion notification doesn't have its advantages, especially when coupled with NT builtin auto-sizing thread pool, but it is not strictly better.
        trentnelson 10y ago
        You could do that if you really wanted on Windows; just set a 0 byte user space send and receive socket buffer. You can do dual synchronous/asynchronous socket I/O in Windows. I use this very approach with PyParallel (and 0 byte send buffers): https://github.com/pyparallel/pyparallel/blob/branches/3.3-px/Python/pyparallel.c#L9471 https://github.com/pyparallel/pyparallel/blob/branches/3.3-p... Depending on current load, that will either immediately do an asynchronous operation, or attempt synchronous non-blocking ones up to a certain number, then fall back to asynchronous. Described here: https://speakerdeck.com/trent/pyparallel-how-we-removed-the-gil-and-exploited-all-cores?slide=120 https://speakerdeck.com/trent/pyparallel-how-we-removed-the-...
    - ksk 10y ago
      To level the playing field, and if we are to take your opinion seriously, it would be beneficial to know what books or articles or whitepapers you have read to inform yourself about the design of the NT kernel.
      - deprave 10y ago
        I'm sorry, who's "we"? You replied to a comment that presented fact. The fact is that Windows use is mostly limited to desktops. As far as you're concerned, I could be entirely illiterate and my argument would still hold because it's based on fact. If you want to claim Windows is superior, please don't point us to design documentation. Show us actual numbers and use cases where Windows outperforms Unix, or is used in critical infrastructure, etc. (FYI, I've read the "internals" book for a relatively old version of Windows, along with plenty of books about attacking the Windows kernel through its huge attack surface that exists to accommodate various needs of various software vendors...)
        bitwize 10y ago
        There are more server deployments of Windows than there are of Linux. This is because Linux's server workload is mainly the Web. But every departmental office needs an Exchange server...
        ksk 10y ago
        >You replied to a comment that presented fact. False. You ASSUMED facts based on co-relation - "Its used everywhere" doesn't mean anything. "But people must have a reason to use them" STILL doesn't mean anything. "Well, so why don't they use windows" STILL doesn't get you anywhere. > I could be entirely illiterate and my argument would still hold because it's based on fact. No. You can't enter into an argument when you know nothing about the subject. That is not how it works. Why don't YOU present actual facts about the design? Show us you actually understand the internals of the kernel or have atleast some rudimentary knowledge. Otherwise you'd just be wasting everyone's time.
    - umanwizard 10y ago
      Why is it valid to conflate every kernel that runs a *nix OS together under "Unix"? Is there any meaningful overarching "Unixy" way in which, say, Xnu and Linux are similar?
    - pjmlp 10y ago
      > I'll just point you to the fact that the internet infrastructure and most of the servers on it, along with every Apple desktop and pretty much every mobile device, run Unix. I wonder how much of the internet infrastructure would run Unix if free (as in beer) clones like *BSD and GNU/Linux did not exist in first place. How much internet infrastructure would run actually Unix if ISPs had to choose between Aix, HP-UX, Solaris, Digital UX, Tru64 and Windows licenses? Free is always more valued than quality.
      - kelnos 10y ago
        I'd guess that, even more important than the cost, up until recently, interacting with Windows in a headless mode was next to useless. Most sysadmins in my experience avoid GUIs like the plague when managing servers.
      - jamespo 10y ago
        Are you too young to remember "the network is the computer"? And of course this worked in reverse, when Netscape released their commercial webserver Microsoft rushed to give away IIS.
        pjmlp 10y ago
        I started coding in the 80's and I remember how the UNIX market was afraid of Windows NT workstations, before they actually started losing market share to the free (beer) UNIX versions in form of BSD and GNU/Linux.
  - xorblurb 10y ago
    Well, that's an (unbacked) opinion, and I don't share it. NT design is not too bad (obviously especially in contrast with Consumer Windows), and especially given what was achieved on the first few releases (that was made easier by Cutler serious experience in the area), but now it is far from brillant, and it has it (huge) share of problems every serious users of both Windows and Unix based OS knows. Now at one point, way in the past, NT was far above Linux, and some Linux fanboys existed that did not even knew what they were talking about, yet had strong opinions of superiority about the kernel they used. Now we are ironically in the opposite situation: Linux has basically caught up on all the things that matters (preemptive kernel, stability, versatility, scalability) and then quickly overtook NT, yet some people like to talk endlessly about the supposed architectural superiority of NT, that did not provide anything concrete in the real world in the long term and widely used, and that MS had to work around and/or redo with an other approach (while keeping vestigial of all the old ones) to do all its modern stuff. What kernel hackers know to do, is to detect problem in architecture that look neat on paper. Brillant ones are able to anticipate. I don't even have to: history has shown were NT has been hold back by its original design.
    - deleted 10y ago
      [deleted]
  - deleted 10y ago
    [deleted]
  - trentnelson 10y ago
    You, I like you. You get it.
bitwize 10y ago
This is a recurring pattern in Windows development. Unix devs look at the Windows API and go "This syscall takes 11 parameters? GROAN." But the NT kernel is much more sophisticated and powerful than Linux, so its system calls are going to be necessarily more complicated.
- zxcvcxz 10y ago
  >the NT kernel is much more sophisticated and powerful than Linux Source? It's not sophisticated enough or powerful enough to be the most used kernel on super computers (and in the world). Windows pretty much only dominates the desktop market. Servers, super computers, mainframes, etc, mostly use Linux. A few years ago there was even a bug in Windows that caused degradation in network performance during multimedia playback that was directly connected with mechanisms employed by the Multimedia Class Scheduler Service (MMCSS), this is used on a lot of audio setups. If they can't even get audio setups right how can people consider anything Windows releases "sophisticated"? It's made to do anything you throw at it I guess, it's definitely complicated, but powerful and sophisticated aren't words I would use to describe NT.
  - recursive 10y ago
    If you're arguing in favor of linux, you probably shouldn't use any arguments that deal with getting audio setups right.
    - cbd1984 10y ago
      Audio on Linux works fine in my experience.
      - tacos 10y ago
        Hey everyone, we found him!
        madez 10y ago
        I don't want to take sides in this discussion but share an anecdote. Hey, maybe even someone knows a solution for this. I have a PC connected via on-board HDMI to a Denon AVR solely for the purpose of getting the audio to the amplifier. Windows doesn't let me use that audio interface without extending or mirroring my desktop to that HDMI port. Since there is no display connected to the AVR I don't want to extend the desktop, and mirroring heavily decreases performance of the system. On Debian Sid the computer by default allows me to use the HDMI audio without doing anything to my desktop display. It seems the system realizes that there is no display connected to the AVR but it's still a valid sink for audio.
tptacek 10y ago
Overloaded system call entrypoints are a fact of life on all mainstream platforms. Consider for instance "ioctl".
- marvy 10y ago
  I've heard that Plan 9 doesn't have ioctl. But I guess that doesn't count as mainstream.
  - wahern 10y ago
    Plan 9 replaces ioctl with special files that require writing magic incantations. It's similar to the various knobs in Linux /proc which require reading and writing specially formatted data. ioctl is simpler in that you don't need to worry as much about formatting the data (the struct declarations take care of that for you), but a file-oriented interface is nicer in that it's a higher-level abstraction--for example, it maps better to different languages, similar to how ioctl requires C or C-like shims whereas /proc can be used from any language that understands open/read/write/close, including the shell.

deprave 10y ago

A lot of Microsoft APIs and subsystems are similarly bloated. There are probably tons of factors at play, but I believe being closed-source and having to support many individual use cases is one fundamental reason. (See for example CreateProcess vs. fork...)

bitwize 10y ago
When it comes to system call interfaces, it's because Dave Cutler has forgotten more than many modern "kernel hackers" will ever know about how to design an OS.
- deprave 10y ago
  I appreciate the name-dropping. Dave Cutler's skills aside, Unix predates Windows by decades, and to anyone remotely familiar with kernel development it is clear that the sheer quantity and complexity of subsystems stem from the fact that nobody but Microsoft can actually see, modify, and redistribute Windows' source code. Unless you can actually say "here's why Windows is qualitatively better" and point out specific tasks Windows does better, I'll just point you to the fact that the internet infrastructure and most of the servers on it, along with every Apple desktop and pretty much every mobile device, run Unix.
  - Locke1689 10y ago
    NT's approach to async IO is at least somewhat empirically better as it does not require an extra context switch between receiving a "ready" event and actually performing the IO operation.
    - deprave 10y ago
      You must mean "theoretically" because "empirically" implies that you're basing your statement on observations. Where are the numbers? :)
    - gpderetta 10y ago
      Completion notification requires committing a (potentially cache-hot) buffer for an operation that might not complete until some time in the future. With readiness notification you only need to have the buffer ready when you know you will use it. Also, on an high performance poll/epoll/kevent based system you only need to poll cold fds, while you can do speculative direct read/writes to hot fds, so no need for extra syscalls in the fast case. That doesn't mean that completion notification doesn't have its advantages, especially when coupled with NT builtin auto-sizing thread pool, but it is not strictly better.
      - trentnelson 10y ago
        You could do that if you really wanted on Windows; just set a 0 byte user space send and receive socket buffer. You can do dual synchronous/asynchronous socket I/O in Windows. I use this very approach with PyParallel (and 0 byte send buffers): https://github.com/pyparallel/pyparallel/blob/branches/3.3-px/Python/pyparallel.c#L9471 https://github.com/pyparallel/pyparallel/blob/branches/3.3-p... Depending on current load, that will either immediately do an asynchronous operation, or attempt synchronous non-blocking ones up to a certain number, then fall back to asynchronous. Described here: https://speakerdeck.com/trent/pyparallel-how-we-removed-the-gil-and-exploited-all-cores?slide=120 https://speakerdeck.com/trent/pyparallel-how-we-removed-the-...
  - ksk 10y ago
    To level the playing field, and if we are to take your opinion seriously, it would be beneficial to know what books or articles or whitepapers you have read to inform yourself about the design of the NT kernel.
    - deprave 10y ago
      I'm sorry, who's "we"? You replied to a comment that presented fact. The fact is that Windows use is mostly limited to desktops. As far as you're concerned, I could be entirely illiterate and my argument would still hold because it's based on fact. If you want to claim Windows is superior, please don't point us to design documentation. Show us actual numbers and use cases where Windows outperforms Unix, or is used in critical infrastructure, etc. (FYI, I've read the "internals" book for a relatively old version of Windows, along with plenty of books about attacking the Windows kernel through its huge attack surface that exists to accommodate various needs of various software vendors...)
      - bitwize 10y ago
        There are more server deployments of Windows than there are of Linux. This is because Linux's server workload is mainly the Web. But every departmental office needs an Exchange server...
      - ksk 10y ago
        >You replied to a comment that presented fact. False. You ASSUMED facts based on co-relation - "Its used everywhere" doesn't mean anything. "But people must have a reason to use them" STILL doesn't mean anything. "Well, so why don't they use windows" STILL doesn't get you anywhere. > I could be entirely illiterate and my argument would still hold because it's based on fact. No. You can't enter into an argument when you know nothing about the subject. That is not how it works. Why don't YOU present actual facts about the design? Show us you actually understand the internals of the kernel or have atleast some rudimentary knowledge. Otherwise you'd just be wasting everyone's time.
  - umanwizard 10y ago
    Why is it valid to conflate every kernel that runs a *nix OS together under "Unix"? Is there any meaningful overarching "Unixy" way in which, say, Xnu and Linux are similar?
  - pjmlp 10y ago
    > I'll just point you to the fact that the internet infrastructure and most of the servers on it, along with every Apple desktop and pretty much every mobile device, run Unix. I wonder how much of the internet infrastructure would run Unix if free (as in beer) clones like *BSD and GNU/Linux did not exist in first place. How much internet infrastructure would run actually Unix if ISPs had to choose between Aix, HP-UX, Solaris, Digital UX, Tru64 and Windows licenses? Free is always more valued than quality.
    - kelnos 10y ago
      I'd guess that, even more important than the cost, up until recently, interacting with Windows in a headless mode was next to useless. Most sysadmins in my experience avoid GUIs like the plague when managing servers.
    - jamespo 10y ago
      Are you too young to remember "the network is the computer"? And of course this worked in reverse, when Netscape released their commercial webserver Microsoft rushed to give away IIS.
      - pjmlp 10y ago
        I started coding in the 80's and I remember how the UNIX market was afraid of Windows NT workstations, before they actually started losing market share to the free (beer) UNIX versions in form of BSD and GNU/Linux.
- xorblurb 10y ago
  Well, that's an (unbacked) opinion, and I don't share it. NT design is not too bad (obviously especially in contrast with Consumer Windows), and especially given what was achieved on the first few releases (that was made easier by Cutler serious experience in the area), but now it is far from brillant, and it has it (huge) share of problems every serious users of both Windows and Unix based OS knows. Now at one point, way in the past, NT was far above Linux, and some Linux fanboys existed that did not even knew what they were talking about, yet had strong opinions of superiority about the kernel they used. Now we are ironically in the opposite situation: Linux has basically caught up on all the things that matters (preemptive kernel, stability, versatility, scalability) and then quickly overtook NT, yet some people like to talk endlessly about the supposed architectural superiority of NT, that did not provide anything concrete in the real world in the long term and widely used, and that MS had to work around and/or redo with an other approach (while keeping vestigial of all the old ones) to do all its modern stuff. What kernel hackers know to do, is to detect problem in architecture that look neat on paper. Brillant ones are able to anticipate. I don't even have to: history has shown were NT has been hold back by its original design.
  - deleted 10y ago
    [deleted]
- deleted 10y ago
  [deleted]
- trentnelson 10y ago
  You, I like you. You get it.

bitwize 10y ago

This is a recurring pattern in Windows development. Unix devs look at the Windows API and go "This syscall takes 11 parameters? GROAN." But the NT kernel is much more sophisticated and powerful than Linux, so its system calls are going to be necessarily more complicated.

zxcvcxz 10y ago
>the NT kernel is much more sophisticated and powerful than Linux Source? It's not sophisticated enough or powerful enough to be the most used kernel on super computers (and in the world). Windows pretty much only dominates the desktop market. Servers, super computers, mainframes, etc, mostly use Linux. A few years ago there was even a bug in Windows that caused degradation in network performance during multimedia playback that was directly connected with mechanisms employed by the Multimedia Class Scheduler Service (MMCSS), this is used on a lot of audio setups. If they can't even get audio setups right how can people consider anything Windows releases "sophisticated"? It's made to do anything you throw at it I guess, it's definitely complicated, but powerful and sophisticated aren't words I would use to describe NT.
- recursive 10y ago
  If you're arguing in favor of linux, you probably shouldn't use any arguments that deal with getting audio setups right.
  - cbd1984 10y ago
    Audio on Linux works fine in my experience.
    - tacos 10y ago
      Hey everyone, we found him!
      - madez 10y ago
        I don't want to take sides in this discussion but share an anecdote. Hey, maybe even someone knows a solution for this. I have a PC connected via on-board HDMI to a Denon AVR solely for the purpose of getting the audio to the amplifier. Windows doesn't let me use that audio interface without extending or mirroring my desktop to that HDMI port. Since there is no display connected to the AVR I don't want to extend the desktop, and mirroring heavily decreases performance of the system. On Debian Sid the computer by default allows me to use the HDMI audio without doing anything to my desktop display. It seems the system realizes that there is no display connected to the AVR but it's still a valid sink for audio.

tptacek 10y ago

Overloaded system call entrypoints are a fact of life on all mainstream platforms. Consider for instance "ioctl".

marvy 10y ago
I've heard that Plan 9 doesn't have ioctl. But I guess that doesn't count as mainstream.
- wahern 10y ago
  Plan 9 replaces ioctl with special files that require writing magic incantations. It's similar to the various knobs in Linux /proc which require reading and writing specially formatted data. ioctl is simpler in that you don't need to worry as much about formatting the data (the struct declarations take care of that for you), but a file-oriented interface is nicer in that it's a higher-level abstraction--for example, it maps better to different languages, similar to how ioctl requires C or C-like shims whereas /proc can be used from any language that understands open/read/write/close, including the shell.