11 ms·
This is a recurring pattern in Windows development. Unix devs look at the Windows API and go "This syscall takes 11 parameters? GROAN." But the NT kernel is muc
by bitwize 10y ago
This is a recurring pattern in Windows development. Unix devs look at the Windows API and go "This syscall takes 11 parameters? GROAN." But the NT kernel is much more sophisticated and powerful than Linux, so its system calls are going to be necessarily more complicated.
- zxcvcxz 10y ago>the NT kernel is much more sophisticated and powerful than Linux Source? It's not sophisticated enough or powerful enough to be the most used kernel on super computers (and in the world). Windows pretty much only dominates the desktop market. Servers, super computers, mainframes, etc, mostly use Linux. A few years ago there was even a bug in Windows that caused degradation in network performance during multimedia playback that was directly connected with mechanisms employed by the Multimedia Class Scheduler Service (MMCSS), this is used on a lot of audio setups. If they can't even get audio setups right how can people consider anything Windows releases "sophisticated"? It's made to do anything you throw at it I guess, it's definitely complicated, but powerful and sophisticated aren't words I would use to describe NT.
- recursive 10y agoIf you're arguing in favor of linux, you probably shouldn't use any arguments that deal with getting audio setups right.
- cbd1984 10y agoAudio on Linux works fine in my experience.
- tacos 10y agoHey everyone, we found him!
- madez 10y agoI don't want to take sides in this discussion but share an anecdote. Hey, maybe even someone knows a solution for this. I have a PC connected via on-board HDMI to a Denon AVR solely for the purpose of getting the audio to the amplifier. Windows doesn't let me use that audio interface without extending or mirroring my desktop to that HDMI port. Since there is no display connected to the AVR I don't want to extend the desktop, and mirroring heavily decreases performance of the system. On Debian Sid the computer by default allows me to use the HDMI audio without doing anything to my desktop display. It seems the system realizes that there is no display connected to the AVR but it's still a valid sink for audio.
- tacos 10y agoCan you use optical instead of HDMI?
- madez 10y agoTOSLINK doesn't support uncompressed surround audio, while HDMI does, and I do use it.
- j_s 10y agoIf I remember correctly, NVIDIA did a decent job with their on-board HDMI driver audio-wise; what brand is yours? I'm 100% the functionality is dependent on the driver rather than the OS.
- zxcvcxz 10y agoWorks very well for me too. Don't know why you got downvoted. It's like it's 1994 in here...
- orbifold 10y agoWell for various definitions of fine I guess.
- cbd1984 10y agoIt works fine as in "I can listen to audio on my laptop from multiple programs at once, with a centralized way to control audio volume on a per-application or per-sound-device basis." I literally cannot imagine any audio system doing better than that given the hardware I have to work with.
- xodjoshd 10y agoBut this is exactly what made me switch, windows was preventing me from accessing my sound card directly in order to record a remote interview. I use Linux regularly to record and edit audio, it's free , it works, and I dont have to worry my OS is active reducing the functionality of my equipment.
- bitwize 10y agoIndeed. I would go so far as to say that a large part of why audio is such a CF under Linux is -- wait for it -- lack of real asynchronous I/O. Audio is asynchronous by nature, and to do that right under Linux you need a "sound server" with all the additional overhead, jank, and shuffling of data between kernel and at least two different user spaces that implies. Audio under Linux was best with OSS, which was synchronous in nature and not suitable for professional applications. JACK mitigated that somewhat, but for an OS to do audio right you need a kernel-level async audio API like Core Audio or whatever Windows is doing these days.
- makomk 10y agoWindows has a sound server too, you know. I believe Core Audio on Mac does too. A large part of why audio is such a CF under Linux is that PulseAudio is incredibly badly written and poorly maintained. My favourite was the half-finished micro-optimization that broke some of the resamplers because the author got bored and never modified them to handle the change, which somehow made it all the way into a release. I dread to think what they'd do with powerful tools like asynchronous I/O.
- Sanddancer 10y agoThey got audio setups right. The reason the network degradation happened is that video and audio playback were given realtime priority so background processes couldn't cause pops, stutters, etc. At the time Vista was released, most home users didn't have a gigabit network, so the performance degradation would only happen on a small number of users, and most would rather prefer good audio and video performance to a slowdown in network performance in a small percentage of users. With today's massively multicore systems, it's even less of an issue, while linux still has a problem with latency on applications like pro audio.
- makomk 10y agoThe reason the network degradation happened is that Microsoft couldn't figure out how to stop heavy network activity causing audio glitches on some systems even after giving audio realtime priority, so they hacked around it by adding a fixed 10,000 packets-per-second cap on network activity regardless of system speed or CPU usage (less if you had multiple network adapters). See https://blogs.technet.microsoft.com/markrussinovich/2007/08/26/vista-multimedia-playback-and-network-throughput/ https://blogs.technet.microsoft.com/markrussinovich/2007/08/... This was just as much of an issue on multicore systems because the cap was unaffected by the system speed and chosen based on a slow single-core machine.
- pbarnes_1 10y agoI don't know why you're getting downvoted since the parent is basically stating random opinions about "power" and "sophistication" without anything to actually back it up. 11 param functions don't say "power" to me. They say "poorly thought out API design". Much can be said for most Windows APIs in general.
- darkengine 10y agoIt may be more "sophisticated" (sounds like a more positive synonym of "complex" to me), but I certainly don't think it's more powerful.
- ckaygusu 10y agoI think the problem here is not a syscall taking 11 parameters, it's a syscall that merely lists what is inside a directory taking 11 parameters. ataylor_284 explained the reasons (how convincingly, I'd argue) but on the first sight that surely smells bloat. I'd also object NT kernel being more "powerful". Sure unixy kernels and NT has their differences but I don't think either one is superior.
- rbanffy 10y agoI remember the struct I had to populate to start a new process in 1997 or 1998...
- bigger_cheese 10y ago11 parameters may seem bloated but in some cases Unix syscalls weren't designed with enough parameters whcih caused a bunch of pain necessitating things like dup->dup2->dup3 pipe->piep2 rename->renameat->renameat2 Best practice nowadays in linux is to allow overloading syscalls via a flags parameter. see https://lwn.net/Articles/585415/ https://lwn.net/Articles/585415/ So modern linux syscalls may be bloated too.
- deprave 10y agoSince when is kernel complexity a measure of quality...? :)
- jasonm23 10y agohmmm... Usage of the adjective "sophisticated" always precedes an outpouring either ignorance or straight bs.
- tremon 10y agoBut the NT kernel is much more sophisticated and powerful than Linux That does not follow from the example. All it shows is that Microsoft prefers to put a lot of functionality in one interface, while Linux probably prefers low-level functions to be as small as possible, and probably offers things like filtering on a higher level (in glibc, for example). Neither explanation has anything to do with sophistication. I personally believe that small interfaces are a better design.
- bitwize 10y agoActually it does, as it mentioned that the extra parameters are for things like async callbacks and partial results. The I/O model that Windows supports is a strict superset of the Unix I/O model. Windows supports true async I/O, allowing process to start I/O operations and wait on an object like an I/O completion port for them to complete. Multiple threads can share a completion port, allowing for useful allocation of thread pools instead of thread-per-request. In Unix all I/O is synchronous; asynchronicity must be faked by setting O_NONBLOCK and buzzing in a select loop, interleaving bits of I/O with other processing. It adds complexity to code to simulate what Windows gives you for real, for free. And sometimes it breaks down; if I/O is hung on a device the kernel considers "fast" like a disk, that process is hosed until the operation completes or errors out.
- SwellJoe 10y agoI have never worked with systems-level Windows programming, so I don't know the answer to this...but, how is what you're describing better than epoll or aio in Linux or kqueues on the BSDs? I'm guessing you're coming from the opposite position of ignorance I am (i.e. you've worked on Windows, but not Linux or other modern UNIX), though, since "setting O_NONBLOCK and buzzing in a select loop, interleaving bits of I/O with other processing" doesn't describe anything developed in many, many years. 15 years ago select was already considered ancient.
- trentnelson 10y agoTo quote myself: > The “Why Windows?” (or “Why not Linux?”) question is one I get asked the most, but it’s also the one I find hardest to answer succinctly without eventually delving into really low-level kernel implementation details. > > You could port PyParallel to Linux or OS X -- there are two parts to the work I’ve done: a) the changes to the CPython interpreter to facilitate simultaneous multithreading (platform agnostic), and b) the pairing of those changes with Windows kernel primitives that provide completion-oriented thread-agnostic high performance I/O. That part is obviously very tied to Windows currently. > > So if you were to port it to POSIX, you’d need to implement all the scaffolding Windows gives you at the kernel level in user space. (OS X Grand Central Dispatch was definitely a step in the right direction.) So you’d have to manage your threadpools yourself, and each thread would have to have its own epoll/kqueue event loop. The problem with adding a file descriptor to a per-thread event loop’s epoll/kqueue set is that it’s just not optimal if you want to continually ensure you’re saturating your hardware (either CPU cores or I/O). You need to be able to disassociate the work from the worker. The work is the invocation of the data_received() callback, the worker is whatever thread is available at the time the data is received. As soon as you’ve bound a file descriptor to a per-thread set, you prevent thread migration > > Then there’s the whole blocking file I/O issue on UNIX. As soon as you issue a blocking file I/O call on one of those threads, you have one thread less doing useful work, which means you’re increasing the time before any other file descriptors associated with that thread’s multiplex set can be served, which adversely affects latency. And if you’re using the threads == ncpu pattern, you’re going to have idle CPU cycles because, say, only 6 out of your 8 threads are in a runnable state. So, what’s the answer? Create 16 threads? 32? The problem with that is you’re going to end up over-scheduling threads to available cores, which results in context switching, which is less optimal than having one (and only one) runnable thread per core. I spend some time discussing that in detail here: https://speakerdeck.com/trent/parallelism-and-concurrency-with-python?slide=24 https://speakerdeck.com/trent/parallelism-and-concurrency-wi.... (The best example of how that manifests as an issue in real life is `make –jN world` -- where N is some magic number derived from experimentation, usually around ncpu X 2. Too low, you’ll have idle CPUs at some point, too high and the CPU is spending time doing work that isn’t directly useful. There’s no way to say `make –j[just-do-whatever-you-need-to-do-to-either-saturate-my-I/O-channels-or-CPU-cores-or-both]`.) > > Alternatively, you’d have to rely on AIO on POSIX for all of your file I/O. I mean, that’s basically how Oracle does it on UNIX – shared memory, lots of forked processes, and “AIO” direct-write threads (bypassing the filesystem cache – the complexities of which have thwarted previous attempts on Linux to implement non-blocking file I/O). But we’re talking about a highly concurrent network server here… so you’d have to implement userspace glue to synchronize the dispatching of asynchronous file I/O and the per-thread non-blocking socket epoll/kqueue event loops… just… ugh. Sure, it’s all possible, but imagine the complexity and portability issues, and how much testing infrastructure you’d need to have. It makes sense for Oracle, but it’s not feasible for a single open source project. The biggest issue in my mind is that the whole thing just feels like forcing a square peg through a round hole… the UNIX readiness file descriptor I/O model just isn’t well suited to this sort of problem if you want to optimally exploit your underlying hardware. > > Now, with Windows, it’s a completely different situation. The whole kernel is architected around the notion of I/O completion and waitable events, not “file descriptor readiness”. This seems subtle but it pervades every single aspect of the system. The cache manager is tightly linked to the memory management and I/O manager – once you factor in asynchronous I/O this becomes incredibly important because of the way you need to handle memory locking for the duration of the I/O request and the conditions for synchronously serving data from the cache manager versus reading it from disk. The waitable events aspect is important too – there’s not really an analog on UNIX. Then there’s the notion of APCs instead of signals which again, are fundamentally different paradigms. The digger you deep the more you appreciate the complexity of what Windows is doing under the hood. > > What was fantastic about Vista+ is that they tied all of these excellent primitives together via the new threadpool APIs, such that you don’t need to worry about creating your own threads at any point. You just submit things to the threadpool – waitable events, I/O or timers – and provide a C callback that you want to be called when the thing has completed, and Windows takes care of everything else. I don’t need to continually check epoll/kqueue sets for file descriptor readiness, I don’t need to have signal handlers to intercept AIO or timers, I don’t need to offload I/O to specific I/O threads… it’s all taken care of, and done in such a way that will efficiently use your underlying hardware (cores and I/O bandwidth), thanks to the thread-agnosticism of Windows I/O model (which separates the work from the worker). > > Is there something simple that could be added to Linux to get a quick win? Or would it require architecting the entire kernel? Is there an element of convergent evolution, where the right solution to this problem is the NT/VMS architecture, or is there some other way of solving it? I’m too far down the Windows path now to answer that without bias. The next 10 years are going to be interesting, though. https://groups.google.com/forum/#!topic/framework-benchmarks/vtTvwpTeznI https://groups.google.com/forum/#!topic/framework-benchmarks...
- trentnelson 10y agoCuriosity got the better of me recently when I re-read Russinovich's [NT and VMS - The Rest Of The Story](http://windowsitpro.com/windows-client/windows-nt-and-vms-rest-story http://windowsitpro.com/windows-client/windows-nt-and-vms-re...), and I bought a copy of [VMS Internals and Data Structures](http://www.amazon.com/VAX-VMS-Internals-Data-Structures/dp/1555580599 http://www.amazon.com/VAX-VMS-Internals-Data-Structures/dp/1...). Side-by-side, comparing VMS to UNIX, and VMS's approach to a few key areas like I/O, ASTs and tiered interrupt levels are simply just more sophisticated. NT inherited all of that. It was fundamentally superior, as a kernel, to UNIX, from day 1. I haven't met a single person that has understood NT and Linux/UNIX, and still thinks UNIX is superior as far as the kernels go. I have definitely alienated myself the more I've discovered that though, as it's such a wildly unpopular sentiment in open source land. Cutler got a call from Gates in 89, and from 89-93, NT was built. He was 47 at the time, and was one of the lead developers of VMS, which was a rock-solid operating system. In 93, Linus was 22, and starting "implementing enough syscalls until bash ran" as a fun project to work on. Cutler despised the UNIX I/O model. "Getta byte getta byte getta byte byte byte." The I/O request packet approach to I/O (and tiered interrupts) is one of the key reasons behind NT's superiority. And once you've grok'd things like APCs and structured exception handling, signals just seem absolutely ghastly in comparison.
- jen20 10y agoI've never met a single person who understood what they were talking about and referred to a "UNIX kernels". It may be true that Linux was once less advanced than NT - this is no longer the case, despite egregious design flaws in things like epoll. It has simply never been true (for example) for the Illumos (nee Solaris) kernel.
- trentnelson 10y agoI qualified it as "Linux/UNIX kernel" because I wanted to emphasize the kernel and not userspace. Solaris event ports are good, but they're still ultimately backed by a readiness-oriented I/O model, and can't be used for asynchronous file I/O.
- uudecode 10y ago"... so its system calls are going to be necessarily more complicated." Are you implying that an increase in "power" can never be achieved through increasing simplicity?
- bitwize 10y agoThat's the thing. Just by glancing at the API docs, Windows looks more complicated but where the rubber meets the road in terms of real high-performance application development, Windows is way simpler. In Windows you can do in one syscall what would take several in Linux. You can schedule I/O calls across multiple threads in a completely thread-safe manner without having to manage the synchronization yourself -- and since threads go to sleep entirely while waiting for I/O operations to complete, there is no chewing up CPU cycles in a select/epoll loop. So yes, writing "hello world" or simple filters is simpler in Unix -- but writing multithreaded server applications that maximize throughput is simpler in Windows. Unix is bristling with features designed to "allow you to save me some time". It was designed to make it easy to write quick, "one-off" programs in C. VMS -- the predecessor to Windows NT -- was designed to run long-lasting, high-performance, high-reliability business applications for real users with money on the line (i.e., not just hackers) and Windows NT inherits this legacy.
- uudecode 10y ago"It was designed to make it easy to write quick, "one-off" programs in C." OK, so it just so happens this is what I love to do. I like writing small programs and continually trying to improve them. So I guess I should be a UNIX user? Is NT not good for this too? BTW, I do like VMS. But despite the NT kernel, using NT feels nothing like using VMS.
- trentnelson 10y agoI love writing NT-style C-level (no CRT, just pure C and whatever the CNF/Cutler Normal Form style is).
- pjmlp 10y agoAlso UNIX devs seem to forget how cumbersome the X11, Xlib and Motif APIs are.
- pbarnes_1 10y agoThis was maybe the case at Linux 2.0, but is not the case now. Also, Windows development is infinitely more painful than Unix/Linux.