4 ms·

I agree in principle. NT IO is way more elegant than Linux IO, especially when it comes to AIO, and especially AIO to disk files. I don't agree with your examp

by rossy 8y ago

I agree in principle. NT IO is way more elegant than Linux IO, especially when it comes to AIO, and especially AIO to disk files. I don't agree with your examples though. I think NT has those flaws as well. Accidental handle reuse in NT seems just as easy as fd reuse in Linux. Internally, NT handles have the first two bits cleared, and the rest are an index into a handle table that is allocated sequentially (handles look like 0x4, 0x8, 0xc, 0x10, etc.) When allocating a new handle, NT doesn't strictly reuse the lowest unused value like Linux/Unix, but it seems to prefer to reuse handles rather than expanding the handle table, so handle values are reused often. And yeah, Linux file descriptions are synchronous by default, but so are NT file objects, and unlike Linux, an NT file object that was opened in synchronous mode can't be changed to asynchronous mode. This can make it annoying to write programs and libraries that work with arbitrary handles, since even if all your handles are asynchronous, handles created out of your control might not be. stdin, stdout and inherited CRT fds are all synchronous by convention. This combined with the fact that asynchronous handles can act like synchronous handles for various reasons (eg. NTFS compression/encryption, resource contention in the cache manager,) means you have to write asynchronous code with the assumption that every IO operation on a particular handle will always block. I've heard this is why libuv chooses to use synchronous IO with file handles on Windows, even though asynchronous IO is possible. So we could stand to learn some lessons from NT, but Linux IO isn't "worse in every way." I think an operating system with a good IO/AIO system would have to leapfrog both NT and Linux and aim for something better.

loeg 8y ago
> Internally, NT handles have the first two bits cleared, and the rest are an index into a handle table that is allocated sequentially (handles look like 0x4, 0x8, 0xc, 0x10, etc.) When allocating a new handle, NT doesn't strictly reuse the lowest unused value like Linux/Unix, but it seems to prefer to reuse handles rather than expanding the handle table, so handle values are reused often. This is a valid criticism of the implementation, but the major consideration is that this property is not baked into the API. In Unix, the next fd allocated must be the lowest fd number available. Applications rely on this property. An implementation is constrained in that it cannot allocate randomly to attempt to avoid reuse problems. The NT API does not have this problem. > an NT file object that was opened in synchronous mode can't be changed to asynchronous mode. This can make it annoying to write programs and libraries that work with arbitrary handles, since even if all your handles are asynchronous, handles created out of your control might not be. This is a good criticism of the NT API. I agree. A newly designed API probably shouldn't attach synchronicity to a property of a handle, but instead to individual operations. > This combined with the fact that asynchronous handles can act like synchronous handles for various reasons (eg. NTFS compression/encryption, resource contention in the cache manager,) IMO, this is an implementation flaw in the internal implementation of the NTFS filesystem on NT. For exactly the reasons you go on to describe — it means the async API operation cannot be trusted to be async. If I were designing this today, such an implementation would be a violation of API invariants. > So we could stand to learn some lessons from NT, but Linux IO isn't "worse in every way." Sure. Linux AIO is only worse most ways, and needs a thorough treatment like you've given here to really dive into the details. I think "Linux AIO is worse" is still the accurate one sentence summary.

by rossy 8y ago

loeg 8y ago

> Internally, NT handles have the first two bits cleared, and the rest are an index into a handle table that is allocated sequentially (handles look like 0x4, 0x8, 0xc, 0x10, etc.) When allocating a new handle, NT doesn't strictly reuse the lowest unused value like Linux/Unix, but it seems to prefer to reuse handles rather than expanding the handle table, so handle values are reused often. This is a valid criticism of the implementation, but the major consideration is that this property is not baked into the API. In Unix, the next fd allocated must be the lowest fd number available. Applications rely on this property. An implementation is constrained in that it cannot allocate randomly to attempt to avoid reuse problems. The NT API does not have this problem. > an NT file object that was opened in synchronous mode can't be changed to asynchronous mode. This can make it annoying to write programs and libraries that work with arbitrary handles, since even if all your handles are asynchronous, handles created out of your control might not be. This is a good criticism of the NT API. I agree. A newly designed API probably shouldn't attach synchronicity to a property of a handle, but instead to individual operations. > This combined with the fact that asynchronous handles can act like synchronous handles for various reasons (eg. NTFS compression/encryption, resource contention in the cache manager,) IMO, this is an implementation flaw in the internal implementation of the NTFS filesystem on NT. For exactly the reasons you go on to describe — it means the async API operation cannot be trusted to be async. If I were designing this today, such an implementation would be a violation of API invariants. > So we could stand to learn some lessons from NT, but Linux IO isn't "worse in every way." Sure. Linux AIO is only worse most ways, and needs a thorough treatment like you've given here to really dive into the details. I think "Linux AIO is worse" is still the accurate one sentence summary.