8 ms·
NT is different from Unix, it is heavily inspired by VMS, which was the anti-Unix of its time.
The basic architectural metaphors are different, and the way you
NT is different from Unix, it is heavily inspired by VMS, which was the anti-Unix of its time.
The basic architectural metaphors are different, and the way you think about things is different.
It is more architectured and more opinionated, which may look like over-engineering for some and a superbly coherent design for others.
There are some pain points, of course, a lot of them related to the need to support code written for a completely different OS (MS-DOS + Windows non-nt), but overall, I think that the general architecture of NT is pretty neat, and I would dare say, beautiful in some places.
This is really the New Jersey approach vs the MIT approach:
https://www.jwz.org/doc/worse-is-better.html https://www.jwz.org/doc/worse-is-better.html
It appears that author does not like HN as this redirects to https://cdn.jwz.org/images/2016/hn.png https://cdn.jwz.org/images/2016/hn.png
Is that the same wanker that got in a hissy fit over distros including xscreensaver ?
Better link (from the actual author of the text, and without referrer tricks for links from HN): https://dreamsongs.com/RiseOfWorseIsBetter.html https://dreamsongs.com/RiseOfWorseIsBetter.html
One thing I have trouble with in windows is debugging problems. If say I plug in a usb device and nothing happens, I know what logs to look at, and they are verbose and if I don't, I can search the internet and find five or six possible solutions depending on distro and age. In windows the search generally says "download this malware and it will solve your problem", "restore to a save point" or "grab your install media and start over". I would be surprised if there is a good way to do the same things, they just aren't discoverable. When the device manager and services.msc show all is well, but it clearly isn't, there doesn't seem to be much advice out there.
Edit: I liked vms fine, I don't think this is inheritance from that, or even dos. I knew how to debug in dos, using debug.com even. I think it has something to do with the lack of expectations for a user to be able to debug a problem. Osx has abit of this, but sitting on a bsd saves it to some extent.
It's absolutely a problem. Aka the sysinternals / Bryce Cogswell and Mark Russinovich problem.
To wit, Windows has plenty of debugging functionality. Historically not as extensive as Unix, but it was there.
However, 0% of that was exposed in any sane way, because Microsoft...? Actually, I have no idea why it wasn't. For the price of 2 senior engineers and a willingness to actually release tools, Microsoft could have drastically improved the situation.
Instead they just (eventually) bought sysinternals. :(
> superbly coherent design [...] NT is pretty neat, and I would dare say, beautiful in some places
Practically, though, it's a shit show. My "favorite" is the inability to delete a file that is open by another process. Combine this with Windows Defender that scans every file that your application creates, and now you have no reliable way of deleting or moving your files.
As someone who works on a C/C++ build toolchain and has to deal with Windows design decisions every day, I can tell you without any hesitation there is nothing coherent or beautiful about it. I don't think even Microsoft believes this.
> Practically, though, it's a shit show.
I usually put it as "Deep inside windows is a really beautiful OS just screaming to be let out from under all that junk on top."
The NT kernel isn't that bad, different from unix for sure but still good. It then just keeps getting progressively worse as you go upwards in the stack from there.
> My "favorite" is the inability to delete a file that is open by another process.
What does the alternative look like?
Lets say a process has a file open. Lets say it is a really big one, like 200GB. Ok, so our disk is nearly out of space, better delete this 200GB file, problem solved right?
Only not. Because now even though the file looks deleted, the process still has it open and that space is still allocated to it.
So either you're hiding reality from the user, or you're preventing them from deleting a file that is in use. Windows went one way and UNIX went another.
NT upto 4 was beautiful when it was released in my opinion.
It was a "serious" "workstation" operating system.
Supporting multiple platforms.
The ability to run different "operating systems" was neat.
The graphics stack was all in user space.
The biggest hope was a selling WindowsNT with Alpha CPUs
to the Defense department if memory serves.
(Which is why the POSIX parts were added)
That never happened.
NT4 was not made to be the best OS for games.
After that Microsoft has cut a lot of neat parts out
and bastardized the clean separation between the kernels
and user space in order to make the OS faster.
I wish they had branched it
"Windows Home"
and
"Windows Workstation"
It would be a lot of work I guess.
They do have Windows Service but I think that
it has undergone the same changes.
NT4 already had the graphics stack in kernel mode.
They tried that.
It was called Windows Me.
Also, the selling point of the Professional editions was and continues to be use in Active Directory and business environments, not developer workstations.
IO completion ports (IOCP) was such a brilliant design, and still holds up today. It's fascinating how long it took for similar paradigms to become more dominant elsewhere eg with kqueue
"A process is writing to a 200GB file"
There's two problems here, the process and the file, it seems you need to deal with both situations on both styles of OS. You could truncate the file (there's also a `truncate` command I have never used):
cat /dev/null > somefile
and let the process continue writing, or you could just `rm` the file, then either kill the process (or the other way round), or if you've got mysterious disk space usage issues you think are due to still open but deleted files you can:
lsof | grep deleted
I'm an ex (mostly) sysadmin and so this stuff is easy, but is obviously out of reach for "normal" users. Personally as someone who still uses Unix-like machines (Mac, servers) and Windows (games, chkdsk) and I greatly prefer the Linux/Unix way and find the Windows way frustrating.
> Lets say a process has a file open. Lets say it is a really big one, like 200GB. Ok, so our disk is nearly out of space, better delete this 200GB file, problem solved right?
Not naively intuitive, but for this you can write an empty file to the large open file with shell redirection.
cat /dev/null > /path/to/big/file
and that will free up the space on disk immediately. Not great, but more or less nicely handles the somewhat common runaway log file case.
I get how this corner case sucks, but to me it seems so much better than the endless parade of reboots and application restarts necessitated by file locking in the Windows world.
This is by design though, if another process has a lock on a file, they don't want you to delete it, since it could break another process. They actually do allow it if the applications open it with the createfile flag FILE_SHARE_DELETE. Or you can do it as a transacted operation, which will delete the file "eventually". Again, it's by design, you have to be explicit if you want someone to delete a file you're using, which is honestly a better design than having it be the default with the option of locking the file like Linux does.
If you really want to close it and know you could break another process but don't care(Which you should), you can use NtQueryFileInformation to get the process ID's that have the file opened, then use OpenProcess with the PID, then use NtQueryProcessInformation to get the open handles. Call DuplicateHandle on each of the handles. Then use NtQueryObject on each of them to get the name of the file, then close it.
> since it could break another process.
This isn't how things work on Linux. It keeps the file "alive" as long as there is at least one process using it. Any processes which do not have the file open already will not be able to see it but those that do can keep using it. The file will be deleted when all processes release the file.
"don't want you to delete it, since it could break another process."
There is no such problem.
On all the unix-likes, the kernel handles that perfectly naturally.
Think of deleting a file as requesting the file deletion from the kernel rather than directly doing it like in assembly with no os.
When you delete a file, the only thing that happens for sure immediately is no other process can see it in the filesystem, so no new access can be made. To all processes that didn't already have an open file handle, it is effectively deleted and no longer exists.
The contents are not necessarily touched yet, and the filesysyem has not yet necessarily released the occupied inodes and blocks for any other use yet, they are all still tracked as the original file, but now invisible that nothing else except the kernel can see or access.
It stays like that as long as any process anywhere has an open file handle to that file.
Any process with a handle can continue using that handle as normal. If it was opened for write, it can continue to write, seek, read, etc, whatever modes the handle was created with.
The kernel even keeps on coordinating between multiple processes accessing the same now-invisible file. The open file handles aren't just pacifiers, it's still a real file.
But no new file handles can be opened. One by one as processes close file handles, they can no longer open new ones, until the last user has released the last handle.
Only at that point the kernel frees the inodes and blocks in the filesystem making the disk blocks available for new files.
No other processes break, and it's all perfectly graceful and not a problem at all.
For NT not to have that is like some 70's trs-80 stuff.
And holy cow that stuff you just described about querying all other processes...holy cow, not an advertisement for a great, desirable, slick system. "37 easy steps!" It's almost like you were writing a joke to say something is reasonable and then proceed to describe an absolutely laughable process.
> "But no new file handles can be opened. One by one as processes close file handles, they can no longer open new ones, until the last user has released the last handle."
Or, you know... the computer has to restart itself for some reason before the file was fully deleted, and you get a fragmented disk.
If I'm asking to delete a file, I want the file deleted. I want to know whether the deletion succeeded or not. I don't want it to secretly hide in a purgatory somewhere.
Is this the year of the Linux desktop already?
I recently discovered that Linux handles processes holding files open a little differently from FreeBSD, too, when I tried to unmount a ZFS volume. Something had ahold of a file on it, so I couldn't unmount it—on FreeBSD you can just say "I want all programs that are preventing this to fuck off and, if they're so inclined, to crash"—and ZFS provides a flag to do just that—and go on with your day, but Linux evidently won't let you (I ended up on some ZFS mailing list archive to confirm this was the case and I wasn't just holding it wrong). You have to go track the process(es) down and kill it(/them) yourself, not unlike how you can't delete a file that's in use on Windows and have to go kill the processes. It was faster to just reboot the damn machine than to do that (a very Windows solution).
[EDIT] It may have been a ZFS export operation, not exactly an unmount? I can't recall for sure. Point is, Linux made you go find and kill the relevant processes yourself, while FreeBSD made it trivial to override that and do what you needed to do with no further fuss.
Back in the day there was a patch floating around that let you mark mounts as "bad", and all open files would be shunted to a "badfs" which gave I/O errors for all operations.
Most of the discussion around it has linkrotted. https://lwn.net/Articles/192632/ https://lwn.net/Articles/192632/ has some mentions.
Some filesystems implement a less-comprehensive variant that's `umount(2)` with `MNT_FORCE` -- but generally that's only network/FUSE-style ones:
MNT_FORCE (since Linux 2.1.116)
Ask the filesystem to abort pending requests before attempting the
unmount. This may allow the unmount to complete without waiting
for an inaccessible server, but could cause data loss. If, after
aborting requests, some processes still have active references to
the filesystem, the unmount will still fail. As at Linux 4.12,
MNT_FORCE is supported only on the following filesystems: 9p (since
Linux 2.6.16), ceph (since Linux 2.6.34), cifs (since Linux
2.6.12), fuse (since Linux 2.6.16), lustre (since Linux 3.11), and
NFS (since Linux 2.1.116).
Also, in theory Windows doesn't need reboots because like you said, the rename and replace works just fine for hot patches. Windows reboots as often as it does in part for "backwards compatibility" and the long tail of applications built under the broken 90s assumption that windows "doesn't" hot patch things.
Linux assumes that long running processes just randomly exploding because they are living in a pre-hotfix ghost state is fine (and usually free of security concerns). Windows assumes users notice and get frustrated when long running apps start acting "weird" and don't know why and that there are too many security implications if you just leave unpatched apps running alongside patched ones.
It's another one of those things on the list that the NT Kernel has the raw design to do a lot of smart, deliberate things, but at the end of the day can't trust user space to play nice with them happening and is maybe a little too safety conscious of that.
(My understanding too is that Ubuntu is an interesting example here because Canonical now explicitly uses a more Microsoft-influenced security model in user-focused distributions for when to require reboots. Under the same general concerns: avoid user-noticeable "weirdness" and avoid unpatched CVEs remaining long running in user space. While it could just kill unpatched processes, that creates "user noticeable weirdness", and for better and worse a full "restart" is much less shocking to the average user than long running processes getting killed "weirdly".)
> And I didn't delete the zip file, I deleted the original file that I zipped.
OK, I think that's where a lot of the confusion comes from.
When you originally said "I zipped a file and deleted it, but the process which created it was still writing to it.", I thought the "it"s referred to the zip file, not the original file, and "the process that created it" was the zip process.
Going by your actual meaning - are you saying you zipped a file on disk that was still being written to? That's... weird.
Given that reading from disk is faster than writing to it, and the written data may well have been in the disk cache anyway, it's likely that the zip program could have "caught up" with the program that was writing the data, tried another read and been told "this is the end of the file", and then finished creating the zip file. That would have created an incomplete zip.
But also, as others have pointed out, just because you delete a file and it gets removed from the directory, the actual file and its contents stay around until all open handles to it are gone. So if one process has the file open and is writing to it, and a second process has the file open and is reading from it, if you delete the file then both processes should still be able to access it through their existing file handles, even though it doesn't have a directory entry any more. So you shouldn't have experienced any data loss from that series of events.
(And that's why having the "it"s refer to the zip file is a logical interpretation of what you said, and why a few repliers - including me! - seem confused)
This is a ridiculous comment.
If the kernel gets to shut doen gracefully, then there is no problem at all. The file handles are closed when the processed are killed and the filesystem is unmounted in a consistent state.
If the kernel does not get to shut down gracefully (crash or power loss) the filesystem is left no more or less inconsistent than by any other kind of ongoing activity (nothing worse about a pending free-these-blocks vs a pending anything else), and is cleaned up by fsck or by the journal if there is one. Either way the file is finished being deleted on reboot because no processes have open handles and the kenel and the fs know that those blocks are deleted. The last on-disk state of those inodes was that the blocks are not used by any normal file. They are marked pending to be freed but hadn't been freed yet, but now during the mounting process at boot there are no processes that own them, so they are freed. It's no different than the normal process if there were no power loss.
There is no such problem.
As for "some kind of purgatory" and "I want to know it's done now" these imply ignorance of what a multi process operating system is, what it needs to do, the basic job of a kernel, etc. Not just in linux but nt and any other os that allows multiple concurrent processes.
It's the primary job of the kernel to manage exactly such coordination between a process, another process, and the hardware. Slightly virtualizing things like ram and disk is the very job the kernel is there to do, and what makes multiple concurrent processes even posible.
When you delete a file from a process and get your return from that syscall, it IS "really done right now" as far as that process and all other processes are concerned, and "from your point of view" is as "real" as anything ever gets for any process. The fact that some other process can continue reading and writing to a file handle you no longer care about is none of your business any more, and your intent that the file no longer exist in the filesystem so that it could be created again without conflict, or so that it no longer appears in listings, all does happen as you requested, and immediately.
If you had a file named foo, and you needed to delete that because you need to create a directory named foo, you can do that. It happens exactly as you need, even though some other process now has a file handle open to some invisible file that used to be named foo. That other file handle is just not your problem any more. It's as if the other process simply opened a new temp file in some other firectory that has nothing to do with you.
If you read my comment, you can see in the first few words I mention how you can do a delete transaction and it will delete it when it's not being used anymore. Or you can also just open the file with the share delete flag and it will allow the same. And linux doesn't handle it perfectly, you see errors like "text file busy". I don't get why you chose to argue how the non recommended, brute force method, is lengthy. It's not intended to be used and was mainly just to show you can, and there's nothing stopping you.
Why is explicitness bad? Would you rather have those weird errors on linux where it's open by default? Or have it where the program author has to say "Yes, this file is fine to be deleted while i'm using it".
I'm just going by the comments of others but:
"Yes, this file is fine to be deleted while i'm using it"
is fine if all processes play nice, which apparently Windows Defender _does not_. Edit: likely Windows Defender couldn't work if it did? So, in order to consistently delete a file without triggering the file lock, you need to use the work-around.
I'm just guessing here, but wouldn't Windows Defender need to be some privileged process (as it's e.g. an anti-virus) where then this file-locked file deletion process would be impossible to perform? Otherwise, a computer virus could just keep killing the Windows Defender file handler, making it impossible to read its files and keeping Windows Defender from performing its duty.
Exactly, the developers of Defender decided it needed to lock the file, arguably for alot of reasons. To prevent tampering etc, it's an antivirus, it doesn't want you to be able to bypass a scan of a file, or change it. Windows has things like Datastreams(Which come from Mac), so the same "file", can actually have two different files in it, and require different file openings. IE, "Foo.exe" and "Foo.exe:MALWARE" would be completely different file contents, despite both being foo.exe. And OpenFile("Foo.exe"), would not open foo.exe:MALWARE.
Alot of the confusion comes from people not understanding the functionality. Delete file means "Delete it now" on windows(ignoring slack space). But on linux it means "Queue up the file to be deleted". Windows also has this, but it's going to be a transacted operation, which is literally called DeleteFileTransacted vs DeleteFile, so the Kernel sees noone is using it and then will delete it.
And honestly, if you go up to any lay person and ask them "What does 'DeleteFile'" do. They'll probably respond it deletes the file. Not queues it up for deleted, but other people can still access it. So I think the windows verbage makes more sense, with another function appending Transacted to it, which signifies it will be done eventually.
There is nothing about antivirus that requires file deletion block on other processes including the antivirus. For one thing, you just said Windows has a similar functionality available anyway just invoked a different way. Is that a way around Defender? It better not be. If it exists as something you can use, and yet still isn't a way around Defender, then it's silly to even be talking about Defender to justify this 70's behavior.
The understanding of laypeople in how a multitasking operating system kernel coodinates a graceful teardown of a shared resource, or fails to in the case of Windows, has exactly no bearing on anything. Why isn't it exactly as reasonable to presume that "any lay person" would expect that they can simply issue a delete command and it happens, without having sit there and wonder why they're stuck and go worry about other processes that they didn't write and don't have any knowledge or control over? It is exactly as reasonable. And both are pure meaningless presumption, and don't matter anyway since the understanding of a lay person would be a ridiculous way to design the inner workings of ... anything, definitely including an operating system.
These arguments are swiss cheese.