6 ms·

Why is “Everything is a File” unique to Unix operating systems?

51 points by sathyabhat 15y ago

joe_the_user 15y ago
Googling on the topic, I found: http://www.linuxtopia.org/online_books/programming_books/art_of_unix_programming/ch03s02_3.html http://www.linuxtopia.org/online_books/programming_books/art... An interesting discussion of how Window NT's lack of everything-is-a-file and standard command line tools metastasizes into a security and configuration nightmare. It gives an idea of how important unix's the unifying metaphors are for creating a good operating system.
- 4ad 15y ago
  Windows NT, just like any relevant operating in the last 30 years, operates on the same I/O to files abstraction as Unix. I'd say Windows NT does it even more than modern Unix, at least in NT NICs are real devices in the namespace. The BSD people in the '80s forgot this fundamental abstraction and introduced sockets, then X came, again neglecting 15 years of good practice and introduced a completely incompatible model to do I/O. Rob Pike, Ken Thompson and other Unix people were dissatisfied with this and created Plan9, a Unix descendant where truly all I/O happens by operating on the namespace and without an ioctl(2) system call. Many, including myself, believe Unix took a very wrong path in the 80s and that modern Unix is more clumsy and inconsistent than the systems it was designed to replace, but architecturally speaking Windows NT is pretty much the same as modern Unix. /edit: I've seen both in the cited article and in OPs link that people confuse files with files descriptors.
  - drothlis 15y ago
    What does "operating on the namespace" mean?
    - 4ad 15y ago
      To do any kind of I/O you just open a file from that process namespace using a set of standard system calls. It doesn't matter what resource you're accessing (regular file, network, window on the screen, window on someone else's screen, etc), the interface to do so is the same. Compare that to using specialized APIs for different type of I/O, for example using the socket API to do networking, or to use special X libraries to do graphics programming, or to hide another API inside ioctl(2). Namespaces in Plan9 are not global, each process has a private namespace (though many processes share most of it) allowing the composition of interfaces. (/dev/tty in Unix is different for each process as well). Each graphics program on Plan9 writes to /dev/draw and reads from /dev/mouse and /dev/console, but those are synthetic files implemented by the window manager by multiplexing the framebuffer, mouse and console. As you can see even though the files are synthetized the names in the namespace are the same because namespaces are private for each application and the window manager binds the synthetic files to those names. /edit: Ah, and one more thing, file-based I/O means shell scripts can do everything a compiled program could do, like networking.
  - geocar 15y ago
    What exactly do you mean by this? > at least in NT NICs are real devices in the namespace.
    - 4ad 15y ago
      In Windows NT device objects get instantiated in \Device, including network cards. Also, different TCP/IP layers are accessible as well, like \Device\Tcp, \Device\Ip. In Unix the network stack is not accessible in the file system at all, there's nothing in /dev, it's just a special API in the kernel. In Windows you'd use a networking API as well, but the networking API is a convenience layer over file based I/O. The difference between Windows NT and Plan9 is that Windows NT device I/O is IOCTL based, not composable, not replaceable and not exportable, but it's just how you'd do things in Unix v7.
- evmar 15y ago
  I used to think this, but I did more Windows programming and discovered it's not as bad as the Unix viewpoint makes it out to be. It's true that on Windows a file descriptor for a socket has different APIs than for a file, but that's only because file descriptors appear to have been implemented as some half-assed Unix compat layer. The Windows API instead uses a "handle" type, and you can wait for events (with a similar API to select() or poll()) on handles of heterogeneous types, including files, sockets, processes (to listen for process death), message loops (to listen for incoming messages), or condition variables. (See the "remarks" section of http://msdn.microsoft.com/en-us/library/windows/desktop/ms687025(v=vs.85).aspx http://msdn.microsoft.com/en-us/library/windows/desktop/ms68... .) It's ultimately sorta similar to how even-driven loops work in the Unix world, but in practice it's actually much harder to do that sort of thing on Unix, where you instead need to e.g. create a wakeup pipe to fudge `SIGCHLD` into a select loop or rely on newer stuff like `FUTEX_FD` to hook a mutex in. (I'm no Windows apologist, and I think some of the APIs like how async IO work are pretty complex and painful, but I do think the superficial "they got file descriptors wrong" claim is wrong.)
ralfd 15y ago
On the linked superuser site there is repeatedly mentioned that Mac OS X is build "on top" of Unix. This is not quite right. OS X has a BSD userland, but in many ways its developers thought differently. For example "/dev/audio" doesn't exist on OS X. CoreAudio is taking a different route than the "everything is a file" Unix approach.
- 4ad 15y ago
  And /dev/tcp does not exist in Unix. Neither does /dev/draw. Everything is a file broke in Unix in the '80s when sockets and X windows came. Original Unix' developers tried (and succeeded, IMO) to fix these problems and created Plan9. In Plan9 all I/O, being regular I/O, GUI applications, networking etc are done through the namespace. Unix means two things today, one is a useless certification, usually referred as UNIX and the other one is a loose set of tools, ideas and encompassing practices. Mac OS X fits both these definitions.
- zokier 15y ago
  On the other hand OS X is actually certified UNIX. So saying that OS X is built on top of UNIX is imho very correct. If the current specifications for UNIX match the original ideologies of Unix is completely different matter.
- rat87 15y ago
  """For example "/dev/audio" doesn't exist on OS X.""" or linux
JoeAltmaier 15y ago
It may be handy at one level to treat everything as a file. But squashing complex objects through the file api is a hack. I'd hope somebody had some better ideas in the last 20 years.
- grandalf 15y ago
  It's not a hack, it's just one set of tradeoffs with pluses and minuses, like any other engineering decision.
- 1337p337 15y ago
  If you have a look at, e.g., Plan 9 and Inferno, I think you might be surprised at how expressive a file-based API can be. For a trivial example, webfs supports REST quite handily without a loss of expressive power (and REST itself is a simple but powerful abstraction), and because things need not be piped through curl, you can, for example, point your image editor at what it believes to just be another file on the filesystem, rather than requiring the image editor to support HTTP. The other half of this is that abstractions can and are built atop this. The Limbo programming language, for example, provides typed channels (which, as I understand it, also made it into Go) and its standard library provides functions for treating arbitrary files as typed channels. So, the OS need only concern itself with "files" (really, in Plan 9 and Inferno, these are less like files and more like names in the processes' respective namespaces), and whatever complex objects the language concerns itself with are easy to ship around between a process and the outside world without hacks (unless marshalling objects is considered a hack).
Craiggybear 15y ago
In DOS everything was a file too. Its not just *nix.
- colanderman 15y ago
  Definitely not. Once you got past CON, LPT1, and COM1, it was BIOS calls and I/O ports all the way.
  - Craiggybear 15y ago
    COM and LPT are files, though. And can be treated as such. As in COPY *.PS LPT1: Devices such as screens were treated as files by their device drivers as well.
    - colanderman 15y ago
      COM and LPT are files, though. Right, that's what I said. Devices such as screens were treated as files by their device drivers as well. I remember writing a whole hell of a lot of port-banging code to make graphics and sound work on DOS.
deleted 15y ago
[deleted]

51 points by sathyabhat 15y ago

Googling on the topic, I found: http://www.linuxtopia.org/online_books/programming_books/art_of_unix_programming/ch03s02_3.html http://www.linuxtopia.org/online_books/programming_books/art... An interesting discussion of how Window NT's lack of everything-is-a-file and standard command line tools metastasizes into a security and configuration nightmare. It gives an idea of how important unix's the unifying metaphors are for creating a good operating system.

4ad 15y ago
Windows NT, just like any relevant operating in the last 30 years, operates on the same I/O to files abstraction as Unix. I'd say Windows NT does it even more than modern Unix, at least in NT NICs are real devices in the namespace. The BSD people in the '80s forgot this fundamental abstraction and introduced sockets, then X came, again neglecting 15 years of good practice and introduced a completely incompatible model to do I/O. Rob Pike, Ken Thompson and other Unix people were dissatisfied with this and created Plan9, a Unix descendant where truly all I/O happens by operating on the namespace and without an ioctl(2) system call. Many, including myself, believe Unix took a very wrong path in the 80s and that modern Unix is more clumsy and inconsistent than the systems it was designed to replace, but architecturally speaking Windows NT is pretty much the same as modern Unix. /edit: I've seen both in the cited article and in OPs link that people confuse files with files descriptors.
- drothlis 15y ago
  What does "operating on the namespace" mean?
  - 4ad 15y ago
    To do any kind of I/O you just open a file from that process namespace using a set of standard system calls. It doesn't matter what resource you're accessing (regular file, network, window on the screen, window on someone else's screen, etc), the interface to do so is the same. Compare that to using specialized APIs for different type of I/O, for example using the socket API to do networking, or to use special X libraries to do graphics programming, or to hide another API inside ioctl(2). Namespaces in Plan9 are not global, each process has a private namespace (though many processes share most of it) allowing the composition of interfaces. (/dev/tty in Unix is different for each process as well). Each graphics program on Plan9 writes to /dev/draw and reads from /dev/mouse and /dev/console, but those are synthetic files implemented by the window manager by multiplexing the framebuffer, mouse and console. As you can see even though the files are synthetized the names in the namespace are the same because namespaces are private for each application and the window manager binds the synthetic files to those names. /edit: Ah, and one more thing, file-based I/O means shell scripts can do everything a compiled program could do, like networking.
- geocar 15y ago
  What exactly do you mean by this? > at least in NT NICs are real devices in the namespace.
  - 4ad 15y ago
    In Windows NT device objects get instantiated in \Device, including network cards. Also, different TCP/IP layers are accessible as well, like \Device\Tcp, \Device\Ip. In Unix the network stack is not accessible in the file system at all, there's nothing in /dev, it's just a special API in the kernel. In Windows you'd use a networking API as well, but the networking API is a convenience layer over file based I/O. The difference between Windows NT and Plan9 is that Windows NT device I/O is IOCTL based, not composable, not replaceable and not exportable, but it's just how you'd do things in Unix v7.
evmar 15y ago
I used to think this, but I did more Windows programming and discovered it's not as bad as the Unix viewpoint makes it out to be. It's true that on Windows a file descriptor for a socket has different APIs than for a file, but that's only because file descriptors appear to have been implemented as some half-assed Unix compat layer. The Windows API instead uses a "handle" type, and you can wait for events (with a similar API to select() or poll()) on handles of heterogeneous types, including files, sockets, processes (to listen for process death), message loops (to listen for incoming messages), or condition variables. (See the "remarks" section of http://msdn.microsoft.com/en-us/library/windows/desktop/ms687025(v=vs.85).aspx http://msdn.microsoft.com/en-us/library/windows/desktop/ms68... .) It's ultimately sorta similar to how even-driven loops work in the Unix world, but in practice it's actually much harder to do that sort of thing on Unix, where you instead need to e.g. create a wakeup pipe to fudge `SIGCHLD` into a select loop or rely on newer stuff like `FUTEX_FD` to hook a mutex in. (I'm no Windows apologist, and I think some of the APIs like how async IO work are pretty complex and painful, but I do think the superficial "they got file descriptors wrong" claim is wrong.)

ralfd 15y ago

On the linked superuser site there is repeatedly mentioned that Mac OS X is build "on top" of Unix. This is not quite right. OS X has a BSD userland, but in many ways its developers thought differently. For example "/dev/audio" doesn't exist on OS X. CoreAudio is taking a different route than the "everything is a file" Unix approach.

4ad 15y ago
And /dev/tcp does not exist in Unix. Neither does /dev/draw. Everything is a file broke in Unix in the '80s when sockets and X windows came. Original Unix' developers tried (and succeeded, IMO) to fix these problems and created Plan9. In Plan9 all I/O, being regular I/O, GUI applications, networking etc are done through the namespace. Unix means two things today, one is a useless certification, usually referred as UNIX and the other one is a loose set of tools, ideas and encompassing practices. Mac OS X fits both these definitions.
zokier 15y ago
On the other hand OS X is actually certified UNIX. So saying that OS X is built on top of UNIX is imho very correct. If the current specifications for UNIX match the original ideologies of Unix is completely different matter.
rat87 15y ago
"""For example "/dev/audio" doesn't exist on OS X.""" or linux

JoeAltmaier 15y ago

It may be handy at one level to treat everything as a file. But squashing complex objects through the file api is a hack. I'd hope somebody had some better ideas in the last 20 years.

grandalf 15y ago
It's not a hack, it's just one set of tradeoffs with pluses and minuses, like any other engineering decision.
1337p337 15y ago
If you have a look at, e.g., Plan 9 and Inferno, I think you might be surprised at how expressive a file-based API can be. For a trivial example, webfs supports REST quite handily without a loss of expressive power (and REST itself is a simple but powerful abstraction), and because things need not be piped through curl, you can, for example, point your image editor at what it believes to just be another file on the filesystem, rather than requiring the image editor to support HTTP. The other half of this is that abstractions can and are built atop this. The Limbo programming language, for example, provides typed channels (which, as I understand it, also made it into Go) and its standard library provides functions for treating arbitrary files as typed channels. So, the OS need only concern itself with "files" (really, in Plan 9 and Inferno, these are less like files and more like names in the processes' respective namespaces), and whatever complex objects the language concerns itself with are easy to ship around between a process and the outside world without hacks (unless marshalling objects is considered a hack).

Craiggybear 15y ago

In DOS everything was a file too. Its not just *nix.

colanderman 15y ago
Definitely not. Once you got past CON, LPT1, and COM1, it was BIOS calls and I/O ports all the way.
- Craiggybear 15y ago
  COM and LPT are files, though. And can be treated as such. As in COPY *.PS LPT1: Devices such as screens were treated as files by their device drivers as well.
  - colanderman 15y ago
    COM and LPT are files, though. Right, that's what I said. Devices such as screens were treated as files by their device drivers as well. I remember writing a whole hell of a lot of port-banging code to make graphics and sound work on DOS.

deleted 15y ago

[deleted]