5 ms·

I'd like to know, from anyone with Plan 9 experience, does its approach to "everything is a file" solve these problems?

by robert_tweed 7y ago

pjmlp 7y ago
No, everything is a file doesn't work at all if you care about high performance graphics and real time audio.
- kragen 7y ago
  qznc made the same claim. Why would "everything is a file" be incompatible with high-performance graphics and real-time audio? I'm particularly interested in this question because my knowledge of those areas is not very deep, so I'm wondering what unknown unknowns I'm missing. Here's my naïve thinking: · Real-time audio requires low-latency scheduling, which is unrelated to the API for audio itself, and FIFO depth monitoring so that you don't get either buffer overruns or buffers that are so full that they induce latency in excess of what your scheduling and processing latency unavoidably adds. For output, you could provide this with a /dev/audio for writing data and a /dev/audiooutputbuffersize for reading the FIFO buffer depth. For input, you could provide this by making reads from /dev/audio nonblocking, by providing a /dev/audioinputbuffersize you would read before risking a blocking read on /dev/audio, or by reading in a separate thread. · High-performance graphics covers a lot of ground, but one of the big pain points is unnecessary memcpying on the paper path from your CPU graphics program into the GPU, whether that's textures, vertex buffers, or prerendered pixel data (though maybe we could argue that that last case is never going to be high performance.) One way that unnecessary memcpying arises is from buffers for write() that are not correctly aligned with respect to the start of a page, which unavoidably means that they will have to get copied to a place with the correct alignment. This is a big problem if you're streaming out gigabytes per second of graphics data. In particular you don't want the data that goes to the GPU concatenated with data that tells the kernel graphics driver what to do with it. Solution: open a new file in a /dev/gfx directory for the data to be sent to the GPU, then write the data to it from a properly aligned buffer. Fork a separate thread to write this data, which is done by marking the pages COW rather than copying them. (A more radical and less sneaky solution I'm exploring for Wercam and BubbleOS is to transfer ownership of pixel data buffers from the application to the window system, causing them to disappear from the application's memory map, avoiding the need to either copy or COW them. But this definitely departs from the Unix read()/write() model.) · A different aspect of high-performance graphics, other than raw throughput, is that you might need to synchronize your drawing with monitor refreshes to avoid an extra half-frame of latency, particularly at low refresh rates like 60 Hz. This seems straightforward to solve in a filesystem interface: reading from /dev/vbi produces a byte just before each VBI, giving you time to send any necessary new data to the GPU. Certainly it's true that Plan 9 does not provide high-performance graphics or low-latency audio. But I don't think that proves that filesystem-based interfaces can't provide them, just that the Plan 9 designers had spent the 1980s researching DSLs, fault-tolerance, concurrency, and typesetting, not rasterization, real-time control, and animation. In general, system-call or IPC interfaces of the form "invoke function foobar with a baz handle and a quux struct or memory buffer" can be cleanly replaced with "open /bazzes/$i/foobar and write a quux struct or memory buffer to it", can't they? I mean, that's three system calls instead of one, so about 1 μs of system-call overhead instead of 300 ns, but in the cases where that's the performance bottleneck you can usually solve it by writing N quuxes instead of one. Can't you? A potential problem arises when you need to combine multiple handles to extra-process resources in a single system call, like SCM_RIGHTS or rename(), but I don't know where those would come up in the particular application domains you identify as problematic. But maybe my proposed solutions above won't solve the problems I think they will, or maybe I'm missing the biggest issues entirely. What am I missing? (I'm not claiming that this approach would be easier, safer, or more discoverable than the approach of adding a bunch of system calls; Benno's talk discusses the many ingenious ways a filesystem-based API can be hard to use, and no doubt some of the same criticisms can be leveled at the above strawman proposals. I'm just saying I don't see where it inherently fails to meet performance requirements.)
  - tedunangst 7y ago
    Perhaps getting into no true ship of theseus territory, but you can redefine read/write to do anything but it's arguable if that's still the same philosophy. Like, why even have open()? You could start each process with "/" already open, and then open other files by writing "open /etc/passwd" and reading back a new fd.
    - kragen 7y ago
      Yeah, so, is there a there there? I think there is. The open/close/read/write/getdents interface at least offers the possibility of certain kinds of REST-like uniformity, providing some benefits: - A thin hourglass-waist interface makes it relatively easy to add new components to the system, whether clients, servers, or proxies. Plan 9 implemented both network fileservice and GUI windowing as filesystem-interface proxies, and they composed properly, allowing you to remote your windowing system over the network and to test new versions of the windowing system in a window. Less remarked on is the fact that if you need 500 system calls to invoke the full functionality of your operating system, most of those calls are going to remain inaccessible to newly ported scripting languages until you write a C extension module for them; and if implementing a fileserver involves handling hundreds of protocol messages, you aren't going to have very many kinds of fileservers. (CIFS, as far as I know, has only two: Windows and Samba.) ioctl and its demonspawn brethren are the reason we didn't have rr in the 1990s, when Michael Elizabeth Chastain wrote mec-replay. - Putting all the system resources into a single namespace means that you can handle them uniformly in some other ways, like setting permissions and interactively exploring the hierarchy. - A lot of system state can be meaningfully treated as data that can be either read or written, with certain useful properties: if you write and then read, then what you read is what you wrote (or is, unless there was an error or a subsequent state change), and if you write back something you read at some previous time, you restore it to the state it had at that time. The framebuffer is one example from Plan 9. The contents of NVRAM are a thing it might be more important to be able to back up and restore in this way. - Also, byte-oriented things don't care what size your reads and writes are; you read the same sequence of bytes whether you read them one at a time or a million at a time. Usually. - Naming things with strings means you can add more things later without breaking backward compatibility. - A slightly richer interface that provides cache invalidation notifications (like inotify) can enable caching proxies and polling proxies, which are sort of dual to each other. Application containers (like Docker, but also like Vesta's build environment, and like rr) can provide isolation, reproducibility, observability, and auditing, but the difficulty of building them is multiplied by the number of different namespaces and system calls they need to interpose on. - It's useful to unify the interprocess communication interface for sending a series of employee records from one process to another with the interface for writing them to a file, a tape, or a terminal, and the interface for receiving them from a file, a tape, or a terminal. script(1) and ttyrec and their kin take advantage of this polymorphic interface to make it possible to replay a terminal session later. Unfortunately as far as I know nobody has implemented a system that lets you record and play back GUI screencasts or mouse-click test scripts in such a simple way. Now, open/close/read/write is really oriented toward the last point more than anything else, treating files as nothing more than recorded output streams that can be replayed. And you lose it if you start reading and writing multiple files! There are other uniform interfaces that have similar benefits for composability; SNMP provides one, REST provides another, and Named Data Networking proposes a third, one which unifies asynchronous notifications with satisfaction of read requests, which is sort of what Unix does too, except that as Benno complains, in Unix you have to bend over backwards to wait for any of multiple events, the polar opposite of the Win16 message loop. (Hilariously, as Benno points out, the Win32 APIs for nonblocking file and socket access are a total mess.) The particular set of restrictions imposed by your chosen architectural style and protocols will determine what kinds of things you can do to your system once you have it running. Surely we can do better than the Unix filesystem interface in 2020.

I'd like to know, from anyone with Plan 9 experience, does its approach to "everything is a file" solve these problems?

pjmlp 7y ago

No, everything is a file doesn't work at all if you care about high performance graphics and real time audio.

kragen 7y ago
qznc made the same claim. Why would "everything is a file" be incompatible with high-performance graphics and real-time audio? I'm particularly interested in this question because my knowledge of those areas is not very deep, so I'm wondering what unknown unknowns I'm missing. Here's my naïve thinking: · Real-time audio requires low-latency scheduling, which is unrelated to the API for audio itself, and FIFO depth monitoring so that you don't get either buffer overruns or buffers that are so full that they induce latency in excess of what your scheduling and processing latency unavoidably adds. For output, you could provide this with a /dev/audio for writing data and a /dev/audiooutputbuffersize for reading the FIFO buffer depth. For input, you could provide this by making reads from /dev/audio nonblocking, by providing a /dev/audioinputbuffersize you would read before risking a blocking read on /dev/audio, or by reading in a separate thread. · High-performance graphics covers a lot of ground, but one of the big pain points is unnecessary memcpying on the paper path from your CPU graphics program into the GPU, whether that's textures, vertex buffers, or prerendered pixel data (though maybe we could argue that that last case is never going to be high performance.) One way that unnecessary memcpying arises is from buffers for write() that are not correctly aligned with respect to the start of a page, which unavoidably means that they will have to get copied to a place with the correct alignment. This is a big problem if you're streaming out gigabytes per second of graphics data. In particular you don't want the data that goes to the GPU concatenated with data that tells the kernel graphics driver what to do with it. Solution: open a new file in a /dev/gfx directory for the data to be sent to the GPU, then write the data to it from a properly aligned buffer. Fork a separate thread to write this data, which is done by marking the pages COW rather than copying them. (A more radical and less sneaky solution I'm exploring for Wercam and BubbleOS is to transfer ownership of pixel data buffers from the application to the window system, causing them to disappear from the application's memory map, avoiding the need to either copy or COW them. But this definitely departs from the Unix read()/write() model.) · A different aspect of high-performance graphics, other than raw throughput, is that you might need to synchronize your drawing with monitor refreshes to avoid an extra half-frame of latency, particularly at low refresh rates like 60 Hz. This seems straightforward to solve in a filesystem interface: reading from /dev/vbi produces a byte just before each VBI, giving you time to send any necessary new data to the GPU. Certainly it's true that Plan 9 does not provide high-performance graphics or low-latency audio. But I don't think that proves that filesystem-based interfaces can't provide them, just that the Plan 9 designers had spent the 1980s researching DSLs, fault-tolerance, concurrency, and typesetting, not rasterization, real-time control, and animation. In general, system-call or IPC interfaces of the form "invoke function foobar with a baz handle and a quux struct or memory buffer" can be cleanly replaced with "open /bazzes/$i/foobar and write a quux struct or memory buffer to it", can't they? I mean, that's three system calls instead of one, so about 1 μs of system-call overhead instead of 300 ns, but in the cases where that's the performance bottleneck you can usually solve it by writing N quuxes instead of one. Can't you? A potential problem arises when you need to combine multiple handles to extra-process resources in a single system call, like SCM_RIGHTS or rename(), but I don't know where those would come up in the particular application domains you identify as problematic. But maybe my proposed solutions above won't solve the problems I think they will, or maybe I'm missing the biggest issues entirely. What am I missing? (I'm not claiming that this approach would be easier, safer, or more discoverable than the approach of adding a bunch of system calls; Benno's talk discusses the many ingenious ways a filesystem-based API can be hard to use, and no doubt some of the same criticisms can be leveled at the above strawman proposals. I'm just saying I don't see where it inherently fails to meet performance requirements.)
- tedunangst 7y ago
  Perhaps getting into no true ship of theseus territory, but you can redefine read/write to do anything but it's arguable if that's still the same philosophy. Like, why even have open()? You could start each process with "/" already open, and then open other files by writing "open /etc/passwd" and reading back a new fd.
  - kragen 7y ago
    Yeah, so, is there a there there? I think there is. The open/close/read/write/getdents interface at least offers the possibility of certain kinds of REST-like uniformity, providing some benefits: - A thin hourglass-waist interface makes it relatively easy to add new components to the system, whether clients, servers, or proxies. Plan 9 implemented both network fileservice and GUI windowing as filesystem-interface proxies, and they composed properly, allowing you to remote your windowing system over the network and to test new versions of the windowing system in a window. Less remarked on is the fact that if you need 500 system calls to invoke the full functionality of your operating system, most of those calls are going to remain inaccessible to newly ported scripting languages until you write a C extension module for them; and if implementing a fileserver involves handling hundreds of protocol messages, you aren't going to have very many kinds of fileservers. (CIFS, as far as I know, has only two: Windows and Samba.) ioctl and its demonspawn brethren are the reason we didn't have rr in the 1990s, when Michael Elizabeth Chastain wrote mec-replay. - Putting all the system resources into a single namespace means that you can handle them uniformly in some other ways, like setting permissions and interactively exploring the hierarchy. - A lot of system state can be meaningfully treated as data that can be either read or written, with certain useful properties: if you write and then read, then what you read is what you wrote (or is, unless there was an error or a subsequent state change), and if you write back something you read at some previous time, you restore it to the state it had at that time. The framebuffer is one example from Plan 9. The contents of NVRAM are a thing it might be more important to be able to back up and restore in this way. - Also, byte-oriented things don't care what size your reads and writes are; you read the same sequence of bytes whether you read them one at a time or a million at a time. Usually. - Naming things with strings means you can add more things later without breaking backward compatibility. - A slightly richer interface that provides cache invalidation notifications (like inotify) can enable caching proxies and polling proxies, which are sort of dual to each other. Application containers (like Docker, but also like Vesta's build environment, and like rr) can provide isolation, reproducibility, observability, and auditing, but the difficulty of building them is multiplied by the number of different namespaces and system calls they need to interpose on. - It's useful to unify the interprocess communication interface for sending a series of employee records from one process to another with the interface for writing them to a file, a tape, or a terminal, and the interface for receiving them from a file, a tape, or a terminal. script(1) and ttyrec and their kin take advantage of this polymorphic interface to make it possible to replay a terminal session later. Unfortunately as far as I know nobody has implemented a system that lets you record and play back GUI screencasts or mouse-click test scripts in such a simple way. Now, open/close/read/write is really oriented toward the last point more than anything else, treating files as nothing more than recorded output streams that can be replayed. And you lose it if you start reading and writing multiple files! There are other uniform interfaces that have similar benefits for composability; SNMP provides one, REST provides another, and Named Data Networking proposes a third, one which unifies asynchronous notifications with satisfaction of read requests, which is sort of what Unix does too, except that as Benno complains, in Unix you have to bend over backwards to wait for any of multiple events, the polar opposite of the Win16 message loop. (Hilariously, as Benno points out, the Win32 APIs for nonblocking file and socket access are a total mess.) The particular set of restrictions imposed by your chosen architectural style and protocols will determine what kinds of things you can do to your system once you have it running. Surely we can do better than the Unix filesystem interface in 2020.