6 ms·

> First, it's not always easy to map every object operation into either an open read or write. Heck, even for basic files on disk, and even more so for sockets

• xiphias2 3y ago
  io_uring is a more modern API to file access. Actually I think it would be great if everything was a file and the communication with the kernel was only with io_uring.
  • rcxdude 3y ago
    it's a more modern API to system calls in general, though most of the focus is on IO.
  • codedokode 3y ago
    > it would be great if everything was a file This is a bad design. Process is not a file because you cannot send signals to a file, or cannot debug a file. Network socket is not a file because you cannot get file's peer address. Shared memory is not a file. And so on.
    • deleted 3y ago
      [deleted]
    • c00lio 3y ago
      Why would (pseudocode, nonexisting but possible example) write(SIGKILL, "/proc/12345/signals") not be possible? For the other direction, there is signalfd in Linux. And of course you can get a peer address from /dev/tcp (https://andreafortuna.org/2021/03/06/some-useful-tips-about-dev-tcp/ https://andreafortuna.org/2021/03/06/some-useful-tips-about-... ). Yes, /dev/tcp is not an OS primitive but a bash builtin, but there isn't really a reason you cannot do this in the OS. Shared memory can be a file, you just have to mmap() it. mmap is always left out of the open/close/read/write-enumeration of the traditional file API, but I think it is actually extremely useful and should be the fifth alongside those. Debugging is hard to imagine, yes. One could look at /proc/12345/mem, write breakpoints in there, but I'm not sure about how to do the more exotic things.
      • samus 3y ago
        This means layering additional protocols on top of the file APIs. Of course you can do that, but eventually there will be a similar explosion such as on top of `ioctl`, and it's doubtful whether the resulting interfaces will be any easier to use than the existing ones.
        • c00lio 3y ago
          On the other hand, layering tons of stuff on unsuitable interfaces has been all the rage in the last 25 years. Just think of everything-over-http, stuff-it-in-xml/json, program-it-in-yaml and similar industry trends. ;)
          • civilitty 3y ago
            I can’t wait till we’re configuring our OS using CSS-in-JS and booting using webpack. npm install -g styled-linux
            • c00lio 3y ago
              As always, Fabrice Bellard is ahead of his time: https://bellard.org/jslinux/ https://bellard.org/jslinux/ :)
        • ori_b 3y ago
          Plan 9 does this. The resulting interfaces are indeed easier to use, forward over the network, and redirect or emulate for testing.
        • butterisgood 3y ago
          Exactly? The alternative is ioctl whereas Plan 9 does everything over serialized streams to files. The serialized streams make it easier to think of these ioctl-alikes as something you can easily access over a network (9p). And that’s how resource sharing is done in plan 9.
        • TheRealPomax 3y ago
          Let me introduce you to the Linux Audio Stack. Laying protocols on top of each other, when everything is just files, is no different from layering protocols on top of each other when some things are files and some thing aren't. The question isn't "will it be just as bad" but "would some things become easier, and if so, how many things would become easier?". Because if the latter, then it's a worthwhile topic to think about even if we never use it in any commercial sense of of the word.
    • 3cats-in-a-coat 3y ago
      When everything is a file, what is a "file" becomes flexible. In Plan 9, you send signals and messages to a file by writing to it. And read messages by reading from it. It's in essence no different than OOP or actor model or what have you.
      • SubjectToChange 3y ago
        When everything is a file, what is a "file" becomes flexible. It’s really “Everything has a File Descriptor”, nothing about the concept of a file has changed. It's in essence no different than OOP or actor model or what have you. I’m sure there’s an isomorphism that could be drawn but it does nothing to show that it’s an equally good paradigm to write software in. IMO, it’s a tortured abstraction.
        • 3cats-in-a-coat 3y ago
          I wouldn't sit here and claim Plan 9 was perfect, because if it was, we'd be using it. In particular, the issue is that you're reading raw stream of content on the way in and out of those file descriptors. This is akin to how shell piping in Unix is also just... text and bytes. This is limiting and produces many ad-hoc protocols. But take what Plan 9 was trying to do, and add to it what Microsoft's PowerShell tried to do, where you stream objects, structured information, instead of just bytes, on the way in and out of commands and files... And suddenly... we got ourselves an Erlang.
          • thesuperbigfrog 3y ago
            >> I wouldn't sit here and claim Plan 9 was perfect, because if it was, we'd be using it. Many perfect and awesome systems have been built that never saw significant adoption. I agree with ESR's observation: "Plan 9 failed simply because it fell short of being a compelling enough improvement on Unix to displace its ancestor. Compared to Plan 9, Unix creaks and clanks and has obvious rust spots, but it gets the job done well enough to hold its position. There is a lesson here for ambitious system architects: the most dangerous enemy of a better solution is an existing codebase that is just good enough." Source: https://www.catb.org/~esr/writings/taoup/html/plan9.html https://www.catb.org/~esr/writings/taoup/html/plan9.html Another amazing OS that never caught on was BeOS: https://en.wikipedia.org/wiki/BeOS https://en.wikipedia.org/wiki/BeOS Neal Stephenson's description of BeOS as "fully operational Batmobiles" is accurate and it is sad that BeOS did not become more mainstream. See https://people.cs.georgetown.edu/~clay/classes/spring2010/os/inthebeginning.pdf https://people.cs.georgetown.edu/~clay/classes/spring2010/os...
    • giamma 3y ago
      I am probably looking at it from a too high-level perspective, but the RESTful paradigm was widely adopted in every domain and proved that you can model pretty much any concept as resources, with CRUD primitives and hyperlinks between them. Wouldn't the same be applicable to files, processes, devices and so on?
      • 3cats-in-a-coat 3y ago
        It is. I don't think people are well familiar with what Plan 9 calls "files". They're objects or resources, which also happen to be files. REST/OOP/Actors/Plan9 are very similar systems. And like it or not OOP shows that it's possible for one idiom to describe all the things when it's flexible enough.
        • miga 3y ago
          These are called "special files" in Linux. "Device files" are just a special case. Both "special" and "device" files are still "files".
        • throwway120385 3y ago
          Yeah, it's not such a laughable idea when you consider just how much of the dynamic behavior of a software system can be modeled as a series of messages between independent resources. This is part of why UML was so pervasive. People had spent intense amounts of energy modeling systems using messaging and interconnection, and UML gave us a uniform way of doing this. I've got a lot of experience working in systems that model everything as a series of resources passing messages and it works very well. The entire QNX operating system right down to its POSIX support uses this underlying primitive and, while you would never see it in your own code their system-wide profiler leans on this design to make it easy to see how control flow moves between isolated threads and processes within the software system.
    • arghwhat 3y ago
      We now have pidfd, because we actually need process handles and treating processes as files is the best way to do so.
    • osigurdson 3y ago
      Maybe instead of saying everything is a file, perhaps it should be "all resources are organized in a tree structure". The nodes of the tree can be different things where different operations apply.
    • arghwhat 3y ago
      To elaborate on those points a bit further, past what I already said about pidfd being introduced exactly to treat processes as files: 1. ioctls can make any "syscall" on a file. 2. a process does not have to be a singular file. All processes have most if not all their attributes exposed as files /proc/$PID/ as files, and can have this arbitrarily extended. In plan9, passing a signal (technically a note) is done by writing to /proc/$PID/note, and there is no technical reason for not allowing the same on Linux. 3. the entire concept of memory mapping is based around files, with anonymous memory - i.e., non-disk-backed memory - just being a subset of this. POSIX shared memory (shm_open) is provided through /dev/shm, which is a tmpfs folder and is indeed just files. 4. sockets are file descriptors, and file descriptors is what makes a file, and as such you can get a peers address of a file descriptor when such is present. Ways to expose creating sockets in the filesystem also exist, and not just for plan. The special socket-bits could easily be made less special, with the only justification for the current BSD socket API being that it became dominant and so everyone copied it.
      • mike_hock 3y ago
        This is an absurd interpretation of "everything is a file." The kind that makes you go "arghwhat?!" A file descriptor is exactly nothing like a file. You cannot write to a pidfd. You cannot waitid an eventfd. You cannot getsockopt on a regular file. The only operations that all file descriptors have in common is close, dup, poll and some other basic operations. So "file descriptor" basically just means "kernel interface object" and the available operations depend on the type of object. A file is a container for arbitrary data that I can read from, write to, and reposition the read/write cursor in. If you call anything else a "file" then you haven't made everything a file, you've just redefined "file" to mean "thing." What business does a socket have in a physical, on-disk filesystem? (Let alone a clunky hack to invoke the much simpler `signal` syscall in a roundabout way?) The socket "file" is completely meaningless unless the process that opened it is currently alive and still listening on it. So why the fuck should it get written to a persistent storage device? How do I specify the socket type, which is a meaningless concept for an actual file, when I open a socket "file"? Oh that's right, I don't. Because I don't open a socket. I bind or connect. I don't use "file" APIs because they're not applicable. I use a dedicated socket API that's fit for the purpose. Pidfd was not introduced to treat processes as "files," it was introduced so they could share the operations that they do meaningfully share with other kernel objects (e.g. poll). The overloaded ioctl syscall is bad design. The proc "filesystem" is bad design. /dev/shm is ridiculous design. So I have to mock a fake filesystem in memory so I can create a fake file in that "filesystem" just so I can get the same memory pages mapped into my virtual address space as some other process, all of which has absolutely nothing to do with files or a filesystem (and is much lower level than that). lolwat?
    • GoblinSlayer 3y ago
      >Process is not a file because you cannot send signals to a file https://man7.org/linux/man-pages/man2/pidfd_send_signal.2.html https://man7.org/linux/man-pages/man2/pidfd_send_signal.2.ht... Also sockets have the concept of ancillary channels used to deliver out of band data (see recvmsg). If every system resource had such channel, it could be used for control similar to how sockets do it.
  • kps 3y ago
    > io_uring is a more modern API to file access. With a remarkable similarity to certain mainframe systems from half a century ago.
• mananaysiempre 3y ago
  > Ultimately IO is [an] intrinsically complex topic, and trying to paper over that complexity with simple interfaces is disingenuous and falls flat on edge cases. I’m not against pointing out the edge cases in the Plan 9 file model[1,2], but the thing is, I haven’t seen complex I/O interfaces that aren’t a horror show, either. Granted, I haven’t seen that many of those at all, but I’ve had a thorough look at the ones in OS/2, Win32, and NT, and none of them seem particularly inspiring. I’d very much like to see some nice alternatives, to be clear! The reference to io_uring also doesn’t seem all that strong of an argument, honestly. I’d like to say there are three layers to the idea of “traditional” “Unix” “files” as an OS (not storage) interface: - System and user resources you have access to are identified by unforgeable references (called “fds”; the merits of allowing userspace to control their naming as opposed to having the kernel assign the names are debatable). You can feed bytes into these, (ask to) get bytes out of them, and perhaps have a out-of-band call to e.g. transmit one of the other references you hold to a peer. You can of course also delete a reference. So far this is just dynamically typed object-capabilities by another name. It’s going to require higher-level protocols on top, but so does basically everything else on this level of generality. Plan 9 mostly (if not completely) eliminated ioctls here by using separate control files instead. (The part where the OS merges the payloads of write calls into a byte stream then cuts it back up into reads is more opinionated, but I don’t think even Bell Labs systems ever adhered to that principle strictly[1].) - You obtain (most of) these references by navigating a stringy hierarchical namespace. There are additional calls to do so and to modify that namespace. This is less of an obviously correct least common denominator, and as history shows the consensus is less strong here as well. Mountpoints, symlinks, namespaces per TFA, even the *at() calls all change how this part functions. On the other hand, I don’t think anybody longs for version numbers or nesting limits (or even drive letters) of other systems that have used naming approaches similar enough for a comparison. - You access these services through synchronous system calls write(), read(), ioctl(), close(), open(), etc. This is the part that is changed by the introduction by io_uring... But I don’t feel it’s all that important for the conceptual model, unlike the preceding points. [1] Cutting a bytestream into packets is as always a tedious slog of buffering so some of the protocol implementations use write() / read() boundaries thus actually (depend on being able to) use the API in a datagram-like fashion (which 9P enables IIUC). [2] Auth is based on a /proc/self-like hack wherein the kernel-side implementation of the “file” inspects the opening process through kernel-side knowledge you can’t access nor proxy from userspace.