14 ms·
> it would be great if everything was a file This is a bad design. Process is not a file because you cannot send signals to a file, or cannot debug a file. Ne
by codedokode 3y ago
> it would be great if everything was a file
This is a bad design. Process is not a file because you cannot send signals to a file, or cannot debug a file. Network socket is not a file because you cannot get file's peer address. Shared memory is not a file. And so on.
- deleted 3y ago[deleted]
- c00lio 3y agoWhy would (pseudocode, nonexisting but possible example) write(SIGKILL, "/proc/12345/signals") not be possible? For the other direction, there is signalfd in Linux. And of course you can get a peer address from /dev/tcp (https://andreafortuna.org/2021/03/06/some-useful-tips-about-dev-tcp/ https://andreafortuna.org/2021/03/06/some-useful-tips-about-... ). Yes, /dev/tcp is not an OS primitive but a bash builtin, but there isn't really a reason you cannot do this in the OS. Shared memory can be a file, you just have to mmap() it. mmap is always left out of the open/close/read/write-enumeration of the traditional file API, but I think it is actually extremely useful and should be the fifth alongside those. Debugging is hard to imagine, yes. One could look at /proc/12345/mem, write breakpoints in there, but I'm not sure about how to do the more exotic things.
- samus 3y agoThis means layering additional protocols on top of the file APIs. Of course you can do that, but eventually there will be a similar explosion such as on top of `ioctl`, and it's doubtful whether the resulting interfaces will be any easier to use than the existing ones.
- c00lio 3y agoOn the other hand, layering tons of stuff on unsuitable interfaces has been all the rage in the last 25 years. Just think of everything-over-http, stuff-it-in-xml/json, program-it-in-yaml and similar industry trends. ;)
- ori_b 3y agoPlan 9 does this. The resulting interfaces are indeed easier to use, forward over the network, and redirect or emulate for testing.
- butterisgood 3y agoExactly? The alternative is ioctl whereas Plan 9 does everything over serialized streams to files. The serialized streams make it easier to think of these ioctl-alikes as something you can easily access over a network (9p). And that’s how resource sharing is done in plan 9.
- TheRealPomax 3y agoLet me introduce you to the Linux Audio Stack. Laying protocols on top of each other, when everything is just files, is no different from layering protocols on top of each other when some things are files and some thing aren't. The question isn't "will it be just as bad" but "would some things become easier, and if so, how many things would become easier?". Because if the latter, then it's a worthwhile topic to think about even if we never use it in any commercial sense of of the word.
- samus 3y agoThe main problem with Linux audio is the bloom of APIs that appeared over the years. While a file-based interface would be cool, it would be just yet another API that competes with all the others for its place in the ecosystem. Fortunately, ALSA and PulseAudio seem to dominate right now.
- adql 3y agoFor one, encoding and decoding text is slower than binary calls to a function with solid parameters that don't need to be converted. It's far too easy to pretend reality is not complex and that "elegant" solution somehow will fit everything
- c00lio 3y agoNetlink sockets in Linux input and output packed C structures. Doesn't get more efficient than that. Such an interface definitely doesn't have to be strings-only. But of course, you cannot use it with just 'echo' in that case.
- codedokode 3y agoWhy not make real syscalls rather than emulate them using socket-like interface? Doesn't make much sense to me. For example, if you want to filter syscalls then it becomes more difficult (need to remember which type of socket it is, need to parse the structures and so on).
- c00lio 3y agoThe original idea for netlink sockets (where the name comes from) is to be able to do some network packet processing in userspace. E.g. to do stuff like virus-scanning on TCP connections. The situation there is exactly the opposite of a syscall, it is rather that the kernel calls into userspace to perform a helper function. Communication with the socket is still read()/write()/..., so there are still syscalls. The userspace program will do a read() to get the next struct+packet out of the socket. The modern, syscall-less interface for stuff would be io_uring. There, you do not need to read(), you can just get your data written into a userspace buffer that you can mwait or poll on.
- markhahn 3y ago/proc pseudofiles could have a mode (perhaps on open) that determines whether the protocol is ascii or packed/binary. there could even be a side-band interface that provides the packed schema (assuming not everything is exploded to atomic type-evident items).
- marcosdumay 3y ago> write(SIGKILL, "/proc/12345/signals") Outside of lots of non-obvious problems with synchronization, this is your example that best fits the idea. This interface is probably a good one. > And of course you can get a peer address from /dev/tcp You will have lots and lots of problems with access controls if this is your only interface. > Shared memory can be a file Coercing random access memory into a serial file just to go and emulate a random access over that file is... not a great way to deal with a high-performance primitive.
- codedokode 3y ago> This interface is probably a good one. This interface is a bad one because if you want to filter syscalls then it will be difficult to distinguish write to a file from sending a signal.
- miga 3y agoYou will filter syscalls by filename anyway. Try `strace|grep open` on any program, and you will be spammed with a number of shared libraries. You need to filter them out anyway.
- c00lio 3y ago> Coercing random access memory into a serial file just to go and emulate a random access over that file is... not a great way to deal with a high-performance primitive. The file doesn't need to have an on-disk representation. I don't see why an mmap-ed file should behave any different from a SHM segment. They are basically the same thing, the SHM segment even has a file descriptor. It just doesn't have a name somewhere in the filesystem hierarchy. https://man7.org/linux/man-pages/man7/shm_overview.7.html https://man7.org/linux/man-pages/man7/shm_overview.7.html
- ori_b 3y agoIt can have a name in the file system: /dev/shm
- marcosdumay 3y ago
- afr0ck 3y agommap() is very different in its semantics from read() and write(). The latters have a clear, precise definition, a byte stream received by your file driver. mmap() is a different beast, it can be a ring buffer, it can a spinlock, it can be anything. The moment you have mmap(), the "eveything is a file" model is already broken and the namespace API can longer work reliably over the network. How would you transport an mmap() over the network? if /dev/tcp in plan9 uses mmap to asynchronously write the packets, you can no longer rely on the 9p filesystem protocol to implement a proxy by mounting the remote computer's /dev/tcp, because there is no clear and reliable way to map remote memory. It's no more about transporting a random sequence of bytes to another computer. You need more than that to make it work. It's just one example where this "everything is a fike" model is overly simple for the harsh real world.
- 3cats-in-a-coat 3y agoWhen everything is a file, what is a "file" becomes flexible. In Plan 9, you send signals and messages to a file by writing to it. And read messages by reading from it. It's in essence no different than OOP or actor model or what have you.
- SubjectToChange 3y agoWhen everything is a file, what is a "file" becomes flexible. It’s really “Everything has a File Descriptor”, nothing about the concept of a file has changed. It's in essence no different than OOP or actor model or what have you. I’m sure there’s an isomorphism that could be drawn but it does nothing to show that it’s an equally good paradigm to write software in. IMO, it’s a tortured abstraction.
- 3cats-in-a-coat 3y agoI wouldn't sit here and claim Plan 9 was perfect, because if it was, we'd be using it. In particular, the issue is that you're reading raw stream of content on the way in and out of those file descriptors. This is akin to how shell piping in Unix is also just... text and bytes. This is limiting and produces many ad-hoc protocols. But take what Plan 9 was trying to do, and add to it what Microsoft's PowerShell tried to do, where you stream objects, structured information, instead of just bytes, on the way in and out of commands and files... And suddenly... we got ourselves an Erlang.
- thesuperbigfrog 3y ago>> I wouldn't sit here and claim Plan 9 was perfect, because if it was, we'd be using it. Many perfect and awesome systems have been built that never saw significant adoption. I agree with ESR's observation: "Plan 9 failed simply because it fell short of being a compelling enough improvement on Unix to displace its ancestor. Compared to Plan 9, Unix creaks and clanks and has obvious rust spots, but it gets the job done well enough to hold its position. There is a lesson here for ambitious system architects: the most dangerous enemy of a better solution is an existing codebase that is just good enough." Source: https://www.catb.org/~esr/writings/taoup/html/plan9.html https://www.catb.org/~esr/writings/taoup/html/plan9.html Another amazing OS that never caught on was BeOS: https://en.wikipedia.org/wiki/BeOS https://en.wikipedia.org/wiki/BeOS Neal Stephenson's description of BeOS as "fully operational Batmobiles" is accurate and it is sad that BeOS did not become more mainstream. See https://people.cs.georgetown.edu/~clay/classes/spring2010/os/inthebeginning.pdf https://people.cs.georgetown.edu/~clay/classes/spring2010/os...
- giamma 3y agoI am probably looking at it from a too high-level perspective, but the RESTful paradigm was widely adopted in every domain and proved that you can model pretty much any concept as resources, with CRUD primitives and hyperlinks between them. Wouldn't the same be applicable to files, processes, devices and so on?
- 3cats-in-a-coat 3y agoIt is. I don't think people are well familiar with what Plan 9 calls "files". They're objects or resources, which also happen to be files. REST/OOP/Actors/Plan9 are very similar systems. And like it or not OOP shows that it's possible for one idiom to describe all the things when it's flexible enough.
- miga 3y agoThese are called "special files" in Linux. "Device files" are just a special case. Both "special" and "device" files are still "files".
- throwway120385 3y agoYeah, it's not such a laughable idea when you consider just how much of the dynamic behavior of a software system can be modeled as a series of messages between independent resources. This is part of why UML was so pervasive. People had spent intense amounts of energy modeling systems using messaging and interconnection, and UML gave us a uniform way of doing this. I've got a lot of experience working in systems that model everything as a series of resources passing messages and it works very well. The entire QNX operating system right down to its POSIX support uses this underlying primitive and, while you would never see it in your own code their system-wide profiler leans on this design to make it easy to see how control flow moves between isolated threads and processes within the software system.
- arghwhat 3y agoWe now have pidfd, because we actually need process handles and treating processes as files is the best way to do so.
- osigurdson 3y agoMaybe instead of saying everything is a file, perhaps it should be "all resources are organized in a tree structure". The nodes of the tree can be different things where different operations apply.
- arghwhat 3y agoTo elaborate on those points a bit further, past what I already said about pidfd being introduced exactly to treat processes as files: 1. ioctls can make any "syscall" on a file. 2. a process does not have to be a singular file. All processes have most if not all their attributes exposed as files /proc/$PID/ as files, and can have this arbitrarily extended. In plan9, passing a signal (technically a note) is done by writing to /proc/$PID/note, and there is no technical reason for not allowing the same on Linux. 3. the entire concept of memory mapping is based around files, with anonymous memory - i.e., non-disk-backed memory - just being a subset of this. POSIX shared memory (shm_open) is provided through /dev/shm, which is a tmpfs folder and is indeed just files. 4. sockets are file descriptors, and file descriptors is what makes a file, and as such you can get a peers address of a file descriptor when such is present. Ways to expose creating sockets in the filesystem also exist, and not just for plan. The special socket-bits could easily be made less special, with the only justification for the current BSD socket API being that it became dominant and so everyone copied it.
- mike_hock 3y agoThis is an absurd interpretation of "everything is a file." The kind that makes you go "arghwhat?!" A file descriptor is exactly nothing like a file. You cannot write to a pidfd. You cannot waitid an eventfd. You cannot getsockopt on a regular file. The only operations that all file descriptors have in common is close, dup, poll and some other basic operations. So "file descriptor" basically just means "kernel interface object" and the available operations depend on the type of object. A file is a container for arbitrary data that I can read from, write to, and reposition the read/write cursor in. If you call anything else a "file" then you haven't made everything a file, you've just redefined "file" to mean "thing." What business does a socket have in a physical, on-disk filesystem? (Let alone a clunky hack to invoke the much simpler `signal` syscall in a roundabout way?) The socket "file" is completely meaningless unless the process that opened it is currently alive and still listening on it. So why the fuck should it get written to a persistent storage device? How do I specify the socket type, which is a meaningless concept for an actual file, when I open a socket "file"? Oh that's right, I don't. Because I don't open a socket. I bind or connect. I don't use "file" APIs because they're not applicable. I use a dedicated socket API that's fit for the purpose. Pidfd was not introduced to treat processes as "files," it was introduced so they could share the operations that they do meaningfully share with other kernel objects (e.g. poll). The overloaded ioctl syscall is bad design. The proc "filesystem" is bad design. /dev/shm is ridiculous design. So I have to mock a fake filesystem in memory so I can create a fake file in that "filesystem" just so I can get the same memory pages mapped into my virtual address space as some other process, all of which has absolutely nothing to do with files or a filesystem (and is much lower level than that). lolwat?
- GoblinSlayer 3y ago>Process is not a file because you cannot send signals to a file https://man7.org/linux/man-pages/man2/pidfd_send_signal.2.html https://man7.org/linux/man-pages/man2/pidfd_send_signal.2.ht... Also sockets have the concept of ancillary channels used to deliver out of band data (see recvmsg). If every system resource had such channel, it could be used for control similar to how sockets do it.