3 ms·

An operating system with an interface that's a substantial improvement over POSIX. I'd like to see statically typed files and pipes, and a built-in file conver

by elihu 10y ago

An operating system with an interface that's a substantial improvement over POSIX. I'd like to see statically typed files and pipes, and a built-in file conversion solver so that if I have, say, a postscript file and I want it in pdf format, I can just tell the OS to use any conversion utility with a matching type signature and I don't have to remember what it's called. I'd like a process/thread to be able to have multiple current-working-directories at the same time, so that a library can change the directory without effecting the rest of the program. Similarly, I'd like processes to be able to operate on behalf of multiple users at the same time. I'd like stricter security policies that deny network access, file system access, etc.. unless they've been explicitly turned on (like Android or IOS). I'd like transactional semantics for filesystem updates. No process should be able to see changes made by another process until that process does a "commit". I'd like to have a general command-line undo. I'd like to be able to do "rm -rf /*" and then undo the operation and have everything be restored. I'd like to have something like proc files, but for user space applications. My full wish list is quite a bit longer, but that's good enough for now.

jpt4 10y ago
For typed, convertible files, global revision control, and possibly the access control enhancements you mention (for the other features ignorance begets silence), Urbit [0] may be of interest. [0] urbit.org
dschiptsov 10y ago
> I'd like to see statically typed files and pipes You mean homogeneous files and pipes - only text, only numbers, etc - that kind of nonsense? Static typing falls apart with heterogeneous containers. Or you are just putting that static typing everywhere because it is so cool - typescript, huh huh?
- elihu 10y ago
  The idea of types is a little different for files than it would be in a programming language. In a sense, all files are trivially of type [Char], but that's not really what I mean. If program A emits XML on stdout, and program B accepts JSON on stdin, "A | B" should be rejected by the shell. A related feature that might be useful is to be able to register checkers or filters for any file type so that if, say, same application creates a pdf file but the data it writes out doesn't look like valid pdf data, it's rejected. Or, if a process tries to write out an mp3 file, the OS converts it to an ogg file to store on disk and converts it back if the application tries to open it later as an mp3.
  - Senji 10y ago
    For media files you would be able to achieve something similar with FUSE and FFMPEG.
  - frou_dh 10y ago
    Would this codify that .xyz filename "extensions" are mandatory.. or would a file's "type" not be stored in-band with its name?
alexdowad 10y ago
elihu, these are some good ideas here. I would love to see your "full wish list". Can you post it somewhere? More improvements on the POSIX interface: - Processes should be able to reserve some % of both CPU and I/O bandwidth. So you can have video players, games, etc. which remain perfectly smooth no matter what other processes are running at the same time. - I would also like it if the kernel and its drivers would refuse to 1) write to the boot sector of any disk, and 2) reflash firmware on any device, even if requested by root, unless the computer is booted in a special "maintenance mode". That would eliminate persistent rootkits.
- prattmic 10y ago
  > - Processes should be able to reserve some % of both CPU and I/O bandwidth. So you can have video players, games, etc. which remain perfectly smooth no matter what other processes are running at the same time. It is far from the nicest interface, but I believe this can be achieved with cgroups in Linux (see `man 7 cgroups` "Cgroups version 1 subsystems"). The cpu subsystem: "Cgroups can be guaranteed a minimum number of "CPU shares" when a system is busy. This does not limit a cgroup's CPU usage if the CPUs are not busy." The blkio subsystem offers "a proportional-weight time-based division of disk [I/O] implemented with CFQ." Though this only guarantees a proportion of I/O, not a specific bandwidth.
- elihu 10y ago
  Okay, I found the list where I wrote these down. Here's some more: - The ability to make changes to /proc files that only persist for the lifetime of the process that made them. (Basically, turn proc files into a stack, for cases where some program requires a particular setting, but don't want to make the change permanent.) - globally managed weak pointers. If multiple processes are managing large amounts of cached data that can be regenerated if it's thrown away, the OS should be in charge of decided when to keep the data and when to thow it away, as it has a better view of the complete system than individual processes. (There's been some work to add this to the Linux kernel, but I don't think it's available to user space. https://lwn.net/Articles/340080/ https://lwn.net/Articles/340080/) - let's get rid of statically-sized partition tables. There's no reason they should be a fixed size on SSDs. (It may be necessary to set a minimum and maximum size, but within those bounds they should just grow and shrink as needed.) - A regular user should be able to create sub-user accounts with a subset of their own permissions. This could be useful to run untrusted applications in a sandbox. - It should be easy to suspend a process to disk, migrate it to another machine, and start it back up, or to kill it and have it restart in the same state it was in when it was killed. - It should be possible to restrict a process from opening any files that weren't passed in as arguments on the command line. (I think this has already been done. I don't remember what the project was called.) - Package files (like rpms or debs) shouldn't have fixed file paths in them, or make any assumptions that the local filesystem will be organized in a particular way. Similarly, package files shouldn't run scripts. Ideally, only the package manager would be able to write to /usr (or whatever the administrator wants to call it on their system). - It may be possible to do away with the user-space / kernel transition, and just run everything in a single address space. The way to do this safely is to write all software in a language that doesn't allow shenanigans (such as Rust) and only permit the OS to execute binaries that have been created by a trusted compiler. If the compiler can verify that types and memory boundaries and API conventions are always respected, then having the hardware check it again at run-time is redundant and an unnecessary drag on performance. For cases where untrusted code blocks are necessary, those would behave like kernel code; you need to be root user to tell the OS it's safe to run such code. Legacy C and C++ applications can run under an x86 emulator or inside a partitioned-off address space, but performant applications would have to be written in a safer language.
borplk 10y ago
My item added to that would be this, "There's a universal dedicated storage mechanism for the state of the operating system and its related packages similar to a SQL database." It really annoys me how ad-hoc state and values are treated in an operating system with ASCII config files. You can always have means of human-friendliness even if the underlying data is more strictly structured such as in a database but you can't go the other way around. Imagine almost like a SQL-like language to provision/configure/restore an operating system. SELECT name FROM installed_packages WHERE installed_date > '2016-01-01 00:00:00'; BEGIN; DROP FROM installed_packages WHERE package_name = "python2"; INSTALL PACKAGE WHERE package_name = "python3"; COMMIT; Operating systems are still stuck with an over-zealous "the unix way or the highway" mentality of cobbling together fragile ad-hoc config files with arbitrary config languages. Each package invents their own config language and forces it down your throat.
- Senji 10y ago
  WinFS tried and failed to do something similar. >Each package invents their own config language and forces it down your throat. Windows Registry.

by elihu 10y ago

jpt4 10y ago

For typed, convertible files, global revision control, and possibly the access control enhancements you mention (for the other features ignorance begets silence), Urbit [0] may be of interest. [0] urbit.org

dschiptsov 10y ago

> I'd like to see statically typed files and pipes You mean homogeneous files and pipes - only text, only numbers, etc - that kind of nonsense? Static typing falls apart with heterogeneous containers. Or you are just putting that static typing everywhere because it is so cool - typescript, huh huh?

elihu 10y ago
The idea of types is a little different for files than it would be in a programming language. In a sense, all files are trivially of type [Char], but that's not really what I mean. If program A emits XML on stdout, and program B accepts JSON on stdin, "A | B" should be rejected by the shell. A related feature that might be useful is to be able to register checkers or filters for any file type so that if, say, same application creates a pdf file but the data it writes out doesn't look like valid pdf data, it's rejected. Or, if a process tries to write out an mp3 file, the OS converts it to an ogg file to store on disk and converts it back if the application tries to open it later as an mp3.
- Senji 10y ago
  For media files you would be able to achieve something similar with FUSE and FFMPEG.
- frou_dh 10y ago
  Would this codify that .xyz filename "extensions" are mandatory.. or would a file's "type" not be stored in-band with its name?

alexdowad 10y ago

elihu, these are some good ideas here. I would love to see your "full wish list". Can you post it somewhere? More improvements on the POSIX interface: - Processes should be able to reserve some % of both CPU and I/O bandwidth. So you can have video players, games, etc. which remain perfectly smooth no matter what other processes are running at the same time. - I would also like it if the kernel and its drivers would refuse to 1) write to the boot sector of any disk, and 2) reflash firmware on any device, even if requested by root, unless the computer is booted in a special "maintenance mode". That would eliminate persistent rootkits.

prattmic 10y ago
> - Processes should be able to reserve some % of both CPU and I/O bandwidth. So you can have video players, games, etc. which remain perfectly smooth no matter what other processes are running at the same time. It is far from the nicest interface, but I believe this can be achieved with cgroups in Linux (see `man 7 cgroups` "Cgroups version 1 subsystems"). The cpu subsystem: "Cgroups can be guaranteed a minimum number of "CPU shares" when a system is busy. This does not limit a cgroup's CPU usage if the CPUs are not busy." The blkio subsystem offers "a proportional-weight time-based division of disk [I/O] implemented with CFQ." Though this only guarantees a proportion of I/O, not a specific bandwidth.
elihu 10y ago
Okay, I found the list where I wrote these down. Here's some more: - The ability to make changes to /proc files that only persist for the lifetime of the process that made them. (Basically, turn proc files into a stack, for cases where some program requires a particular setting, but don't want to make the change permanent.) - globally managed weak pointers. If multiple processes are managing large amounts of cached data that can be regenerated if it's thrown away, the OS should be in charge of decided when to keep the data and when to thow it away, as it has a better view of the complete system than individual processes. (There's been some work to add this to the Linux kernel, but I don't think it's available to user space. https://lwn.net/Articles/340080/ https://lwn.net/Articles/340080/) - let's get rid of statically-sized partition tables. There's no reason they should be a fixed size on SSDs. (It may be necessary to set a minimum and maximum size, but within those bounds they should just grow and shrink as needed.) - A regular user should be able to create sub-user accounts with a subset of their own permissions. This could be useful to run untrusted applications in a sandbox. - It should be easy to suspend a process to disk, migrate it to another machine, and start it back up, or to kill it and have it restart in the same state it was in when it was killed. - It should be possible to restrict a process from opening any files that weren't passed in as arguments on the command line. (I think this has already been done. I don't remember what the project was called.) - Package files (like rpms or debs) shouldn't have fixed file paths in them, or make any assumptions that the local filesystem will be organized in a particular way. Similarly, package files shouldn't run scripts. Ideally, only the package manager would be able to write to /usr (or whatever the administrator wants to call it on their system). - It may be possible to do away with the user-space / kernel transition, and just run everything in a single address space. The way to do this safely is to write all software in a language that doesn't allow shenanigans (such as Rust) and only permit the OS to execute binaries that have been created by a trusted compiler. If the compiler can verify that types and memory boundaries and API conventions are always respected, then having the hardware check it again at run-time is redundant and an unnecessary drag on performance. For cases where untrusted code blocks are necessary, those would behave like kernel code; you need to be root user to tell the OS it's safe to run such code. Legacy C and C++ applications can run under an x86 emulator or inside a partitioned-off address space, but performant applications would have to be written in a safer language.

borplk 10y ago

My item added to that would be this, "There's a universal dedicated storage mechanism for the state of the operating system and its related packages similar to a SQL database." It really annoys me how ad-hoc state and values are treated in an operating system with ASCII config files. You can always have means of human-friendliness even if the underlying data is more strictly structured such as in a database but you can't go the other way around. Imagine almost like a SQL-like language to provision/configure/restore an operating system. SELECT name FROM installed_packages WHERE installed_date > '2016-01-01 00:00:00'; BEGIN; DROP FROM installed_packages WHERE package_name = "python2"; INSTALL PACKAGE WHERE package_name = "python3"; COMMIT; Operating systems are still stuck with an over-zealous "the unix way or the highway" mentality of cobbling together fragile ad-hoc config files with arbitrary config languages. Each package invents their own config language and forces it down your throat.

Senji 10y ago
WinFS tried and failed to do something similar. >Each package invents their own config language and forces it down your throat. Windows Registry.