7 ms·

Every piece of hardware on a computer is a piece of global mutable state. There are many good reasons to hide that fact behind an abstraction, but I can't help

Every piece of hardware on a computer is a piece of global mutable state. There are many good reasons to hide that fact behind an abstraction, but I can't help but think hiding global mutability is better handled at the application level than the OS level, because there are too many cases where the abstraction becomes extremely limiting. As an example, most people would want to be able to import an image into a word processor regardless of where that image is located (local drive, network drive, floppy disk, etc.). To support that, most end user programs would want to be offered access to the entire filesystem. The moment two applications do this at once, you have all the shared mutable state problems we do now.

satyenr 7y ago
It doesn't have to be that way. Consider the following scenario: 1. Every file is owned by application. 2. A file-store application is the custodian of files meant to be used by multiple applications. 3. An application that needs to edit a file must take ownership of the file for the duration of the editing. 4. An application that needs to read a file must borrow the files. 5. A file can be borrowed by multiple applications, but owned by only one. 6. Applications can provide the ability for shared ownership, but those would by specialized applications capable of handling and merging simultaneous changes. This is a straw-man solution, and I am sure multiple problems will have to be solved before such a system can become a reality. > As an example, most people would want to be able to import an image into a word processor regardless of where that image is located (local drive, network drive, floppy disk, etc.). To support that, most end user programs would want to be offered access to the entire filesystem. The moment two applications do this at once, you have all the shared mutable state problems we do now. I don't see why the custodian of photos can't be a photo management application. In fact, a filesystem is the lowest common denominator. It is possible to build higher level abstractions like PhotoStore, MusicStore, MovieStore, CodeStore etc. which accommodate and make use of the properties of individual data types to offer an enhanced experience.
- Retra 7y ago
  I have photos on my computer, even though I don't use a photo management application. I have compressed files. I have files taken from other machines. I remove applications. I create my own file formats. Applications themselves are files. Who owns all the unassigned stuff? The file system. Give it a different name and you change nothing.
  - satyenr 7y ago
    You don't use a photo management application because the file-system exposes photos as files. It is like saying -- in java parlance -- I am perfectly fine using object for everything -- I don't need any other data type. In the hypothetical system I proposed, the photo store would be exposed via an application like Explorer/Finder -- just tailored for viewing photos. Compressed files are containers for files -- they are nothing on their own. > Applications themselves are files. Sure -- they go into ApplicationStore -- which tailored for say, quick application discovery and launch. > Who owns all the unassigned stuff? T What exactly is the unassigned stuff? Files taken from other machines? I don't see why they can't be put into an appropriate store based on type. > I create my own file formats. I assume you create your own file formats because you develop your own applications? In that case, your application will be the owner of the file in question.
    - michaelmrose 7y ago
      What if I don't desire a gui application store or desire a different one or switch application stores or move a dir with executable files from one machine to another? Is the application store the only way to start at app? If I bind a global hotkey to start an app how is that handled? How about I have an addon for firefox that allows me to bind keys to operations which can include javascript which can itself start applications which can then write to files. I use this for example to open links in mpv with a singular line in my config file for the addon, tridactyl, bind V hint -W ! mpv this means show a hint on links and for the given link send the chosen link to mpv, wherein mpv will look at the link and if it can read it will use youtube-dl to download the link to a temp dir and display the video. Who owns what there? Do I have to access my photos via a a singular photo manager? I can imagine images accessed in 17 different contexts by 30 different apps does each of them need permission to access each dir which contains images or just generically to access images. Does this mean that a singular permission would control both access to the browsers cached image of the ycombinator logo on this page and some ones nude selfies? I just don't think you can cleanly map apps -> files beyond the trivial cases without making something inflexible that sucks to use.
- empath75 7y ago
  Sounds like rust’s borrow checker but for files.
  - satyenr 7y ago
    That is where I got the idea. :-)
- pixl97 7y ago
  2 just sounds like reinventing the file system, now with more bugs.
  - satyenr 7y ago
    No, it is not reinventing the filesystem -- it's more like a file-vault or a file-bank. Anyone looking to operate on the files must either borrow (to read) or take ownership of (to write). The difference is that while a filesystem unlimited access to all files, the app in question is meant to be more sandboxed -- handling only the files it is asked to -- e.g. shared documents that can be edited by multiple apps. Each user (on a multi-user system) will have their file-vault/file-bank and files must be explicitly transferred for shared access. Finally, as I said, it is a straw-man proposal -- I haven't spent time working out all the kinks.
    - zrm 7y ago
      > No, it is not reinventing the filesystem -- it's more like a file-vault or a file-bank. Anyone looking to operate on the files must either borrow (to read) or take ownership of (to write). So flock() and file permissions then? There are improvements that could be made here -- app-level permissions in addition to user-level permissions for example. But it's still fundamentally a filesystem.
      - satyenr 7y ago
        All of this can be implemented on top of a file system, of course. What do you think iPadOS does? We are talking about what the user/application sees and is capable of accessing.
        zrm 7y ago
        > We are talking about what the user/application sees and is capable of accessing. But how is that different than a filesystem? Suppose we add application-based ACLs to file permissions. Then the app does open("/path/to/file", O_RDONLY) as ever. If the app has permission to the file, it gets the new fd. If it doesn't, it gets EACCES as usual. Or the OS displays a dialog asking whether the app should have permanent or one-time access to that file, and then the call doesn't return until the user chooses one. I don't see a fundamental change here. The application wouldn't necessarily even have to be modified.
- amelius 7y ago
  Instead of "ownership transfer" you can also use locks. Read and write locks are already supported on many filesystems.
  - satyenr 7y ago
    Well, you can also use global variables safely with locks.
- auggierose 7y ago
  How much is that different from how it works on iPadOS?
  - satyenr 7y ago
    It is meant to be more powerful and flexible than what iPad OS does, but somewhat similar.
- heavenlyblue 7y ago
  Your example would fail to accommodate the most basic application of files: logging and reading file logs at the same time.
  - satyenr 7y ago
    Who says logs have to be written to a file — at least as far as the user is concerned? Logs are a series of events — a LogStore will let you read events as they are appended. And even if you do want to store logs in “files”, the solution proposed here here let’s you borrow them for the owning application.
    - mchristen 7y ago
      Where do you plan to store the logs then?
      - satyenr 7y ago
        On the disk, as usual. Files are, after all, an interface.
    - heavenlyblue 7y ago
      My question goes more like this: “why the hell do you want to re-approach the idea of a filesystem if the new approach can’t even accommodate something as simple as storing log files?”. It may be time for something completely different. Just stop calling it a filesystem.
      - satyenr 7y ago
        Again, why do you care about files? Logs have nothing to do with files. Files are just the interface used by filesystem to show the event stream that is logs so you can use file based tools to access the. I am not calling it a filesystem. Let’s call it a data bank?
        heavenlyblue 7y ago
- deleted 7y ago
  [deleted]
- LoSboccacc 7y ago
  basically the windows document folder
  - satyenr 7y ago
    Except, you won’t see files, just photos, movies, documents, etc.
- taffer 7y ago
  That's interesting. Some time ago I had a similar idea, but with the following differences: 3. Ownership cannot be transferred. A file remains in the same application throughout its lifecycle. 4. An application can give another application read-only access to a namespace containing one or more files. 6. An application can give another application read/write access to a namespace containing one or more files through a transaction mechanism. Do you see any advantages or disadvantages of both approaches from your point of view?
  - satyenr 7y ago
    The idea behind ownership transfer is to ensure that an application never sees another application’s namespace. Ownership transfer doesn’t really involve copying files as it does on iPadOS — it is still held at the same location on disk. The original application can just no longer see it as it no longer remains in its namespace. At the end of the operation, the second application may choose to retain or relinquish the file as needed.
titzer 7y ago
> Every piece of hardware on a computer is a piece of global mutable state. It's a tenuous analogy. DRAM, disks, caches...effectively the entire memory hierarchy is implemented as message-passing. Flip flops can be well, flipped, but those are super low-level. Almost all the rest of it requires you to send messages. It's kind of ironic that all those messages basically orchestrate the illusion of global mutable state.
- yomly 7y ago
  >It's kind of ironic that all those messages basically orchestrate the illusion of global mutable state Presumably because for certain things it was easier to reason about when things were mutable... And now we're taking this facade of mutable state and trying to add a veneer over it to make it seem immutable. It's circular :)
- dooglius 7y ago
  No, communication is implemented as message-passing (but that is largely irrelevant), but ultimately the end result is changing physical state.
- BubRoss 7y ago
  I think this is a really good point. QNX aligned much closer with this architecture as far as I can tell.
dooglius 7y ago
Doesn't POSIX already let you lock files? I don't see any reason to make it mandatory though, it not as if locking gives you a security boundary.
BubRoss 7y ago
Sounds like QNX, which we still haven't managed to get back to.

satyenr 7y ago

It doesn't have to be that way. Consider the following scenario: 1. Every file is owned by application. 2. A file-store application is the custodian of files meant to be used by multiple applications. 3. An application that needs to edit a file must take ownership of the file for the duration of the editing. 4. An application that needs to read a file must borrow the files. 5. A file can be borrowed by multiple applications, but owned by only one. 6. Applications can provide the ability for shared ownership, but those would by specialized applications capable of handling and merging simultaneous changes. This is a straw-man solution, and I am sure multiple problems will have to be solved before such a system can become a reality. > As an example, most people would want to be able to import an image into a word processor regardless of where that image is located (local drive, network drive, floppy disk, etc.). To support that, most end user programs would want to be offered access to the entire filesystem. The moment two applications do this at once, you have all the shared mutable state problems we do now. I don't see why the custodian of photos can't be a photo management application. In fact, a filesystem is the lowest common denominator. It is possible to build higher level abstractions like PhotoStore, MusicStore, MovieStore, CodeStore etc. which accommodate and make use of the properties of individual data types to offer an enhanced experience.

Retra 7y ago
I have photos on my computer, even though I don't use a photo management application. I have compressed files. I have files taken from other machines. I remove applications. I create my own file formats. Applications themselves are files. Who owns all the unassigned stuff? The file system. Give it a different name and you change nothing.
- satyenr 7y ago
  You don't use a photo management application because the file-system exposes photos as files. It is like saying -- in java parlance -- I am perfectly fine using object for everything -- I don't need any other data type. In the hypothetical system I proposed, the photo store would be exposed via an application like Explorer/Finder -- just tailored for viewing photos. Compressed files are containers for files -- they are nothing on their own. > Applications themselves are files. Sure -- they go into ApplicationStore -- which tailored for say, quick application discovery and launch. > Who owns all the unassigned stuff? T What exactly is the unassigned stuff? Files taken from other machines? I don't see why they can't be put into an appropriate store based on type. > I create my own file formats. I assume you create your own file formats because you develop your own applications? In that case, your application will be the owner of the file in question.
  - michaelmrose 7y ago
    What if I don't desire a gui application store or desire a different one or switch application stores or move a dir with executable files from one machine to another? Is the application store the only way to start at app? If I bind a global hotkey to start an app how is that handled? How about I have an addon for firefox that allows me to bind keys to operations which can include javascript which can itself start applications which can then write to files. I use this for example to open links in mpv with a singular line in my config file for the addon, tridactyl, bind V hint -W ! mpv this means show a hint on links and for the given link send the chosen link to mpv, wherein mpv will look at the link and if it can read it will use youtube-dl to download the link to a temp dir and display the video. Who owns what there? Do I have to access my photos via a a singular photo manager? I can imagine images accessed in 17 different contexts by 30 different apps does each of them need permission to access each dir which contains images or just generically to access images. Does this mean that a singular permission would control both access to the browsers cached image of the ycombinator logo on this page and some ones nude selfies? I just don't think you can cleanly map apps -> files beyond the trivial cases without making something inflexible that sucks to use.
empath75 7y ago
Sounds like rust’s borrow checker but for files.
- satyenr 7y ago
  That is where I got the idea. :-)
pixl97 7y ago
2 just sounds like reinventing the file system, now with more bugs.
- satyenr 7y ago
  No, it is not reinventing the filesystem -- it's more like a file-vault or a file-bank. Anyone looking to operate on the files must either borrow (to read) or take ownership of (to write). The difference is that while a filesystem unlimited access to all files, the app in question is meant to be more sandboxed -- handling only the files it is asked to -- e.g. shared documents that can be edited by multiple apps. Each user (on a multi-user system) will have their file-vault/file-bank and files must be explicitly transferred for shared access. Finally, as I said, it is a straw-man proposal -- I haven't spent time working out all the kinks.
  - zrm 7y ago
    > No, it is not reinventing the filesystem -- it's more like a file-vault or a file-bank. Anyone looking to operate on the files must either borrow (to read) or take ownership of (to write). So flock() and file permissions then? There are improvements that could be made here -- app-level permissions in addition to user-level permissions for example. But it's still fundamentally a filesystem.
    - satyenr 7y ago
      All of this can be implemented on top of a file system, of course. What do you think iPadOS does? We are talking about what the user/application sees and is capable of accessing.
      - zrm 7y ago
        > We are talking about what the user/application sees and is capable of accessing. But how is that different than a filesystem? Suppose we add application-based ACLs to file permissions. Then the app does open("/path/to/file", O_RDONLY) as ever. If the app has permission to the file, it gets the new fd. If it doesn't, it gets EACCES as usual. Or the OS displays a dialog asking whether the app should have permanent or one-time access to that file, and then the call doesn't return until the user chooses one. I don't see a fundamental change here. The application wouldn't necessarily even have to be modified.
amelius 7y ago
Instead of "ownership transfer" you can also use locks. Read and write locks are already supported on many filesystems.
- satyenr 7y ago
  Well, you can also use global variables safely with locks.
auggierose 7y ago
How much is that different from how it works on iPadOS?
- satyenr 7y ago
  It is meant to be more powerful and flexible than what iPad OS does, but somewhat similar.
heavenlyblue 7y ago
Your example would fail to accommodate the most basic application of files: logging and reading file logs at the same time.
- satyenr 7y ago
  Who says logs have to be written to a file — at least as far as the user is concerned? Logs are a series of events — a LogStore will let you read events as they are appended. And even if you do want to store logs in “files”, the solution proposed here here let’s you borrow them for the owning application.
  - mchristen 7y ago
    Where do you plan to store the logs then?
    - satyenr 7y ago
      On the disk, as usual. Files are, after all, an interface.
  - heavenlyblue 7y ago
    My question goes more like this: “why the hell do you want to re-approach the idea of a filesystem if the new approach can’t even accommodate something as simple as storing log files?”. It may be time for something completely different. Just stop calling it a filesystem.
    - satyenr 7y ago
      Again, why do you care about files? Logs have nothing to do with files. Files are just the interface used by filesystem to show the event stream that is logs so you can use file based tools to access the. I am not calling it a filesystem. Let’s call it a data bank?
      - heavenlyblue 7y ago
deleted 7y ago
[deleted]
LoSboccacc 7y ago
basically the windows document folder
- satyenr 7y ago
  Except, you won’t see files, just photos, movies, documents, etc.
taffer 7y ago
That's interesting. Some time ago I had a similar idea, but with the following differences: 3. Ownership cannot be transferred. A file remains in the same application throughout its lifecycle. 4. An application can give another application read-only access to a namespace containing one or more files. 6. An application can give another application read/write access to a namespace containing one or more files through a transaction mechanism. Do you see any advantages or disadvantages of both approaches from your point of view?
- satyenr 7y ago
  The idea behind ownership transfer is to ensure that an application never sees another application’s namespace. Ownership transfer doesn’t really involve copying files as it does on iPadOS — it is still held at the same location on disk. The original application can just no longer see it as it no longer remains in its namespace. At the end of the operation, the second application may choose to retain or relinquish the file as needed.

titzer 7y ago

> Every piece of hardware on a computer is a piece of global mutable state. It's a tenuous analogy. DRAM, disks, caches...effectively the entire memory hierarchy is implemented as message-passing. Flip flops can be well, flipped, but those are super low-level. Almost all the rest of it requires you to send messages. It's kind of ironic that all those messages basically orchestrate the illusion of global mutable state.

yomly 7y ago
>It's kind of ironic that all those messages basically orchestrate the illusion of global mutable state Presumably because for certain things it was easier to reason about when things were mutable... And now we're taking this facade of mutable state and trying to add a veneer over it to make it seem immutable. It's circular :)
dooglius 7y ago
No, communication is implemented as message-passing (but that is largely irrelevant), but ultimately the end result is changing physical state.
BubRoss 7y ago
I think this is a really good point. QNX aligned much closer with this architecture as far as I can tell.

dooglius 7y ago

Doesn't POSIX already let you lock files? I don't see any reason to make it mandatory though, it not as if locking gives you a security boundary.

BubRoss 7y ago

Sounds like QNX, which we still haven't managed to get back to.