3 ms·

I applaud the drive and vision to experiment with a fundamentally different operating system paradigm. But even if your persistent storage is RAM, I don't see

I applaud the drive and vision to experiment with a fundamentally different operating system paradigm. But even if your persistent storage is RAM, I don't see why you wouldn't still want a filesystem. Filesystems arose as a method of organizing data. However fast your access time, you still want that. Maybe you could have some kind of database backed tagging system instead of a directory hierarchy, but the benefits and drawbacks of such a system should apply to an SSD or even HDD as much as to something in RAM. This goes for user data (eg. photos, documents) as well the organization of the operating system. Anyway, I'm curious to see what what this project comes up with. I suspect it will be something that looks and functions a lot like a filesystem, but I'd be happy to be proven wrong.

karmakaze 6y ago
Advances in performance are largely about impedance matching, from sequential access tapes to registers in a CPU. You want to use an efficient interface and abstraction for the level you're at. The direction of progress is toward faster, more randomly accessible with less seek cost: HDD -> SSD -> XPoint, etc. This totally makes sense--using RAM through a disk interface (e.g. RAM disk) is useful if you want a faster disk, to get something faster change the interface to take advantage of the medium.
hderms 6y ago
I think the main benefits would be rethinking the core of persistence to be more about atomic operations on datastructures as opposed to POSIX filesystem IO. I personally believe that files are still a useful metaphor, and this hypothetical OS that we're discussing would likely still provide them, at least for interop. An example being a database. Provided we didn't have to batch writes, be aware of dirty pages, write things in certain order to maintain coherence in the event of a crash, etc... we could build something really simple. Ultimately it would depend a lot on the details like memory bandwidth+latency between DRAM and NVRAM
blackrock 6y ago
Even AWS S3 is a key-value storage system. Meaning your key can be anything. But over time, people tend to name the key using a path structure, separated by forward slashes, to give a logical separation of data into subdirectories.
guidoism 6y ago
I see what you are saying but I think it misses the point. We need a filesystem right now because data disappears from RAM when electricity goes away. With persistent memory you don't need that extra serialization/deserialization step to/from the filesystem. Look, think it like this: 1. In memory we store our program code and data as blobs in various parts, in the stack, in a heap, some only for the current program, other stuff like shared libraries linked at runtime to various programs. It's a rats nest of pointers. 2. When we "save" or "restore" we takes that rats nest and pull out some of the important parts (e.g. we ignore the shared libraries in memory and the app code) and we serialize that state out to disk. 3. Then the file system takes that logically contiguous "file" and breaks it up into pieces and stores those across the disk in various blocks and writes metadata which describe how to put the pieces back together again. With persistent memory we can ignore steps 2 and 3 most of the time, but they still might be useful for sending data to other computers, especially step 2. Disclaimer: I read the slides but didn't watch the talk
- pmontra 6y ago
  Sending data to other programs in the same computer too. Example: 1. Download pictures from camera. You need a program to transfer them somewhere you can find them. 2. Choose 12 of them to make a calendar for next year (OK, let's assume we're already in November). You need a program to view them. 3. Edit them with your favorite. Save as copies, not to overwrite the originals. 4. Upload them to a web service using a browser. A file system is useful to support all of these steps. Android's share to other apps kind of hides the file system but it doesn't succeed 100%. Sometimes no app registered to handle some file type and we end up picking the file in a file explorer interface. Sometimes we even have to know where that app stores its files. My most usual case for this are the gpx files of OsmAnd, when I transfer them from the small phone I use as tracker to my main phone.
  - 0xCMP 6y ago
    I'm very interested in getting rid of "Files" and making programs use/manipulate databases more, but the issue of sharing things has always been a problem. Like you say in Android, but also iOS, it's annoying. However, you've given me the idea of what if instead of registering to handle a file type and giving the user a way to select a file we made sharing better by making applications provide a kind of "Files" view in to their data which may simply be items in their databases. On iOS the app ShellFish provides a seamless view to the files on my servers inside of the Files app and the Files picker of the OS. In macOS the finder's file picker has a section for Photos which lets you select a photo without needing to export it to a specific file. We could instead select from applications on our system instead of going to each application and sharing each piece of data we need.

karmakaze 6y ago

Advances in performance are largely about impedance matching, from sequential access tapes to registers in a CPU. You want to use an efficient interface and abstraction for the level you're at. The direction of progress is toward faster, more randomly accessible with less seek cost: HDD -> SSD -> XPoint, etc. This totally makes sense--using RAM through a disk interface (e.g. RAM disk) is useful if you want a faster disk, to get something faster change the interface to take advantage of the medium.

hderms 6y ago

I think the main benefits would be rethinking the core of persistence to be more about atomic operations on datastructures as opposed to POSIX filesystem IO. I personally believe that files are still a useful metaphor, and this hypothetical OS that we're discussing would likely still provide them, at least for interop. An example being a database. Provided we didn't have to batch writes, be aware of dirty pages, write things in certain order to maintain coherence in the event of a crash, etc... we could build something really simple. Ultimately it would depend a lot on the details like memory bandwidth+latency between DRAM and NVRAM

blackrock 6y ago

Even AWS S3 is a key-value storage system. Meaning your key can be anything. But over time, people tend to name the key using a path structure, separated by forward slashes, to give a logical separation of data into subdirectories.

guidoism 6y ago

I see what you are saying but I think it misses the point. We need a filesystem right now because data disappears from RAM when electricity goes away. With persistent memory you don't need that extra serialization/deserialization step to/from the filesystem. Look, think it like this: 1. In memory we store our program code and data as blobs in various parts, in the stack, in a heap, some only for the current program, other stuff like shared libraries linked at runtime to various programs. It's a rats nest of pointers. 2. When we "save" or "restore" we takes that rats nest and pull out some of the important parts (e.g. we ignore the shared libraries in memory and the app code) and we serialize that state out to disk. 3. Then the file system takes that logically contiguous "file" and breaks it up into pieces and stores those across the disk in various blocks and writes metadata which describe how to put the pieces back together again. With persistent memory we can ignore steps 2 and 3 most of the time, but they still might be useful for sending data to other computers, especially step 2. Disclaimer: I read the slides but didn't watch the talk

pmontra 6y ago
Sending data to other programs in the same computer too. Example: 1. Download pictures from camera. You need a program to transfer them somewhere you can find them. 2. Choose 12 of them to make a calendar for next year (OK, let's assume we're already in November). You need a program to view them. 3. Edit them with your favorite. Save as copies, not to overwrite the originals. 4. Upload them to a web service using a browser. A file system is useful to support all of these steps. Android's share to other apps kind of hides the file system but it doesn't succeed 100%. Sometimes no app registered to handle some file type and we end up picking the file in a file explorer interface. Sometimes we even have to know where that app stores its files. My most usual case for this are the gpx files of OsmAnd, when I transfer them from the small phone I use as tracker to my main phone.
- 0xCMP 6y ago
  I'm very interested in getting rid of "Files" and making programs use/manipulate databases more, but the issue of sharing things has always been a problem. Like you say in Android, but also iOS, it's annoying. However, you've given me the idea of what if instead of registering to handle a file type and giving the user a way to select a file we made sharing better by making applications provide a kind of "Files" view in to their data which may simply be items in their databases. On iOS the app ShellFish provides a seamless view to the files on my servers inside of the Files app and the Files picker of the OS. In macOS the finder's file picker has a section for Photos which lets you select a photo without needing to export it to a specific file. We could instead select from applications on our system instead of going to each application and sharing each piece of data we need.