12 ms·

Crazy futurist rant... Traditional operating systems such as Linux and windows are 100% dead when non volatile memory comes along in force. Paradigm shift time

Crazy futurist rant... Traditional operating systems such as Linux and windows are 100% dead when non volatile memory comes along in force. Paradigm shift time. There is pretty much no reason to use any filesystem APIs or filesystem any more. You just keep your data in the process address space - its just not going to go anywhere. Just make a data segment persistent across processes and you can survive restarts. If you backup, you can just dump the address space. Screw hard disks as well. i imagine some form of rpc will be in place between processes so they can talk to each other and that is it. Lots of small redis instances would be a similar concept. Imagine an mp3 server process which can provide persistence in the heap, metadata and decoding services and you're there. it'll be like a small internet inside your machine. Lisp would fit nicely in this world. Imagine A persistent root environment. load a defun once and it's forever. Teracotta do something similar with java. Then again I could just be insane.

guard-of-terra 14y ago
Being able to reboot is nice in case you screw up really bad. Having a transparent storage (file system) instead of opaque cross-linked in-process structures is a good thing too.
- gouranga 14y ago
  I disagree. If you use a safe programming language which doesn't piss over memory (I.e. Haskell, Python, Ruby, Lisp etc) you won't need to reboot. Just reload the broken function into your environment and carry on. As for data, the same thing can happen to your disk... The filesystem only exists because we couldn't put data in process due to the cost of memory. We cram everything through the filesystem API if it fits or not due to this. It's why our machines are slow and primitive. Consider the case of video - it is better represented as streams of audio, picture and metadata. To get this from a filesystem, we have to mux them all into a single stream and then demux them afterwards and hand them over to other APIs to process. This just wouldn't need to exist.
  - crusso 14y ago
    > Consider the case of video - it is better represented as streams of audio, picture and metadata. Not true. Video components are packaged together because they go together logically. When you want to send the video to another machine or give a copy to a friend, it's entirely logical and useful that the various components are packaged together in some way.
    - gouranga 14y ago
      I'm talking about storage, not distribution. They are two different things.
      - crusso 14y ago
        > Traditional operating systems such as Linux and windows are 100% dead when non volatile memory comes along in force Seems like you were talking about a lot more than just one particular facet of filesystems when you made the futurist proclamation above.
        gouranga 14y ago
        Yes I admit that but for the scope of discussion, it's rather hard to separate the two. My idea is that communication should be transparent.
  - heretohelp 14y ago
    >If you use a safe programming language which doesn't piss over memory (I.e. Haskell, Python, Ruby, Lisp etc) you won't need to reboot. This is a man who's never used Xmonad (which is written in Haskell) for an extended period of time before. If you use it for a long enough time, it gets slower and slower until workspace changes start taking whole seconds. Eventually you say fuck it and reboot... ...'cept you can't do that without a filesystem with a "base state" to reboot from. GG
    - gouranga 14y ago
- lubutu 14y ago
  If all the non-volatile in-process data structures have the same semantics, with type annotations etc, one can imagine those structures being just as transparent and ready for inspection as modern filesystems are.
  - quotemstr 14y ago
    > one can imagine those structures being just as transparent and ready for inspection as modern filesystems Then why not use filesystem APIs to talk about these structures? There's nothing about opendir() that requires it be backed by a block device instead of nvram.
  - guard-of-terra 14y ago
    Modern file systems are not very transparent, but any process guts are much worse. My debugging experience suggests so. Files are data and structures are part code. You can't do much with code 'cause of turing completeness.
ajross 14y ago
I'll buy the crazy part. The "100% dead" bit loses me right off the bat. Surely there will be space for hardware abstraction, process models, memory protection mechanisms, networking, etc... in your futurist OS. And, amusingly, that code is already there in the OSes of today! Storage management is merely a big subsystem. And as for dropping "files", I think that's missing the point. Files are just pickled streams, and there will always be stream metaphors in computing. How else do you represent "inputs" to your program (which, remember, still has to run "from scratch" most of the time, if only on test sets)? I guess I see this as a much narrower optimization. A computer built out of perfectly transparent NVRAM storage is merely one with zero-time suspend/restore and zero suspended power draw. That doesn't sound so paradigm breaking to me, though it sure would be nice to have.
- gouranga 14y ago
  the stream metaphore isn't appropriate for all types of data. memory allows a storage paradigm to be picked for the task at hand. Consider redis which is a great example of this. How do you store redis' data efficiently? Well it turns that aof files are slow to start up and disk backed virtual memory is slow. The problem goes away instantly with NVM - the jib is done with no filesystem api used.
  - ajross 14y ago
    I dare say that the stream metaphor is a better fit for more types of data than Redis is. To first approximation, all data in the modern world is video files. You really want to store those in a raw memory space?
    - gouranga 14y ago
      That's ok until you need to read hundreds of multiple streams from a stream device. You end up with random access then at which point the stream paradigm breaks down and you have to use memory.
    - Cushman 14y ago
      Filmmakers would kill for that ability.
crusso 14y ago
> There is pretty much no reason to use any filesystem APIs or filesystem any more Don't agree. Filesystems are used more for organizational purposes than anything. Most of my folders are logically named and organized rather than physically based because of the types they contain. Probably the filesystem API will evolve into something more tag-based than hierarchical, but you never know. The filesystem hierarchical folder metaphor is re-used in tons of places where there's really no need for it physically. Why? Because there's a logical need for it. Additionally, file system APIs provide lots of useful abstractions such as opening resources, closing them, reading to, writing from, appending to, etc. When you go to do your backup in process space, which data is in a self-consistent state that can be copied? You'll need something like a filesystem API to coordinate.
- gouranga 14y ago
  Tagging works on a heap. Heirarchies are terribly inefficient for categorising information. The metadata is more important than the file location. Look at the way music is stored on smartphones. Ever heard of STM?
  - crusso 14y ago
    > The metadata is more important than the file location. The web is a great example of a system where there's no particular need to store things like you do in a filesystem. The data for most sites is stored in various SQL and NOSQL databases... yet we still predominately see hierarchical paths used for resource identification. I wonder why?
    - gouranga 14y ago
      Hmm like this? http://news.ycombinator.com/reply?id=4044891 http://news.ycombinator.com/reply?id=4044891 Rather than: http://news.ycombinator.com/2012/05/30/non-volatile-memory/gouranga:2/xyz:3/reply http://news.ycombinator.com/2012/05/30/non-volatile-memory/g... Ooh it even wrapped.
      - 14y ago
Splines 14y ago
> Traditional operating systems such as Linux and windows are 100% dead when non volatile memory comes along in force. Paradigm shift time. I doubt they'd be dead, they just won't be able to do the new things that NVM would allow you to do (at first, anyway). At a minimum, mobile battery life could be better, depending on how much power is usually used to keep things going. Get an event from a radio or button and the system can instantaneously wake up. The magic trick that the NDS does when you close its lid will become universal.
batista 14y ago
>Traditional operating systems such as Linux and windows are 100% dead when non volatile memory comes along in force. Paradigm shift time. There is pretty much no reason to use any filesystem APIs or filesystem any more. You just keep your data in the process address space - its just not going to go anywhere. Yes, only we want to have those data shared among many processes, with abstractions, names, rich metadata etc. (Not to mention different machines, backups, etc). It's not like we are currently forced to use filesystems because memory is volatile. Actually, you got it backwards, we use filesystems specifically in NON VOLATILE memory, that is hard disks.
Detrus 14y ago
This could be two decades away. NVM will first be introduced with limiting amounts of storage so there will still be other hardware. Hopefully current OSs will be re-imagined anyway just due to the vast time difference.
- gouranga 14y ago
  I agree about it being a way off but I think more 5 years than 20. Look at the adoption rate of SSDs
  - Detrus 14y ago
    SSDs haven't replaced spinning HDs yet. And probably won't for slow read but massive storage. They handle the functionality of general use with 300GB HDs but that's some ways off current multi-terabyte spinning disks. And those might increase storage capacity faster than SSDs. So server OSs will have to maintain compatibility. And NVM tech will probably be more limiting in storage to current RAM than SSDs are to spinning HDs. To replace even 30% of RAM/Storage in 5 years seems like a stretch.
    - gouranga 14y ago
      Slight modification... HDs haven't replaced tapes yet. And probably won't for slow read but massive storage. They handle the functionality of general use with 20GB tapes but that's some ways off current multi-terabyte tapes. And those might increase storage capacity faster than HDs. So server OSs will have to maintain compatibility.
sliverstorm 14y ago
Traditional operating systems such as Linux and windows are 100% dead when non volatile memory comes along in force. This implies there is something rearing at the starting gate to replace them.
- gouranga 14y ago
  There will be and I'm going to jolly well be the person writing it :) (I'm going to try - 10 years of embedded followed by 10 years of business facing is a good foundation and I've spent the last 15 years on the problem in my mind waiting for the technology to arrive)
marshray 14y ago
> There is pretty much no reason to use any filesystem APIs or filesystem any more. You just keep your data in the process address space The Windows NT kernel is primarily a filesystem-backed address space for committed RAM. Originally you actually had to have a pagefile at least as large as physical RAM. Except for nonpageable kernel structures, all the program accessible RAM was part of a memory-mapped file. [[EDIT: There's plenty of text from Microsoft that implies this, (e.g. "you should set the size of the paging file to the same size as your system RAM plus 12 MB...because Windows NT requires "backing storage" for everything it keeps in RAM") however, offline discussions have convinced me that it was never strictly true.]] This was back when drive capacity was "more larger" than RAM capacity and disk bandwith was "less slower". The kernel has evolved away from this design a bit, but it did bring a certain purity. For example the filesytem cache and virtual memory paging system could be largely the same thing. > You just keep your data in the process address space - its just not going to go anywhere. Just make a data segment persistent across processes and you can survive restarts. This more or less what happens when the kernel bluescreens and the page file was at least as large as RAM. It makes debugging kernel crashes easier. (Spare me the Windows jokes please I'm not advocating for it, just saying this part of it that no one ever sees had a relatively elegant design opposite to what crazy futurist rant suggests.)
jws 14y ago
One might think about the paradigm shift when hard disks became so cheap that tape drives no longer made sense. • Tape drives still make sense, to some people that store insanely large data sets with extremely infrequent (or never) access. • I do not mourn my tape based backups for a single instant when using my rsync based backups. Life is good. • Despite tape being dead for most of us, we all know a program called "tar". We still make tape archives, just on other storage. The paradigm survives on virtual tapes because it is a useful cognitive model. Sure I could make a block file be a virtual disk and put a filesystem in it and send you the files that way, but you'd rather have a good old "tape" archive. Likewise, disk filesystems are not going to go away if disks go away. They are too useful for reasoning about problems. Scratch the surface of an iPad. It is full of files, yet empty of disks. Go to Linux, land of speciation, try to find a persistent storage system for flash memory which does not treat it as a disk. You will find some filesystems optimized for flash, but you will come up (nearly) lacking for a completely new way of looking at storage. Persistent full speed RAM should be enough of a change to spawn that new thing, but I'll bet people keep the "real" copy in a filesystem for a long time. When that alpha particle corrupts a bit and trashes your clever RAM based data structure, what are you going to do? I'll reload from my file.
aiscott 14y ago
Personally I don't like the idea of "image" based computing very much. I mean, it's fine in a lot of circumstances, but the file system provides a nice broadly compatible database that is consistent across programs. I think the need for such a well defined and accepted user organizable data store isn't going away regardless of the underlying storage medium. I like that my file system can be reasoned about in very concrete ways. I'm okay with using a tag based system like gmail, but I find it very flat and more difficult to organize vs traditional hierarchy of folders. And btw, GET OFF MY LAWN!
jpadkins 14y ago
in this world, how do you programs on different computers reference the same piece of data? ipv6 address/memory address? the whole path/file thing is really useful for distributed computing. Also what would users use for object naming(especially across address spaces)? How do I search for a presentation or spreadsheet without a reference to it?
- gouranga 14y ago
  a) global address space. b) no it's not. Distributed computing is normally message based. c) you use an index in (a) and pull it into your local scope.
  - jpadkins 14y ago
    a) global address space for all computer data for all time? how big does that need to be? b)yes, but in those messages are references to data. How do you reference that data across computer address spaces? Right now, we use an URI that is based on host/path/resource semantics. How do you get path/resource without a filestystem like construct? c) I'm not following you.
    - gouranga 14y ago
      a) Large, sorry HUGE. Current CPUs cannot address it. 128 bit is probably enough to encode most of the universe and not insurmountable bit width. PAXOS consensus algorithm + hybrid virtual memory system for management. Literally a GLOBAL P2P heap. Your CPU resides in a unique section of it. b) The above solves that. You just read an address and it pulls that block over the network into your local address space. You write to it, and it pushes it back. (STM comes in here as you can semantically wrap such things in transactions). c) Someone gives you a pointer to the root of a catalogue or there is one built in then you can navigate the data structure be it a full text index or linear linked list to find the data you need. There is no filesystem. I think someone related my ideas to AS/400 which TBH after doing some reading is a pretty good comparison, although I'd do it on a larger scale. This is really "fringe" computer science if you want to call it that. It's pushing thing boundaries of what is possible intentionally.
snprbob86 14y ago
This seems like one of those "sounds good on paper" ideas, like shells that pipe objects. In practice, file and process oriented systems are separate from REPL systems on purpose. The most important property of a filesystem or the Unix-style pipes&filters that permeate them is the fact that serialization is a fundamental property component of persistence and communication. Any finite quantity of space can be addressed by a linear scheme and any unbounded sequence is inherently linear. Consider what happens as soon as you want to send a stream of data across the wire. Or checkpoint an operation. You need to store stuff in a non-volatile and linear form. And since data outlives code, you better invest time into thinking about that linear form. Now, if you use immutable data structures (ie. non-cyclical) with a proper linearization and corresponding reader/printer (like in Clojure), you can get many of the same benefits, but you still need to have a whole bunch of other things to worry about. Just look at the weird things that Clojure needs to do with print-dup and the like. I'm not saying that there isn't a kernel of a good idea in here. I'm just saying that it's going to have a lot more in common with filesystems and traditional shells than you'd first expect coming from a Lisp REPL school of thought.
- gouranga 14y ago
  > shells that pipe objects Powershell rocks! No wait, sorry it doesn't. I agree!
abecedarius 14y ago
http://www.cis.upenn.edu/~KeyKOS/ http://www.cis.upenn.edu/~KeyKOS/ looked pretty much like that. The implementation treated RAM as a cache of the disk, IIRC. This seems like a better way to live than Unix, to me, but you can't say it hasn't been tried. See also http://en.wikipedia.org/wiki/Single-level_store http://en.wikipedia.org/wiki/Single-level_store
- gouranga 14y ago
  That's exactly the sort of thing I'm thinking.
bigiain 14y ago
Hmmm, I'm remembering back to the Apple Newton's "soup" memory… (You don't want each process to have exclusive access to it's own data, sometimes you want different processes to access the same data)
rwl4 14y ago
A descendent of Smalltalk seems like it might fit the bill here..
ricardobeat 14y ago
Is it possible that the switch to no filesystem in iOS userspace (and probably OSX in the future) is in anticipation to this in some degree? <tinfoil hat on>
gaius 14y ago
Crazy futurist rant... Haha, this is exactly how the IBM AS/400 works! http://en.wikipedia.org/wiki/Single_level_store http://en.wikipedia.org/wiki/Single_level_store
- gouranga 14y ago
  thanks for the pointer. I now to you IBM (but not the COBOL bit)
- deleted 14y ago
  [deleted]

guard-of-terra 14y ago

Being able to reboot is nice in case you screw up really bad. Having a transparent storage (file system) instead of opaque cross-linked in-process structures is a good thing too.

gouranga 14y ago
I disagree. If you use a safe programming language which doesn't piss over memory (I.e. Haskell, Python, Ruby, Lisp etc) you won't need to reboot. Just reload the broken function into your environment and carry on. As for data, the same thing can happen to your disk... The filesystem only exists because we couldn't put data in process due to the cost of memory. We cram everything through the filesystem API if it fits or not due to this. It's why our machines are slow and primitive. Consider the case of video - it is better represented as streams of audio, picture and metadata. To get this from a filesystem, we have to mux them all into a single stream and then demux them afterwards and hand them over to other APIs to process. This just wouldn't need to exist.
- crusso 14y ago
  > Consider the case of video - it is better represented as streams of audio, picture and metadata. Not true. Video components are packaged together because they go together logically. When you want to send the video to another machine or give a copy to a friend, it's entirely logical and useful that the various components are packaged together in some way.
  - gouranga 14y ago
    I'm talking about storage, not distribution. They are two different things.
    - crusso 14y ago
      > Traditional operating systems such as Linux and windows are 100% dead when non volatile memory comes along in force Seems like you were talking about a lot more than just one particular facet of filesystems when you made the futurist proclamation above.
      - gouranga 14y ago
        Yes I admit that but for the scope of discussion, it's rather hard to separate the two. My idea is that communication should be transparent.
- heretohelp 14y ago
  >If you use a safe programming language which doesn't piss over memory (I.e. Haskell, Python, Ruby, Lisp etc) you won't need to reboot. This is a man who's never used Xmonad (which is written in Haskell) for an extended period of time before. If you use it for a long enough time, it gets slower and slower until workspace changes start taking whole seconds. Eventually you say fuck it and reboot... ...'cept you can't do that without a filesystem with a "base state" to reboot from. GG
  - gouranga 14y ago
lubutu 14y ago
If all the non-volatile in-process data structures have the same semantics, with type annotations etc, one can imagine those structures being just as transparent and ready for inspection as modern filesystems are.
- quotemstr 14y ago
  > one can imagine those structures being just as transparent and ready for inspection as modern filesystems Then why not use filesystem APIs to talk about these structures? There's nothing about opendir() that requires it be backed by a block device instead of nvram.
- guard-of-terra 14y ago
  Modern file systems are not very transparent, but any process guts are much worse. My debugging experience suggests so. Files are data and structures are part code. You can't do much with code 'cause of turing completeness.

ajross 14y ago

I'll buy the crazy part. The "100% dead" bit loses me right off the bat. Surely there will be space for hardware abstraction, process models, memory protection mechanisms, networking, etc... in your futurist OS. And, amusingly, that code is already there in the OSes of today! Storage management is merely a big subsystem. And as for dropping "files", I think that's missing the point. Files are just pickled streams, and there will always be stream metaphors in computing. How else do you represent "inputs" to your program (which, remember, still has to run "from scratch" most of the time, if only on test sets)? I guess I see this as a much narrower optimization. A computer built out of perfectly transparent NVRAM storage is merely one with zero-time suspend/restore and zero suspended power draw. That doesn't sound so paradigm breaking to me, though it sure would be nice to have.

gouranga 14y ago
the stream metaphore isn't appropriate for all types of data. memory allows a storage paradigm to be picked for the task at hand. Consider redis which is a great example of this. How do you store redis' data efficiently? Well it turns that aof files are slow to start up and disk backed virtual memory is slow. The problem goes away instantly with NVM - the jib is done with no filesystem api used.
- ajross 14y ago
  I dare say that the stream metaphor is a better fit for more types of data than Redis is. To first approximation, all data in the modern world is video files. You really want to store those in a raw memory space?
  - gouranga 14y ago
    That's ok until you need to read hundreds of multiple streams from a stream device. You end up with random access then at which point the stream paradigm breaks down and you have to use memory.
  - Cushman 14y ago
    Filmmakers would kill for that ability.

crusso 14y ago

> There is pretty much no reason to use any filesystem APIs or filesystem any more Don't agree. Filesystems are used more for organizational purposes than anything. Most of my folders are logically named and organized rather than physically based because of the types they contain. Probably the filesystem API will evolve into something more tag-based than hierarchical, but you never know. The filesystem hierarchical folder metaphor is re-used in tons of places where there's really no need for it physically. Why? Because there's a logical need for it. Additionally, file system APIs provide lots of useful abstractions such as opening resources, closing them, reading to, writing from, appending to, etc. When you go to do your backup in process space, which data is in a self-consistent state that can be copied? You'll need something like a filesystem API to coordinate.

gouranga 14y ago
Tagging works on a heap. Heirarchies are terribly inefficient for categorising information. The metadata is more important than the file location. Look at the way music is stored on smartphones. Ever heard of STM?
- crusso 14y ago
  > The metadata is more important than the file location. The web is a great example of a system where there's no particular need to store things like you do in a filesystem. The data for most sites is stored in various SQL and NOSQL databases... yet we still predominately see hierarchical paths used for resource identification. I wonder why?
  - gouranga 14y ago
    Hmm like this? http://news.ycombinator.com/reply?id=4044891 http://news.ycombinator.com/reply?id=4044891 Rather than: http://news.ycombinator.com/2012/05/30/non-volatile-memory/gouranga:2/xyz:3/reply http://news.ycombinator.com/2012/05/30/non-volatile-memory/g... Ooh it even wrapped.
    - 14y ago

Splines 14y ago

> Traditional operating systems such as Linux and windows are 100% dead when non volatile memory comes along in force. Paradigm shift time. I doubt they'd be dead, they just won't be able to do the new things that NVM would allow you to do (at first, anyway). At a minimum, mobile battery life could be better, depending on how much power is usually used to keep things going. Get an event from a radio or button and the system can instantaneously wake up. The magic trick that the NDS does when you close its lid will become universal.

batista 14y ago

>Traditional operating systems such as Linux and windows are 100% dead when non volatile memory comes along in force. Paradigm shift time. There is pretty much no reason to use any filesystem APIs or filesystem any more. You just keep your data in the process address space - its just not going to go anywhere. Yes, only we want to have those data shared among many processes, with abstractions, names, rich metadata etc. (Not to mention different machines, backups, etc). It's not like we are currently forced to use filesystems because memory is volatile. Actually, you got it backwards, we use filesystems specifically in NON VOLATILE memory, that is hard disks.

Detrus 14y ago

This could be two decades away. NVM will first be introduced with limiting amounts of storage so there will still be other hardware. Hopefully current OSs will be re-imagined anyway just due to the vast time difference.

gouranga 14y ago
I agree about it being a way off but I think more 5 years than 20. Look at the adoption rate of SSDs
- Detrus 14y ago
  SSDs haven't replaced spinning HDs yet. And probably won't for slow read but massive storage. They handle the functionality of general use with 300GB HDs but that's some ways off current multi-terabyte spinning disks. And those might increase storage capacity faster than SSDs. So server OSs will have to maintain compatibility. And NVM tech will probably be more limiting in storage to current RAM than SSDs are to spinning HDs. To replace even 30% of RAM/Storage in 5 years seems like a stretch.
  - gouranga 14y ago
    Slight modification... HDs haven't replaced tapes yet. And probably won't for slow read but massive storage. They handle the functionality of general use with 20GB tapes but that's some ways off current multi-terabyte tapes. And those might increase storage capacity faster than HDs. So server OSs will have to maintain compatibility.

sliverstorm 14y ago

Traditional operating systems such as Linux and windows are 100% dead when non volatile memory comes along in force. This implies there is something rearing at the starting gate to replace them.

gouranga 14y ago
There will be and I'm going to jolly well be the person writing it :) (I'm going to try - 10 years of embedded followed by 10 years of business facing is a good foundation and I've spent the last 15 years on the problem in my mind waiting for the technology to arrive)

marshray 14y ago

> There is pretty much no reason to use any filesystem APIs or filesystem any more. You just keep your data in the process address space The Windows NT kernel is primarily a filesystem-backed address space for committed RAM. Originally you actually had to have a pagefile at least as large as physical RAM. Except for nonpageable kernel structures, all the program accessible RAM was part of a memory-mapped file. [[EDIT: There's plenty of text from Microsoft that implies this, (e.g. "you should set the size of the paging file to the same size as your system RAM plus 12 MB...because Windows NT requires "backing storage" for everything it keeps in RAM") however, offline discussions have convinced me that it was never strictly true.]] This was back when drive capacity was "more larger" than RAM capacity and disk bandwith was "less slower". The kernel has evolved away from this design a bit, but it did bring a certain purity. For example the filesytem cache and virtual memory paging system could be largely the same thing. > You just keep your data in the process address space - its just not going to go anywhere. Just make a data segment persistent across processes and you can survive restarts. This more or less what happens when the kernel bluescreens and the page file was at least as large as RAM. It makes debugging kernel crashes easier. (Spare me the Windows jokes please I'm not advocating for it, just saying this part of it that no one ever sees had a relatively elegant design opposite to what crazy futurist rant suggests.)

jws 14y ago

One might think about the paradigm shift when hard disks became so cheap that tape drives no longer made sense. • Tape drives still make sense, to some people that store insanely large data sets with extremely infrequent (or never) access. • I do not mourn my tape based backups for a single instant when using my rsync based backups. Life is good. • Despite tape being dead for most of us, we all know a program called "tar". We still make tape archives, just on other storage. The paradigm survives on virtual tapes because it is a useful cognitive model. Sure I could make a block file be a virtual disk and put a filesystem in it and send you the files that way, but you'd rather have a good old "tape" archive. Likewise, disk filesystems are not going to go away if disks go away. They are too useful for reasoning about problems. Scratch the surface of an iPad. It is full of files, yet empty of disks. Go to Linux, land of speciation, try to find a persistent storage system for flash memory which does not treat it as a disk. You will find some filesystems optimized for flash, but you will come up (nearly) lacking for a completely new way of looking at storage. Persistent full speed RAM should be enough of a change to spawn that new thing, but I'll bet people keep the "real" copy in a filesystem for a long time. When that alpha particle corrupts a bit and trashes your clever RAM based data structure, what are you going to do? I'll reload from my file.

aiscott 14y ago

Personally I don't like the idea of "image" based computing very much. I mean, it's fine in a lot of circumstances, but the file system provides a nice broadly compatible database that is consistent across programs. I think the need for such a well defined and accepted user organizable data store isn't going away regardless of the underlying storage medium. I like that my file system can be reasoned about in very concrete ways. I'm okay with using a tag based system like gmail, but I find it very flat and more difficult to organize vs traditional hierarchy of folders. And btw, GET OFF MY LAWN!

jpadkins 14y ago

in this world, how do you programs on different computers reference the same piece of data? ipv6 address/memory address? the whole path/file thing is really useful for distributed computing. Also what would users use for object naming(especially across address spaces)? How do I search for a presentation or spreadsheet without a reference to it?

gouranga 14y ago
a) global address space. b) no it's not. Distributed computing is normally message based. c) you use an index in (a) and pull it into your local scope.
- jpadkins 14y ago
  a) global address space for all computer data for all time? how big does that need to be? b)yes, but in those messages are references to data. How do you reference that data across computer address spaces? Right now, we use an URI that is based on host/path/resource semantics. How do you get path/resource without a filestystem like construct? c) I'm not following you.
  - gouranga 14y ago
    a) Large, sorry HUGE. Current CPUs cannot address it. 128 bit is probably enough to encode most of the universe and not insurmountable bit width. PAXOS consensus algorithm + hybrid virtual memory system for management. Literally a GLOBAL P2P heap. Your CPU resides in a unique section of it. b) The above solves that. You just read an address and it pulls that block over the network into your local address space. You write to it, and it pushes it back. (STM comes in here as you can semantically wrap such things in transactions). c) Someone gives you a pointer to the root of a catalogue or there is one built in then you can navigate the data structure be it a full text index or linear linked list to find the data you need. There is no filesystem. I think someone related my ideas to AS/400 which TBH after doing some reading is a pretty good comparison, although I'd do it on a larger scale. This is really "fringe" computer science if you want to call it that. It's pushing thing boundaries of what is possible intentionally.

snprbob86 14y ago

This seems like one of those "sounds good on paper" ideas, like shells that pipe objects. In practice, file and process oriented systems are separate from REPL systems on purpose. The most important property of a filesystem or the Unix-style pipes&filters that permeate them is the fact that serialization is a fundamental property component of persistence and communication. Any finite quantity of space can be addressed by a linear scheme and any unbounded sequence is inherently linear. Consider what happens as soon as you want to send a stream of data across the wire. Or checkpoint an operation. You need to store stuff in a non-volatile and linear form. And since data outlives code, you better invest time into thinking about that linear form. Now, if you use immutable data structures (ie. non-cyclical) with a proper linearization and corresponding reader/printer (like in Clojure), you can get many of the same benefits, but you still need to have a whole bunch of other things to worry about. Just look at the weird things that Clojure needs to do with print-dup and the like. I'm not saying that there isn't a kernel of a good idea in here. I'm just saying that it's going to have a lot more in common with filesystems and traditional shells than you'd first expect coming from a Lisp REPL school of thought.

gouranga 14y ago
> shells that pipe objects Powershell rocks! No wait, sorry it doesn't. I agree!

abecedarius 14y ago

http://www.cis.upenn.edu/~KeyKOS/ http://www.cis.upenn.edu/~KeyKOS/ looked pretty much like that. The implementation treated RAM as a cache of the disk, IIRC. This seems like a better way to live than Unix, to me, but you can't say it hasn't been tried. See also http://en.wikipedia.org/wiki/Single-level_store http://en.wikipedia.org/wiki/Single-level_store

gouranga 14y ago
That's exactly the sort of thing I'm thinking.

bigiain 14y ago

Hmmm, I'm remembering back to the Apple Newton's "soup" memory… (You don't want each process to have exclusive access to it's own data, sometimes you want different processes to access the same data)

rwl4 14y ago

A descendent of Smalltalk seems like it might fit the bill here..

ricardobeat 14y ago

Is it possible that the switch to no filesystem in iOS userspace (and probably OSX in the future) is in anticipation to this in some degree? <tinfoil hat on>

gaius 14y ago

Crazy futurist rant... Haha, this is exactly how the IBM AS/400 works! http://en.wikipedia.org/wiki/Single_level_store http://en.wikipedia.org/wiki/Single_level_store

gouranga 14y ago
thanks for the pointer. I now to you IBM (but not the COBOL bit)
deleted 14y ago
[deleted]